From ipfix-bounces@ietf.org Fri Aug 1 03:14:19 2008 Return-Path: X-Original-To: ipfix-archive@optimus.ietf.org Delivered-To: ietfarch-ipfix-archive@core3.amsl.com Received: from [127.0.0.1] (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 105013A6D96; Fri, 1 Aug 2008 03:14:19 -0700 (PDT) X-Original-To: ipfix@core3.amsl.com Delivered-To: ipfix@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 748A93A6D96 for ; Fri, 1 Aug 2008 03:14:18 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.26 X-Spam-Level: X-Spam-Status: No, score=-6.26 tagged_above=-999 required=5 tests=[AWL=0.339, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id I4RizWX62+J5 for ; Fri, 1 Aug 2008 03:14:17 -0700 (PDT) Received: from ams-iport-1.cisco.com (ams-iport-1.cisco.com [144.254.224.140]) by core3.amsl.com (Postfix) with ESMTP id 3EED93A6897 for ; Fri, 1 Aug 2008 03:14:17 -0700 (PDT) X-IronPort-AV: E=Sophos;i="4.31,292,1215388800"; d="scan'208";a="15990508" Received: from ams-dkim-1.cisco.com ([144.254.224.138]) by ams-iport-1.cisco.com with ESMTP; 01 Aug 2008 10:14:05 +0000 Received: from ams-core-1.cisco.com (ams-core-1.cisco.com [144.254.224.150]) by ams-dkim-1.cisco.com (8.12.11/8.12.11) with ESMTP id m71AE5CE012235; Fri, 1 Aug 2008 12:14:05 +0200 Received: from cisco.com (mrwint.cisco.com [64.103.71.48]) by ams-core-1.cisco.com (8.13.8/8.13.8) with ESMTP id m71AE5Cq005907; Fri, 1 Aug 2008 10:14:05 GMT Received: from [10.61.80.75] (ams3-vpn-dhcp4172.cisco.com [10.61.80.75]) by cisco.com (8.11.7p3+Sun/8.8.8) with ESMTP id m71AE4J09646; Fri, 1 Aug 2008 11:14:04 +0100 (BST) Message-ID: <4892E1EE.1060806@cisco.com> Date: Fri, 01 Aug 2008 11:14:06 +0100 From: Paul Aitken User-Agent: Mozilla/5.0 (X11; U; Linux i686 (x86_64); en-GB; rv:1.8.1.16) Gecko/20080702 SeaMonkey/1.1.11 MIME-Version: 1.0 To: Juergen Quittek References: In-Reply-To: DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; l=836; t=1217585645; x=1218449645; c=relaxed/simple; s=amsdkim1002; h=Content-Type:From:Subject:Content-Transfer-Encoding:MIME-Version; d=cisco.com; i=paitken@cisco.com; z=From:=20Paul=20Aitken=20 |Subject:=20Re=3A=20[IPFIX]=20WG=20last=20call=20on=20draft -ietf-ipfix-file-02 |Sender:=20; bh=m3TrgatbLrjQF9NPJ4L02TsU+3S/W55s7Lo7UndeRis=; b=qTqLbwWdcA6EMXc/CYq3GmZSj1SyT4NXrq+YnTlfmofUI7K+AHQ6arfslt eHEtXWgcIcnFChHNUssefCuIJgojLD6qkVOWu/RLUzTNT7/gX8AmAXpqfkdt L2SfXNbyew; Authentication-Results: ams-dkim-1; header.From=paitken@cisco.com; dkim=pass ( sig from cisco.com/amsdkim1002 verified; ); Cc: IETF IPFIX Working Group Subject: Re: [IPFIX] WG last call on draft-ietf-ipfix-file-02 X-BeenThere: ipfix@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: IPFIX WG discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="us-ascii"; Format="flowed" Sender: ipfix-bounces@ietf.org Errors-To: ipfix-bounces@ietf.org I recently (July 28th and 30th) sent comments on file-02 which could be considered as WGLC input. P. Juergen Quittek wrote: > Dear all, > > At our session on Monday we concluded that the IPFIX file format > draft-ietf-ipfix-file-02 is ready for WG last call. > > The call starts today and will last until Friday August 29. > The period is longer than the usual two weeks, because > during August many people take vacation. > > Please read the draft! > Please send your comments to this mailing list! > Please also send a message if you are fine with the draft as it is! > > Thanks, > > Juergen > > _______________________________________________ > IPFIX mailing list > IPFIX@ietf.org > https://www.ietf.org/mailman/listinfo/ipfix -- Paul Aitken Cisco Systems Ltd, Edinburgh, Scotland. _______________________________________________ IPFIX mailing list IPFIX@ietf.org https://www.ietf.org/mailman/listinfo/ipfix From ipfix-bounces@ietf.org Fri Aug 1 03:14:19 2008 Return-Path: X-Original-To: ipfix-archive@lists.ietf.org Delivered-To: ietfarch-ipfix-archive@core3.amsl.com Received: from [127.0.0.1] (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 105013A6D96; Fri, 1 Aug 2008 03:14:19 -0700 (PDT) X-Original-To: ipfix@core3.amsl.com Delivered-To: ipfix@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 748A93A6D96 for ; Fri, 1 Aug 2008 03:14:18 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.26 X-Spam-Level: X-Spam-Status: No, score=-6.26 tagged_above=-999 required=5 tests=[AWL=0.339, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id I4RizWX62+J5 for ; Fri, 1 Aug 2008 03:14:17 -0700 (PDT) Received: from ams-iport-1.cisco.com (ams-iport-1.cisco.com [144.254.224.140]) by core3.amsl.com (Postfix) with ESMTP id 3EED93A6897 for ; Fri, 1 Aug 2008 03:14:17 -0700 (PDT) X-IronPort-AV: E=Sophos;i="4.31,292,1215388800"; d="scan'208";a="15990508" Received: from ams-dkim-1.cisco.com ([144.254.224.138]) by ams-iport-1.cisco.com with ESMTP; 01 Aug 2008 10:14:05 +0000 Received: from ams-core-1.cisco.com (ams-core-1.cisco.com [144.254.224.150]) by ams-dkim-1.cisco.com (8.12.11/8.12.11) with ESMTP id m71AE5CE012235; Fri, 1 Aug 2008 12:14:05 +0200 Received: from cisco.com (mrwint.cisco.com [64.103.71.48]) by ams-core-1.cisco.com (8.13.8/8.13.8) with ESMTP id m71AE5Cq005907; Fri, 1 Aug 2008 10:14:05 GMT Received: from [10.61.80.75] (ams3-vpn-dhcp4172.cisco.com [10.61.80.75]) by cisco.com (8.11.7p3+Sun/8.8.8) with ESMTP id m71AE4J09646; Fri, 1 Aug 2008 11:14:04 +0100 (BST) Message-ID: <4892E1EE.1060806@cisco.com> Date: Fri, 01 Aug 2008 11:14:06 +0100 From: Paul Aitken User-Agent: Mozilla/5.0 (X11; U; Linux i686 (x86_64); en-GB; rv:1.8.1.16) Gecko/20080702 SeaMonkey/1.1.11 MIME-Version: 1.0 To: Juergen Quittek References: In-Reply-To: DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; l=836; t=1217585645; x=1218449645; c=relaxed/simple; s=amsdkim1002; h=Content-Type:From:Subject:Content-Transfer-Encoding:MIME-Version; d=cisco.com; i=paitken@cisco.com; z=From:=20Paul=20Aitken=20 |Subject:=20Re=3A=20[IPFIX]=20WG=20last=20call=20on=20draft -ietf-ipfix-file-02 |Sender:=20; bh=m3TrgatbLrjQF9NPJ4L02TsU+3S/W55s7Lo7UndeRis=; b=qTqLbwWdcA6EMXc/CYq3GmZSj1SyT4NXrq+YnTlfmofUI7K+AHQ6arfslt eHEtXWgcIcnFChHNUssefCuIJgojLD6qkVOWu/RLUzTNT7/gX8AmAXpqfkdt L2SfXNbyew; Authentication-Results: ams-dkim-1; header.From=paitken@cisco.com; dkim=pass ( sig from cisco.com/amsdkim1002 verified; ); Cc: IETF IPFIX Working Group Subject: Re: [IPFIX] WG last call on draft-ietf-ipfix-file-02 X-BeenThere: ipfix@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: IPFIX WG discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="us-ascii"; Format="flowed" Sender: ipfix-bounces@ietf.org Errors-To: ipfix-bounces@ietf.org I recently (July 28th and 30th) sent comments on file-02 which could be considered as WGLC input. P. Juergen Quittek wrote: > Dear all, > > At our session on Monday we concluded that the IPFIX file format > draft-ietf-ipfix-file-02 is ready for WG last call. > > The call starts today and will last until Friday August 29. > The period is longer than the usual two weeks, because > during August many people take vacation. > > Please read the draft! > Please send your comments to this mailing list! > Please also send a message if you are fine with the draft as it is! > > Thanks, > > Juergen > > _______________________________________________ > IPFIX mailing list > IPFIX@ietf.org > https://www.ietf.org/mailman/listinfo/ipfix -- Paul Aitken Cisco Systems Ltd, Edinburgh, Scotland. _______________________________________________ IPFIX mailing list IPFIX@ietf.org https://www.ietf.org/mailman/listinfo/ipfix From ipfix-bounces@ietf.org Fri Aug 1 23:29:40 2008 Return-Path: X-Original-To: ipfix-archive@lists.ietf.org Delivered-To: ietfarch-ipfix-archive@core3.amsl.com Received: from [127.0.0.1] (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 638FD3A6963; Fri, 1 Aug 2008 23:29:40 -0700 (PDT) X-Original-To: ipfix@core3.amsl.com Delivered-To: ipfix@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 5404C3A6917 for ; Fri, 1 Aug 2008 23:29:39 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.249 X-Spam-Level: X-Spam-Status: No, score=-2.249 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HELO_EQ_DE=0.35] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id m2nOrwYWyJgv for ; Fri, 1 Aug 2008 23:29:38 -0700 (PDT) Received: from smtp.cs.uni-tuebingen.de (u-173-c156.cs.uni-tuebingen.de [134.2.173.156]) by core3.amsl.com (Postfix) with ESMTP id E04763A6951 for ; Fri, 1 Aug 2008 23:29:36 -0700 (PDT) Received: from [193.120.192.61] (helo=[192.168.221.4]) by smtp.cs.uni-tuebingen.de with esmtpsa (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.69) (envelope-from ) id 1KPAcO-0000uj-Co for ipfix@ietf.org; Sat, 02 Aug 2008 08:29:32 +0200 Message-ID: <4893FEE5.3050502@informatik.uni-tuebingen.de> Date: Sat, 02 Aug 2008 08:29:57 +0200 From: Gerhard Muenz User-Agent: Thunderbird 2.0.0.16 (Windows/20080708) MIME-Version: 1.0 To: "ipfix@ietf.org" Subject: [IPFIX] Comments for IPFIX anonymization X-BeenThere: ipfix@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: IPFIX WG discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============0150791484==" Sender: ipfix-bounces@ietf.org Errors-To: ipfix-bounces@ietf.org This is a cryptographically signed message in MIME format. --===============0150791484== Content-Type: multipart/signed; protocol="application/x-pkcs7-signature"; micalg=sha1; boundary="------------ms020706020909080209080705" This is a cryptographically signed message in MIME format. --------------ms020706020909080209080705 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Elisa, Brian, I'm glad to see that somebody takes over this task. With respect to IP address anonymization, there exist a couple of excellent publications. So, I suggest to cite the most important ones. Some comments concerning the current draft version: Throughout the document, there is a mixture of British English and American English. So we might guess who wrote which part, and there is room for anonymization ;) > 4.1.2. Random Permutations >=20 >=20 > When random permutations are used, each IP Address is replaced with = a > random permutation on the set of possible IP Addresses. The > permutation function can be implemented using hash tables. You probably mean hash functions, not hash tables. > 4.1.3. Prefix-preserving Pseudonymisation >=20 >=20 > Prefix-preserving pseudonymisation preserves the structure of IP > Addresses. If two IP Addresses match on a prefix of "n" bits, their= > anonymised versions will match on a prefix of "n" bits too. In contrast to your table, prefix-preserving pseudonymization may be irreversible, too. For example, you can use CryptoPAN and throw away the key after anonymization. > +-----------------------+----------------+---------------+ > | Scheme | Action | Reversibility | > +-----------------------+----------------+---------------+ > | Precision Degradation | Generalisation | N | > | Enumeration | Substitution | Y | > | Random Shifts | Substitution | Y | > +-----------------------+----------------+---------------+ Why is enumeration reversible? If I have timestamps in milliseconds 345, 700, 701 and transform them into 1, 2, 3, how can you revert this transformation? When I saw the table, I assumed the random shifts to be random and different for each timestamp (like noise). Then it would be irreversible.= Regards, Gerhard --=20 Dipl.-Ing. Gerhard M=FCnz Computer Networks and Internet Wilhelm Schickard Institute for Computer Science University of Tuebingen Sand 13 (Room B309), D-72076 Tuebingen, Germany Phone: +49 7071 29-70534 Fax: +49 7071 29-5220 E-mail: muenz@informatik.uni-tuebingen.de WWW: http://net.informatik.uni-tuebingen.de/~muenz --------------ms020706020909080209080705 Content-Type: application/x-pkcs7-signature; name="smime.p7s" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="smime.p7s" Content-Description: S/MIME Cryptographic Signature MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAQAAoIIJdTCC AxUwggJ+oAMCAQICED9aGsYWkMr+s4zmyODhB+IwDQYJKoZIhvcNAQEFBQAwYjELMAkGA1UE BhMCWkExJTAjBgNVBAoTHFRoYXd0ZSBDb25zdWx0aW5nIChQdHkpIEx0ZC4xLDAqBgNVBAMT I1RoYXd0ZSBQZXJzb25hbCBGcmVlbWFpbCBJc3N1aW5nIENBMB4XDTA4MDIxMzE1MTUxN1oX DTA5MDIxMjE1MTUxN1owbDEOMAwGA1UEBBMFTXVlbnoxEDAOBgNVBCoTB0dlcmhhcmQxFjAU BgNVBAMTDUdlcmhhcmQgTXVlbnoxMDAuBgkqhkiG9w0BCQEWIW11ZW56QGluZm9ybWF0aWsu dW5pLXR1ZWJpbmdlbi5kZTCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAMZex/Sq sAxkzTVvKP/YAgkaeXA+ngH59Aa0bbRPsKOWzAndGqty5EKcEzrnKqEJ27qHFvoF/pHp88U2 7SJI/xbqkgWeV2jRaldipZQYlnjYLQcmb4cewIFuGRRSVrm3BquzX38aYazuE4+DVH2Z3a8z n0FcdMXhA1NR2Ma1rh4G7SIeZ+hC7czbvNRPraBliGdQhs8J/6yP/iL8aNYAl9c7CL4ofRj8 Y9orMOV/4vtWTq76/VQUVdbhUMiv0D8aHqI1ZvGskhRRvmITgQRVbbn8N8WTpZ0UCgMDjxPP 9i5IhLfp6oBtsKl4OZ0RXvSLZrbJTkBX3vnEutcyxDvyNgMCAwEAAaM+MDwwLAYDVR0RBCUw I4EhbXVlbnpAaW5mb3JtYXRpay51bmktdHVlYmluZ2VuLmRlMAwGA1UdEwEB/wQCMAAwDQYJ KoZIhvcNAQEFBQADgYEAX5SiD6epJePwBjJumOsTF6wzeuZRDLYlN+fOpXwd2C0Yx6i8iIZ9 l/J/nGaE1YpJPfX5oJDE+tOk1vYh2E9ThLOj9kJ3buZmgOCdVu90qtCWhfhli7RCYcJ+G9M3 FCnqbrzI/waPPXGB8/DY1HKgPj5G+oKPUK+GD2aE1Q3PYGowggMVMIICfqADAgECAhA/WhrG FpDK/rOM5sjg4QfiMA0GCSqGSIb3DQEBBQUAMGIxCzAJBgNVBAYTAlpBMSUwIwYDVQQKExxU aGF3dGUgQ29uc3VsdGluZyAoUHR5KSBMdGQuMSwwKgYDVQQDEyNUaGF3dGUgUGVyc29uYWwg RnJlZW1haWwgSXNzdWluZyBDQTAeFw0wODAyMTMxNTE1MTdaFw0wOTAyMTIxNTE1MTdaMGwx DjAMBgNVBAQTBU11ZW56MRAwDgYDVQQqEwdHZXJoYXJkMRYwFAYDVQQDEw1HZXJoYXJkIE11 ZW56MTAwLgYJKoZIhvcNAQkBFiFtdWVuekBpbmZvcm1hdGlrLnVuaS10dWViaW5nZW4uZGUw ggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQDGXsf0qrAMZM01byj/2AIJGnlwPp4B +fQGtG20T7CjlswJ3RqrcuRCnBM65yqhCdu6hxb6Bf6R6fPFNu0iSP8W6pIFnldo0WpXYqWU GJZ42C0HJm+HHsCBbhkUUla5twars19/GmGs7hOPg1R9md2vM59BXHTF4QNTUdjGta4eBu0i HmfoQu3M27zUT62gZYhnUIbPCf+sj/4i/GjWAJfXOwi+KH0Y/GPaKzDlf+L7Vk6u+v1UFFXW 4VDIr9A/Gh6iNWbxrJIUUb5iE4EEVW25/DfFk6WdFAoDA48Tz/YuSIS36eqAbbCpeDmdEV70 i2a2yU5AV975xLrXMsQ78jYDAgMBAAGjPjA8MCwGA1UdEQQlMCOBIW11ZW56QGluZm9ybWF0 aWsudW5pLXR1ZWJpbmdlbi5kZTAMBgNVHRMBAf8EAjAAMA0GCSqGSIb3DQEBBQUAA4GBAF+U og+nqSXj8AYybpjrExesM3rmUQy2JTfnzqV8HdgtGMeovIiGfZfyf5xmhNWKST31+aCQxPrT pNb2IdhPU4Szo/ZCd27mZoDgnVbvdKrQloX4ZYu0QmHCfhvTNxQp6m68yP8Gjz1xgfPw2NRy oD4+RvqCj1Cvhg9mhNUNz2BqMIIDPzCCAqigAwIBAgIBDTANBgkqhkiG9w0BAQUFADCB0TEL MAkGA1UEBhMCWkExFTATBgNVBAgTDFdlc3Rlcm4gQ2FwZTESMBAGA1UEBxMJQ2FwZSBUb3du MRowGAYDVQQKExFUaGF3dGUgQ29uc3VsdGluZzEoMCYGA1UECxMfQ2VydGlmaWNhdGlvbiBT ZXJ2aWNlcyBEaXZpc2lvbjEkMCIGA1UEAxMbVGhhd3RlIFBlcnNvbmFsIEZyZWVtYWlsIENB MSswKQYJKoZIhvcNAQkBFhxwZXJzb25hbC1mcmVlbWFpbEB0aGF3dGUuY29tMB4XDTAzMDcx NzAwMDAwMFoXDTEzMDcxNjIzNTk1OVowYjELMAkGA1UEBhMCWkExJTAjBgNVBAoTHFRoYXd0 ZSBDb25zdWx0aW5nIChQdHkpIEx0ZC4xLDAqBgNVBAMTI1RoYXd0ZSBQZXJzb25hbCBGcmVl bWFpbCBJc3N1aW5nIENBMIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQDEpjxVc1X7TrnK mVoeaMB1BHCd3+n/ox7svc31W/Iadr1/DDph8r9RzgHU5VAKMNcCY1osiRVwjt3J8CuFWqo/ cVbLrzwLB+fxH5E2JCoTzyvV84J3PQO+K/67GD4Hv0CAAmTXp6a7n2XRxSpUhQ9IBH+nttE8 YQRAHmQZcmC3+wIDAQABo4GUMIGRMBIGA1UdEwEB/wQIMAYBAf8CAQAwQwYDVR0fBDwwOjA4 oDagNIYyaHR0cDovL2NybC50aGF3dGUuY29tL1RoYXd0ZVBlcnNvbmFsRnJlZW1haWxDQS5j cmwwCwYDVR0PBAQDAgEGMCkGA1UdEQQiMCCkHjAcMRowGAYDVQQDExFQcml2YXRlTGFiZWwy LTEzODANBgkqhkiG9w0BAQUFAAOBgQBIjNFQg+oLLswNo2asZw9/r6y+whehQ5aUnX9MIbj4 Nh+qLZ82L8D0HFAgk3A8/a3hYWLD2ToZfoSxmRsAxRoLgnSeJVCUYsfbJ3FXJY3dqZw5jowg T2Vfldr394fWxghOrvbqNOUQGls1TXfjViF4gtwhGTXeJLHTHUb/XV9lTzGCA2QwggNgAgEB MHYwYjELMAkGA1UEBhMCWkExJTAjBgNVBAoTHFRoYXd0ZSBDb25zdWx0aW5nIChQdHkpIEx0 ZC4xLDAqBgNVBAMTI1RoYXd0ZSBQZXJzb25hbCBGcmVlbWFpbCBJc3N1aW5nIENBAhA/WhrG FpDK/rOM5sjg4QfiMAkGBSsOAwIaBQCgggHDMBgGCSqGSIb3DQEJAzELBgkqhkiG9w0BBwEw HAYJKoZIhvcNAQkFMQ8XDTA4MDgwMjA2Mjk1N1owIwYJKoZIhvcNAQkEMRYEFHHQGzqRLouH zcogD9GGW0QWWvkRMFIGCSqGSIb3DQEJDzFFMEMwCgYIKoZIhvcNAwcwDgYIKoZIhvcNAwIC AgCAMA0GCCqGSIb3DQMCAgFAMAcGBSsOAwIHMA0GCCqGSIb3DQMCAgEoMIGFBgkrBgEEAYI3 EAQxeDB2MGIxCzAJBgNVBAYTAlpBMSUwIwYDVQQKExxUaGF3dGUgQ29uc3VsdGluZyAoUHR5 KSBMdGQuMSwwKgYDVQQDEyNUaGF3dGUgUGVyc29uYWwgRnJlZW1haWwgSXNzdWluZyBDQQIQ P1oaxhaQyv6zjObI4OEH4jCBhwYLKoZIhvcNAQkQAgsxeKB2MGIxCzAJBgNVBAYTAlpBMSUw IwYDVQQKExxUaGF3dGUgQ29uc3VsdGluZyAoUHR5KSBMdGQuMSwwKgYDVQQDEyNUaGF3dGUg UGVyc29uYWwgRnJlZW1haWwgSXNzdWluZyBDQQIQP1oaxhaQyv6zjObI4OEH4jANBgkqhkiG 9w0BAQEFAASCAQBCFFPeiFht2tU7+tYGeDS0D1p+bbnoAu0wOLZdTrmfo1irXCclIvWWJ+2V hV9ddZOnBUGiggPGwBEHepAN4w4+GsBjFkrAvAk+Vqb2As5AZjXyjJopinKGOeppYaP1qqC8 yxEI2C1VG/BTri0kz7gBe+muJF3ZwHa+E9qbZOblgTBde4rkrD/m9mn1K6ClC5gAQipBq7jr FFhxBYVf6oFsuACJOtEZHfyMNdll7cCGJsfboLqBUaL/V5pF0e3O+dXgEUgU1VWLoNfBjqoS lJVnmMVHHRuz1QxiND3NYMJ27pxBBp3IhjVhWMYUzRrvogN0eeBz7our7ud9hPCNjO1zAAAA AAAA --------------ms020706020909080209080705-- --===============0150791484== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ IPFIX mailing list IPFIX@ietf.org https://www.ietf.org/mailman/listinfo/ipfix --===============0150791484==-- From ipfix-bounces@ietf.org Fri Aug 1 23:29:40 2008 Return-Path: X-Original-To: ipfix-archive@optimus.ietf.org Delivered-To: ietfarch-ipfix-archive@core3.amsl.com Received: from [127.0.0.1] (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 638FD3A6963; Fri, 1 Aug 2008 23:29:40 -0700 (PDT) X-Original-To: ipfix@core3.amsl.com Delivered-To: ipfix@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 5404C3A6917 for ; Fri, 1 Aug 2008 23:29:39 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.249 X-Spam-Level: X-Spam-Status: No, score=-2.249 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HELO_EQ_DE=0.35] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id m2nOrwYWyJgv for ; Fri, 1 Aug 2008 23:29:38 -0700 (PDT) Received: from smtp.cs.uni-tuebingen.de (u-173-c156.cs.uni-tuebingen.de [134.2.173.156]) by core3.amsl.com (Postfix) with ESMTP id E04763A6951 for ; Fri, 1 Aug 2008 23:29:36 -0700 (PDT) Received: from [193.120.192.61] (helo=[192.168.221.4]) by smtp.cs.uni-tuebingen.de with esmtpsa (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.69) (envelope-from ) id 1KPAcO-0000uj-Co for ipfix@ietf.org; Sat, 02 Aug 2008 08:29:32 +0200 Message-ID: <4893FEE5.3050502@informatik.uni-tuebingen.de> Date: Sat, 02 Aug 2008 08:29:57 +0200 From: Gerhard Muenz User-Agent: Thunderbird 2.0.0.16 (Windows/20080708) MIME-Version: 1.0 To: "ipfix@ietf.org" Subject: [IPFIX] Comments for IPFIX anonymization X-BeenThere: ipfix@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: IPFIX WG discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============0150791484==" Sender: ipfix-bounces@ietf.org Errors-To: ipfix-bounces@ietf.org This is a cryptographically signed message in MIME format. --===============0150791484== Content-Type: multipart/signed; protocol="application/x-pkcs7-signature"; micalg=sha1; boundary="------------ms020706020909080209080705" This is a cryptographically signed message in MIME format. --------------ms020706020909080209080705 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Elisa, Brian, I'm glad to see that somebody takes over this task. With respect to IP address anonymization, there exist a couple of excellent publications. So, I suggest to cite the most important ones. Some comments concerning the current draft version: Throughout the document, there is a mixture of British English and American English. So we might guess who wrote which part, and there is room for anonymization ;) > 4.1.2. Random Permutations >=20 >=20 > When random permutations are used, each IP Address is replaced with = a > random permutation on the set of possible IP Addresses. The > permutation function can be implemented using hash tables. You probably mean hash functions, not hash tables. > 4.1.3. Prefix-preserving Pseudonymisation >=20 >=20 > Prefix-preserving pseudonymisation preserves the structure of IP > Addresses. If two IP Addresses match on a prefix of "n" bits, their= > anonymised versions will match on a prefix of "n" bits too. In contrast to your table, prefix-preserving pseudonymization may be irreversible, too. For example, you can use CryptoPAN and throw away the key after anonymization. > +-----------------------+----------------+---------------+ > | Scheme | Action | Reversibility | > +-----------------------+----------------+---------------+ > | Precision Degradation | Generalisation | N | > | Enumeration | Substitution | Y | > | Random Shifts | Substitution | Y | > +-----------------------+----------------+---------------+ Why is enumeration reversible? If I have timestamps in milliseconds 345, 700, 701 and transform them into 1, 2, 3, how can you revert this transformation? When I saw the table, I assumed the random shifts to be random and different for each timestamp (like noise). Then it would be irreversible.= Regards, Gerhard --=20 Dipl.-Ing. Gerhard M=FCnz Computer Networks and Internet Wilhelm Schickard Institute for Computer Science University of Tuebingen Sand 13 (Room B309), D-72076 Tuebingen, Germany Phone: +49 7071 29-70534 Fax: +49 7071 29-5220 E-mail: muenz@informatik.uni-tuebingen.de WWW: http://net.informatik.uni-tuebingen.de/~muenz --------------ms020706020909080209080705 Content-Type: application/x-pkcs7-signature; name="smime.p7s" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="smime.p7s" Content-Description: S/MIME Cryptographic Signature MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAQAAoIIJdTCC AxUwggJ+oAMCAQICED9aGsYWkMr+s4zmyODhB+IwDQYJKoZIhvcNAQEFBQAwYjELMAkGA1UE BhMCWkExJTAjBgNVBAoTHFRoYXd0ZSBDb25zdWx0aW5nIChQdHkpIEx0ZC4xLDAqBgNVBAMT I1RoYXd0ZSBQZXJzb25hbCBGcmVlbWFpbCBJc3N1aW5nIENBMB4XDTA4MDIxMzE1MTUxN1oX DTA5MDIxMjE1MTUxN1owbDEOMAwGA1UEBBMFTXVlbnoxEDAOBgNVBCoTB0dlcmhhcmQxFjAU BgNVBAMTDUdlcmhhcmQgTXVlbnoxMDAuBgkqhkiG9w0BCQEWIW11ZW56QGluZm9ybWF0aWsu dW5pLXR1ZWJpbmdlbi5kZTCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAMZex/Sq sAxkzTVvKP/YAgkaeXA+ngH59Aa0bbRPsKOWzAndGqty5EKcEzrnKqEJ27qHFvoF/pHp88U2 7SJI/xbqkgWeV2jRaldipZQYlnjYLQcmb4cewIFuGRRSVrm3BquzX38aYazuE4+DVH2Z3a8z n0FcdMXhA1NR2Ma1rh4G7SIeZ+hC7czbvNRPraBliGdQhs8J/6yP/iL8aNYAl9c7CL4ofRj8 Y9orMOV/4vtWTq76/VQUVdbhUMiv0D8aHqI1ZvGskhRRvmITgQRVbbn8N8WTpZ0UCgMDjxPP 9i5IhLfp6oBtsKl4OZ0RXvSLZrbJTkBX3vnEutcyxDvyNgMCAwEAAaM+MDwwLAYDVR0RBCUw I4EhbXVlbnpAaW5mb3JtYXRpay51bmktdHVlYmluZ2VuLmRlMAwGA1UdEwEB/wQCMAAwDQYJ KoZIhvcNAQEFBQADgYEAX5SiD6epJePwBjJumOsTF6wzeuZRDLYlN+fOpXwd2C0Yx6i8iIZ9 l/J/nGaE1YpJPfX5oJDE+tOk1vYh2E9ThLOj9kJ3buZmgOCdVu90qtCWhfhli7RCYcJ+G9M3 FCnqbrzI/waPPXGB8/DY1HKgPj5G+oKPUK+GD2aE1Q3PYGowggMVMIICfqADAgECAhA/WhrG FpDK/rOM5sjg4QfiMA0GCSqGSIb3DQEBBQUAMGIxCzAJBgNVBAYTAlpBMSUwIwYDVQQKExxU aGF3dGUgQ29uc3VsdGluZyAoUHR5KSBMdGQuMSwwKgYDVQQDEyNUaGF3dGUgUGVyc29uYWwg RnJlZW1haWwgSXNzdWluZyBDQTAeFw0wODAyMTMxNTE1MTdaFw0wOTAyMTIxNTE1MTdaMGwx DjAMBgNVBAQTBU11ZW56MRAwDgYDVQQqEwdHZXJoYXJkMRYwFAYDVQQDEw1HZXJoYXJkIE11 ZW56MTAwLgYJKoZIhvcNAQkBFiFtdWVuekBpbmZvcm1hdGlrLnVuaS10dWViaW5nZW4uZGUw ggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQDGXsf0qrAMZM01byj/2AIJGnlwPp4B +fQGtG20T7CjlswJ3RqrcuRCnBM65yqhCdu6hxb6Bf6R6fPFNu0iSP8W6pIFnldo0WpXYqWU GJZ42C0HJm+HHsCBbhkUUla5twars19/GmGs7hOPg1R9md2vM59BXHTF4QNTUdjGta4eBu0i HmfoQu3M27zUT62gZYhnUIbPCf+sj/4i/GjWAJfXOwi+KH0Y/GPaKzDlf+L7Vk6u+v1UFFXW 4VDIr9A/Gh6iNWbxrJIUUb5iE4EEVW25/DfFk6WdFAoDA48Tz/YuSIS36eqAbbCpeDmdEV70 i2a2yU5AV975xLrXMsQ78jYDAgMBAAGjPjA8MCwGA1UdEQQlMCOBIW11ZW56QGluZm9ybWF0 aWsudW5pLXR1ZWJpbmdlbi5kZTAMBgNVHRMBAf8EAjAAMA0GCSqGSIb3DQEBBQUAA4GBAF+U og+nqSXj8AYybpjrExesM3rmUQy2JTfnzqV8HdgtGMeovIiGfZfyf5xmhNWKST31+aCQxPrT pNb2IdhPU4Szo/ZCd27mZoDgnVbvdKrQloX4ZYu0QmHCfhvTNxQp6m68yP8Gjz1xgfPw2NRy oD4+RvqCj1Cvhg9mhNUNz2BqMIIDPzCCAqigAwIBAgIBDTANBgkqhkiG9w0BAQUFADCB0TEL MAkGA1UEBhMCWkExFTATBgNVBAgTDFdlc3Rlcm4gQ2FwZTESMBAGA1UEBxMJQ2FwZSBUb3du MRowGAYDVQQKExFUaGF3dGUgQ29uc3VsdGluZzEoMCYGA1UECxMfQ2VydGlmaWNhdGlvbiBT ZXJ2aWNlcyBEaXZpc2lvbjEkMCIGA1UEAxMbVGhhd3RlIFBlcnNvbmFsIEZyZWVtYWlsIENB MSswKQYJKoZIhvcNAQkBFhxwZXJzb25hbC1mcmVlbWFpbEB0aGF3dGUuY29tMB4XDTAzMDcx NzAwMDAwMFoXDTEzMDcxNjIzNTk1OVowYjELMAkGA1UEBhMCWkExJTAjBgNVBAoTHFRoYXd0 ZSBDb25zdWx0aW5nIChQdHkpIEx0ZC4xLDAqBgNVBAMTI1RoYXd0ZSBQZXJzb25hbCBGcmVl bWFpbCBJc3N1aW5nIENBMIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQDEpjxVc1X7TrnK mVoeaMB1BHCd3+n/ox7svc31W/Iadr1/DDph8r9RzgHU5VAKMNcCY1osiRVwjt3J8CuFWqo/ cVbLrzwLB+fxH5E2JCoTzyvV84J3PQO+K/67GD4Hv0CAAmTXp6a7n2XRxSpUhQ9IBH+nttE8 YQRAHmQZcmC3+wIDAQABo4GUMIGRMBIGA1UdEwEB/wQIMAYBAf8CAQAwQwYDVR0fBDwwOjA4 oDagNIYyaHR0cDovL2NybC50aGF3dGUuY29tL1RoYXd0ZVBlcnNvbmFsRnJlZW1haWxDQS5j cmwwCwYDVR0PBAQDAgEGMCkGA1UdEQQiMCCkHjAcMRowGAYDVQQDExFQcml2YXRlTGFiZWwy LTEzODANBgkqhkiG9w0BAQUFAAOBgQBIjNFQg+oLLswNo2asZw9/r6y+whehQ5aUnX9MIbj4 Nh+qLZ82L8D0HFAgk3A8/a3hYWLD2ToZfoSxmRsAxRoLgnSeJVCUYsfbJ3FXJY3dqZw5jowg T2Vfldr394fWxghOrvbqNOUQGls1TXfjViF4gtwhGTXeJLHTHUb/XV9lTzGCA2QwggNgAgEB MHYwYjELMAkGA1UEBhMCWkExJTAjBgNVBAoTHFRoYXd0ZSBDb25zdWx0aW5nIChQdHkpIEx0 ZC4xLDAqBgNVBAMTI1RoYXd0ZSBQZXJzb25hbCBGcmVlbWFpbCBJc3N1aW5nIENBAhA/WhrG FpDK/rOM5sjg4QfiMAkGBSsOAwIaBQCgggHDMBgGCSqGSIb3DQEJAzELBgkqhkiG9w0BBwEw HAYJKoZIhvcNAQkFMQ8XDTA4MDgwMjA2Mjk1N1owIwYJKoZIhvcNAQkEMRYEFHHQGzqRLouH zcogD9GGW0QWWvkRMFIGCSqGSIb3DQEJDzFFMEMwCgYIKoZIhvcNAwcwDgYIKoZIhvcNAwIC AgCAMA0GCCqGSIb3DQMCAgFAMAcGBSsOAwIHMA0GCCqGSIb3DQMCAgEoMIGFBgkrBgEEAYI3 EAQxeDB2MGIxCzAJBgNVBAYTAlpBMSUwIwYDVQQKExxUaGF3dGUgQ29uc3VsdGluZyAoUHR5 KSBMdGQuMSwwKgYDVQQDEyNUaGF3dGUgUGVyc29uYWwgRnJlZW1haWwgSXNzdWluZyBDQQIQ P1oaxhaQyv6zjObI4OEH4jCBhwYLKoZIhvcNAQkQAgsxeKB2MGIxCzAJBgNVBAYTAlpBMSUw IwYDVQQKExxUaGF3dGUgQ29uc3VsdGluZyAoUHR5KSBMdGQuMSwwKgYDVQQDEyNUaGF3dGUg UGVyc29uYWwgRnJlZW1haWwgSXNzdWluZyBDQQIQP1oaxhaQyv6zjObI4OEH4jANBgkqhkiG 9w0BAQEFAASCAQBCFFPeiFht2tU7+tYGeDS0D1p+bbnoAu0wOLZdTrmfo1irXCclIvWWJ+2V hV9ddZOnBUGiggPGwBEHepAN4w4+GsBjFkrAvAk+Vqb2As5AZjXyjJopinKGOeppYaP1qqC8 yxEI2C1VG/BTri0kz7gBe+muJF3ZwHa+E9qbZOblgTBde4rkrD/m9mn1K6ClC5gAQipBq7jr FFhxBYVf6oFsuACJOtEZHfyMNdll7cCGJsfboLqBUaL/V5pF0e3O+dXgEUgU1VWLoNfBjqoS lJVnmMVHHRuz1QxiND3NYMJ27pxBBp3IhjVhWMYUzRrvogN0eeBz7our7ud9hPCNjO1zAAAA AAAA --------------ms020706020909080209080705-- --===============0150791484== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ IPFIX mailing list IPFIX@ietf.org https://www.ietf.org/mailman/listinfo/ipfix --===============0150791484==-- From ipfix-bounces@ietf.org Mon Aug 4 02:12:51 2008 Return-Path: X-Original-To: ipfix-archive@lists.ietf.org Delivered-To: ietfarch-ipfix-archive@core3.amsl.com Received: from [127.0.0.1] (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 95A3E3A6C35; Mon, 4 Aug 2008 02:12:51 -0700 (PDT) X-Original-To: ipfix@core3.amsl.com Delivered-To: ipfix@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 9FC723A6C27 for ; Mon, 4 Aug 2008 02:12:49 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.599 X-Spam-Level: X-Spam-Status: No, score=-6.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id pVGAeXjxzj4z for ; Mon, 4 Aug 2008 02:12:48 -0700 (PDT) Received: from smtp.ee.ethz.ch (smtp.ee.ethz.ch [129.132.2.219]) by core3.amsl.com (Postfix) with ESMTP id B88963A6BE5 for ; Mon, 4 Aug 2008 02:12:48 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by smtp.ee.ethz.ch (Postfix) with ESMTP id 09082D936E; Mon, 4 Aug 2008 11:13:12 +0200 (MEST) X-Virus-Scanned: by amavisd-new on smtp.ee.ethz.ch Received: from smtp.ee.ethz.ch ([127.0.0.1]) by localhost (.ee.ethz.ch [127.0.0.1]) (amavisd-new, port 10024) with LMTP id XN-KeVbG3-0q; Mon, 4 Aug 2008 11:13:11 +0200 (MEST) Received: from pb-10072.ethz.ch (pb-10072.ethz.ch [82.130.103.195]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) (Authenticated sender: briant) by smtp.ee.ethz.ch (Postfix) with ESMTP id A8CA3D936A; Mon, 4 Aug 2008 11:13:11 +0200 (MEST) Message-Id: From: Brian Trammell To: Paul Aitken In-Reply-To: <4892E1EE.1060806@cisco.com> Mime-Version: 1.0 (Apple Message framework v928.1) Date: Mon, 4 Aug 2008 11:13:11 +0200 References: <4892E1EE.1060806@cisco.com> X-Mailer: Apple Mail (2.928.1) Cc: IETF IPFIX Working Group , Juergen Quittek Subject: Re: [IPFIX] WG last call on draft-ietf-ipfix-file-02 X-BeenThere: ipfix@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: IPFIX WG discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="us-ascii"; Format="flowed"; DelSp="yes" Sender: ipfix-bounces@ietf.org Errors-To: ipfix-bounces@ietf.org All, We've already incorporated these into a new -03 rev to be published after the end of the WGLC and to address all last call comments, so, de facto, they already are considered WGLC input. :) Cheers, Brian On Aug 1, 2008, at 12:14 PM, Paul Aitken wrote: > I recently (July 28th and 30th) sent comments on file-02 which could > be considered as WGLC input. > > P. > > Juergen Quittek wrote: >> Dear all, At our session on Monday we concluded that the IPFIX file >> format >> draft-ietf-ipfix-file-02 is ready for WG last call. >> The call starts today and will last until Friday August 29. >> The period is longer than the usual two weeks, because >> during August many people take vacation. >> Please read the draft! >> Please send your comments to this mailing list! >> Please also send a message if you are fine with the draft as it is! >> Thanks, Juergen _______________________________________________ >> IPFIX mailing list >> IPFIX@ietf.org >> https://www.ietf.org/mailman/listinfo/ipfix > > > -- > Paul Aitken > Cisco Systems Ltd, EdinbuFrom ipfix-bounces@ietf.org Mon Aug 4 02:12:51 2008 Return-Path: X-Original-To: ipfix-archive@optimus.ietf.org Delivered-To: ietfarch-ipfix-archive@core3.amsl.com Received: from [127.0.0.1] (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 95A3E3A6C35; Mon, 4 Aug 2008 02:12:51 -0700 (PDT) X-Original-To: ipfix@core3.amsl.com Delivered-To: ipfix@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 9FC723A6C27 for ; Mon, 4 Aug 2008 02:12:49 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.599 X-Spam-Level: X-Spam-Status: No, score=-6.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id pVGAeXjxzj4z for ; Mon, 4 Aug 2008 02:12:48 -0700 (PDT) Received: from smtp.ee.ethz.ch (smtp.ee.ethz.ch [129.132.2.219]) by core3.amsl.com (Postfix) with ESMTP id B88963A6BE5 for ; Mon, 4 Aug 2008 02:12:48 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by smtp.ee.ethz.ch (Postfix) with ESMTP id 09082D936E; Mon, 4 Aug 2008 11:13:12 +0200 (MEST) X-Virus-Scanned: by amavisd-new on smtp.ee.ethz.ch Received: from smtp.ee.ethz.ch ([127.0.0.1]) by localhost (.ee.ethz.ch [127.0.0.1]) (amavisd-new, port 10024) with LMTP id XN-KeVbG3-0q; Mon, 4 Aug 2008 11:13:11 +0200 (MEST) Received: from pb-10072.ethz.ch (pb-10072.ethz.ch [82.130.103.195]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) (Authenticated sender: briant) by smtp.ee.ethz.ch (Postfix) with ESMTP id A8CA3D936A; Mon, 4 Aug 2008 11:13:11 +0200 (MEST) Message-Id: From: Brian Trammell To: Paul Aitken In-Reply-To: <4892E1EE.1060806@cisco.com> Mime-Version: 1.0 (Apple Message framework v928.1) Date: Mon, 4 Aug 2008 11:13:11 +0200 References: <4892E1EE.1060806@cisco.com> X-Mailer: Apple Mail (2.928.1) Cc: IETF IPFIX Working Group , Juergen Quittek Subject: Re: [IPFIX] WG last call on draft-ietf-ipfix-file-02 X-BeenThere: ipfix@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: IPFIX WG discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="us-ascii"; Format="flowed"; DelSp="yes" Sender: ipfix-bounces@ietf.org Errors-To: ipfix-bounces@ietf.org All, We've already incorporated these into a new -03 rev to be published after the end of the WGLC and to address all last call comments, so, de facto, they already are considered WGLC input. :) Cheers, Brian On Aug 1, 2008, at 12:14 PM, Paul Aitken wrote: > I recently (July 28th and 30th) sent comments on file-02 which could > be considered as WGLC input. > > P. > > Juergen Quittek wrote: >> Dear all, At our session on Monday we concluded that the IPFIX file >> format >> draft-ietf-ipfix-file-02 is ready for WG last call. >> The call starts today and will last until Friday August 29. >> The period is longer than the usual two weeks, because >> during August many people take vacation. >> Please read the draft! >> Please send your comments to this mailing list! >> Please also send a message if you are fine with the draft as it is! >> Thanks, Juergen _______________________________________________ >> IPFIX mailing list >> IPFIX@ietf.org >> https://www.ietf.org/mailman/listinfo/ipfix > > > -- > Paul Aitken > Cisco Systems Ltd, Edinrgh, Scotland. > _______________________________________________ > IPFIX mailing list > IPFIX@ietf.org > https://www.ietf.org/mailman/listinfo/ipfix _______________________________________________ IPFIX mailing list IPFIX@ietf.org https://www.ietf.org/mailman/listinfo/ipfix burgh, Scotland. > _______________________________________________ > IPFIX mailing list > IPFIX@ietf.org > https://www.ietf.org/mailman/listinfo/ipfix _______________________________________________ IPFIX mailing list IPFIX@ietf.org https://www.ietf.org/mailman/listinfo/ipfix From ipfiroa@aperioninc.com Tue Aug 12 13:52:57 2008 Return-Path: X-Original-To: ietfarch-ipfix-archive@core3.amsl.com Delivered-To: ietfarch-ipfix-archive@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 806543A63EC for ; Tue, 12 Aug 2008 13:52:57 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -58.618 X-Spam-Level: X-Spam-Status: No, score=-58.618 tagged_above=-999 required=5 tests=[AWL=-43.527, BAYES_95=3, GB_I_LETTER=-2, HTML_EXTRA_CLOSE=2.809, HTML_FONT_SIZE_HUGE=0.057, HTML_MESSAGE=0.001, MIME_HTML_ONLY=1.457, NORMAL_HTTP_TO_IP=0.001, RAZOR2_CF_RANGE_51_100=0.5, RAZOR2_CF_RANGE_E4_51_100=1.5, RAZOR2_CF_RANGE_E8_51_100=1.5, RAZOR2_CHECK=0.5, RCVD_IN_BL_SPAMCOP_NET=1.96, RCVD_IN_XBL=3.033, SARE_UNI=0.591, URIBL_AB_SURBL=10, URIBL_BLACK=20, URIBL_JP_SURBL=10, URIBL_SBL=20, URIBL_SC_SURBL=10, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id KZ9OKiU-0RNz for ; Tue, 12 Aug 2008 13:52:57 -0700 (PDT) Received: from pool-71-115-52-174.sbndin.dsl-w.verizon.net (pool-71-115-52-174.sbndin.dsl-w.verizon.net [71.115.52.174]) by core3.amsl.com (Postfix) with SMTP id 2AA763A6827 for ; Tue, 12 Aug 2008 13:52:56 -0700 (PDT) Content-Return: allowed X-Mailer: CME-V6.5.4.3; MSN Message-Id: <20080812-35304.47058.qmail@pool-71-115-52-174.sbndin.dsl-w.verizon.net> To: Subject: Download Free ! From: admin@microsoft.com MIME-Version: 1.0 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: 7bit Date: Tue, 12 Aug 2008 13:52:56 -0700 (PDT)
Download the latest version Internet Explorer 7.0!

About this mailing:
You are receiving this e-mail because you subscribed to MSN Featured Offers. Microsoft respects your privacy. If you do not wish to receive this MSN Featured Offers e-mail, please click the "Unsubscribe" link below. This will not unsubscribe you from e-mail communications from third-party advertisers that may appear in MSN Feature Offers. This shall not constitute an offer by MSN. MSN shall not be responsible or liable for the advertisers' content nor any of the goods or service advertised. Prices and item availability subject to change without notice.

©2008 Microsoft | Unsubscribe | More Newsletters | Privacy

Microsoft Corporation, One Microsoft Way, Redmond, WA 98052
From ipfix-bounces@ietf.org Fri Aug 15 09:59:03 2008 Return-Path: X-Original-To: ipfix-archive@optimus.ietf.org Delivered-To: ietfarch-ipfix-archive@core3.amsl.com Received: from [127.0.0.1] (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 9B0B528C247; Fri, 15 Aug 2008 09:59:03 -0700 (PDT) X-Original-To: ipfix@core3.amsl.com Delivered-To: ipfix@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 70E8F28C247 for ; Fri, 15 Aug 2008 09:59:02 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -0.205 X-Spam-Level: X-Spam-Status: No, score=-0.205 tagged_above=-999 required=5 tests=[AWL=-0.555, BAYES_50=0.001, HELO_EQ_DE=0.35] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id X10LyE9xC443 for ; Fri, 15 Aug 2008 09:58:59 -0700 (PDT) Received: from smtp.cs.uni-tuebingen.de (u-173-c156.cs.uni-tuebingen.de [134.2.173.156]) by core3.amsl.com (Postfix) with ESMTP id BDAE228C15C for ; Fri, 15 Aug 2008 09:58:57 -0700 (PDT) Received: from u-172-c138.cs.uni-tuebingen.de ([134.2.172.138]) by smtp.cs.uni-tuebingen.de with esmtpsa (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.69) (envelope-from ) id 1KU2dK-0000wX-OL for ipfix@ietf.org; Fri, 15 Aug 2008 18:58:38 +0200 Message-ID: <48A5B5ED.5040107@informatik.uni-tuebingen.de> Date: Fri, 15 Aug 2008 18:59:25 +0200 From: Gerhard Muenz User-Agent: Thunderbird 2.0.0.16 (Windows/20080708) MIME-Version: 1.0 To: "ipfix@ietf.org" References: In-Reply-To: Subject: Re: [IPFIX] WG last call on draft-ietf-ipfix-file-02 X-BeenThere: ipfix@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: IPFIX WG discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============0692567637==" Sender: ipfix-bounces@ietf.org Errors-To: ipfix-bounces@ietf.org This is a cryptographically signed message in MIME format. --===============0692567637== Content-Type: multipart/signed; protocol="application/x-pkcs7-signature"; micalg=sha1; boundary="------------ms060301010901010606010100" This is a cryptographically signed message in MIME format. --------------ms060301010901010606010100 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Dear all, Here is a review for draft-ietf-ipfix-file-02. It is a very lengthy document. Many things are repeated. Maybe you can shorten it a little bit and make it more concise. Also, there are many unnecessary filler words like "note that". I do not really like that in a standard document. More comments inline. Gerhard > Abstract >=20 > This document describes a file format for the storage of flow data > based upon the IPFIX Message format. It proposes a set of > requirements for flat-file, binary flow data file formats, then flat, binary flow data file formats? > applies the IPFIX message format to these requirements to build a ne= w > file format. This IPFIX-based file format is designed to facilitate= > interoperability and reusability among a wide variety of flow > storage, processing, and analysis tools. > 1.1. IPFIX Documents Overview >=20 > "Specification of the IPFIX Protocol for the Exchange of IP Traffic > Flow Information" [RFC5101] and its associated documents define the Which are the associated documents? Ah, you mention them in the next paragraph. In this section, From ipfix-bounces@ietf.org Fri Aug 15 09:59:03 2008 Return-Path: X-Original-To: ipfix-archive@lists.ietf.org Delivered-To: ietfarch-ipfix-archive@core3.amsl.com Received: from [127.0.0.1] (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 9B0B528C247; Fri, 15 Aug 2008 09:59:03 -0700 (PDT) X-Original-To: ipfix@core3.amsl.com Delivered-To: ipfix@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 70E8F28C247 for ; Fri, 15 Aug 2008 09:59:02 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -0.205 X-Spam-Level: X-Spam-Status: No, score=-0.205 tagged_above=-999 required=5 tests=[AWL=-0.555, BAYES_50=0.001, HELO_EQ_DE=0.35] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id X10LyE9xC443 for ; Fri, 15 Aug 2008 09:58:59 -0700 (PDT) Received: from smtp.cs.uni-tuebingen.de (u-173-c156.cs.uni-tuebingen.de [134.2.173.156]) by core3.amsl.com (Postfix) with ESMTP id BDAE228C15C for ; Fri, 15 Aug 2008 09:58:57 -0700 (PDT) Received: from u-172-c138.cs.uni-tuebingen.de ([134.2.172.138]) by smtp.cs.uni-tuebingen.de with esmtpsa (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.69) (envelope-from ) id 1KU2dK-0000wX-OL for ipfix@ietf.org; Fri, 15 Aug 2008 18:58:38 +0200 Message-ID: <48A5B5ED.5040107@informatik.uni-tuebingen.de> Date: Fri, 15 Aug 2008 18:59:25 +0200 From: Gerhard Muenz User-Agent: Thunderbird 2.0.0.16 (Windows/20080708) MIME-Version: 1.0 To: "ipfix@ietf.org" References: In-Reply-To: Subject: Re: [IPFIX] WG last call on draft-ietf-ipfix-file-02 X-BeenThere: ipfix@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: IPFIX WG discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============0692567637==" Sender: ipfix-bounces@ietf.org Errors-To: ipfix-bounces@ietf.org This is a cryptographically signed message in MIME format. --===============0692567637== Content-Type: multipart/signed; protocol="application/x-pkcs7-signature"; micalg=sha1; boundary="------------ms060301010901010606010100" This is a cryptographically signed message in MIME format. --------------ms060301010901010606010100 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Dear all, Here is a review for draft-ietf-ipfix-file-02. It is a very lengthy document. Many things are repeated. Maybe you can shorten it a little bit and make it more concise. Also, there are many unnecessary filler words like "note that". I do not really like that in a standard document. More comments inline. Gerhard > Abstract >=20 > This document describes a file format for the storage of flow data > based upon the IPFIX Message format. It proposes a set of > requirements for flat-file, binary flow data file formats, then flat, binary flow data file formats? > applies the IPFIX message format to these requirements to build a ne= w > file format. This IPFIX-based file format is designed to facilitate= > interoperability and reusability among a wide variety of flow > storage, processing, and analysis tools. > 1.1. IPFIX Documents Overview >=20 > "Specification of the IPFIX Protocol for the Exchange of IP Traffic > Flow Information" [RFC5101] and its associated documents define the Which are the associated documents? Ah, you mention them in the next paragraph. In this section, ththe division into paragraphs seems to be arbitrary. > IPFIX Protocol, which provides network engineers and administrators > with access to IP traffic flow information. >=20 > "Architecture for IP Flow Information Export" [I-D.ietf-ipfix-arch] > defines the architecture for the export of measured IP flow > information out of an IPFIX Exporting Process to an IPFIX Collecting= > Process, and the basic terminology used to describe the elements of > this architecture, per the requirements defined in "Requirements for= > IP Flow Information Export" [RFC3917]. [RFC5101] then covers the Confusing as you already mentioned RFC 5101 before. I suggest modifying the introductory paragraph of this section such that it does not refer to RFC 5101 - or do not repeat it here. > details of the method for transporting IPFIX Data Records and > Templates via a congestion-aware transport protocol from an IPFIX > Exporting Process to an IPFIX Collecting Process. UDP is not congestion-aware. > "Information Model for IP Flow Information Export" [RFC5102] > describes the Information Elements used by IPFIX, including details > on Information Element naming, numbering, and data type encoding. > Finally, "IPFIX Applicability" [I-D.ietf-ipfix-as] describes the > various applications of the IPFIX protocol and their use of > information exported via IPFIX, and relates the IPFIX architecture t= o > other measurement architectures and frameworks. >=20 > In addition, "Exporting Type Information for IPFIX Information > Elements" [I-D.ietf-ipfix-exporting-type] specifies a method for > encoding Information Model properties within an IPFIX Message stream= =2E >=20 > This document references [RFC5101] and the architecture document for= > terminology, defines IPFIX File Writer and IPFIX File Reader in term= s > of the IPFIX Exporting Processes and IPFIX Collecting Process > definitions from [RFC5101], and extends the IPFIX Information Model > defined in [RFC5102] to provide new Information Elements for IPFIX > File metadata. It uses the method described in "Exporting Type > Information for IPFIX Information Elements" > [I-D.ietf-ipfix-exporting-type] document to support the self- > description of IPFIX Files containing enterprise-specific Informatio= n > Elements. >=20 >=20 > 2. Terminology >=20 > Terms used in this document that are defined in the Terminology > section of [RFC5101] are to be interpreted as defined there. One more sentence explaining that you define additional terms below. > IPFIX File: An IPFIX File is a serialized stream of IPFIX Messages= A stream usually is continuous while a file has a beginning and an end. Consider: series or sequence of IPFIX Messages. > stored on a filesystem. Any IPFIX Message stream that would be > considered valid when transported one or more of the specified > IPFIX transports (SCTP, TCP, or UDP) as defined in [RFC5101] is > considered an IPFIX File for purposes of this document; however, What do you mean by "for purposes of this document"? Possibly remove it? > this document extends that definition with recommendations on the= > construction of IPFIX Files that meet the requirements identified= > herein. Where is "herein"? in the file? in the document? > IPFIX File Reader: An IPFIX File Reader is a Process which reads > IPFIX Files from a filesystem, and is analogous to an IPFIX I'm not a native English speaker, yet "analogous to" sounds very vague as long as you do not say in which respect the analogy holds. I do not have a good idea how to express what you want to say. As a starting point, consider something like: an IPFIX File Reader is used as a replacement of an IPFIX Collection Process. > Collecting Process. An IPFIX File Reader MUST behave as an IPFIX= > Collecting Process as outlined in [RFC5101], except as modified b= y > this document. >=20 > IPFIX File Writer: An IPFIX File Writer is a process which we division into paragraphs seems to be arbitrary. > IPFIX Protocol, which provides network engineers and administrators > with access to IP traffic flow information. >=20 > "Architecture for IP Flow Information Export" [I-D.ietf-ipfix-arch] > defines the architecture for the export of measured IP flow > information out of an IPFIX Exporting Process to an IPFIX Collecting= > Process, and the basic terminology used to describe the elements of > this architecture, per the requirements defined in "Requirements for= > IP Flow Information Export" [RFC3917]. [RFC5101] then covers the Confusing as you already mentioned RFC 5101 before. I suggest modifying the introductory paragraph of this section such that it does not refer to RFC 5101 - or do not repeat it here. > details of the method for transporting IPFIX Data Records and > Templates via a congestion-aware transport protocol from an IPFIX > Exporting Process to an IPFIX Collecting Process. UDP is not congestion-aware. > "Information Model for IP Flow Information Export" [RFC5102] > describes the Information Elements used by IPFIX, including details > on Information Element naming, numbering, and data type encoding. > Finally, "IPFIX Applicability" [I-D.ietf-ipfix-as] describes the > various applications of the IPFIX protocol and their use of > information exported via IPFIX, and relates the IPFIX architecture t= o > other measurement architectures and frameworks. >=20 > In addition, "Exporting Type Information for IPFIX Information > Elements" [I-D.ietf-ipfix-exporting-type] specifies a method for > encoding Information Model properties within an IPFIX Message stream= =2E >=20 > This document references [RFC5101] and the architecture document for= > terminology, defines IPFIX File Writer and IPFIX File Reader in term= s > of the IPFIX Exporting Processes and IPFIX Collecting Process > definitions from [RFC5101], and extends the IPFIX Information Model > defined in [RFC5102] to provide new Information Elements for IPFIX > File metadata. It uses the method described in "Exporting Type > Information for IPFIX Information Elements" > [I-D.ietf-ipfix-exporting-type] document to support the self- > description of IPFIX Files containing enterprise-specific Informatio= n > Elements. >=20 >=20 > 2. Terminology >=20 > Terms used in this document that are defined in the Terminology > section of [RFC5101] are to be interpreted as defined there. One more sentence explaining that you define additional terms below. > IPFIX File: An IPFIX File is a serialized stream of IPFIX Messages= A stream usually is continuous while a file has a beginning and an end. Consider: series or sequence of IPFIX Messages. > stored on a filesystem. Any IPFIX Message stream that would be > considered valid when transported one or more of the specified > IPFIX transports (SCTP, TCP, or UDP) as defined in [RFC5101] is > considered an IPFIX File for purposes of this document; however, What do you mean by "for purposes of this document"? Possibly remove it? > this document extends that definition with recommendations on the= > construction of IPFIX Files that meet the requirements identified= > herein. Where is "herein"? in the file? in the document? > IPFIX File Reader: An IPFIX File Reader is a Process which reads > IPFIX Files from a filesystem, and is analogous to an IPFIX I'm not a native English speaker, yet "analogous to" sounds very vague as long as you do not say in which respect the analogy holds. I do not have a good idea how to express what you want to say. As a starting point, consider something like: an IPFIX File Reader is used as a replacement of an IPFIX Collection Process. > Collecting Process. An IPFIX File Reader MUST behave as an IPFIX= > Collecting Process as outlined in [RFC5101], except as modified b= y > this document. >=20 > IPFIX File Writer: An IPFIX File Writer is a process which wrirites > IPFIX Files to a filesystem, and is analogous to an IPFIX same as above > Exporting Process. An IPFIX File Writer MUST behave as an IPFIX > Exporting Process as outlined in [RFC5101], except as modified by= > this document. > 3. Design Overview >=20 > An IPFIX File, as defined by this document, is simply a stream > containing one or more IPFIX Messages serialized to some filesystem.= What is simple? Again, consider series or sequence instead of stream. > Though any set of valid IPFIX Messages can be serialized into an > IPFIX File, the specification proposes guidelines designed to ease > storage and retrieval of flow data using the format. Which format? > IPFIX Files contain only IPFIX Messages; any file metadata such as > checksums or export session details are stored using Options within > the IPFIX Message. This design has several advantages, including > complete compatibility with the IPFIX Protocol on the wire and free > manipulability of IPFIX Files through concatenation, appending, and > splitting (on IPFIX Message boundaries). A schematic of a typical > file is shown below: >=20 >=20 > +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+ > | IPFIX File | > | +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+ | > | | IPFIX Message | | > | | +-------------------------------+ | | > | | | Options Template Set | | | > | | | Options Template Record | | | > | | | . . . | | | > | | +-------------------------------+ | | > | | +-------------------------------+ | | > | | | Template Set | | | > | | | Template Record | | | > | | | . . . | | | > | | +-------------------------------+ | | > | +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+ | > | | IPFIX Message | | > | | +-------------------------------+ | | > | | | Data Set | | | > | | | Data Record | | | > | | | . . . | | | > | | +-------------------------------+ | | > | | +-------------------------------+ | | > | | | Data Set | | | > | | | Data Record | | | > | | | . . . | | | > | | +-------------------------------+ | | > | | . . . | | > | +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+ | > | . . . | > +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+ >=20 > Figure 1: Typical File Structure Looks a little bit as if the IPFIX File encapsulates the Messages with header and trailer. Consider: ^ +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+ : | IPFIX Message | : | +-------------------------------+ | : | | Options Template Set | | : | | Options Template Record | | : | | . . . | | : | +-------------------------------+ | : | +-------------------------------+ | : | | Template Set | | : | | Template Record | | : | | . . . | | tes > IPFIX Files to a filesystem, and is analogous to an IPFIX same as above > Exporting Process. An IPFIX File Writer MUST behave as an IPFIX > Exporting Process as outlined in [RFC5101], except as modified by= > this document. > 3. Design Overview >=20 > An IPFIX File, as defined by this document, is simply a stream > containing one or more IPFIX Messages serialized to some filesystem.= What is simple? Again, consider series or sequence instead of stream. > Though any set of valid IPFIX Messages can be serialized into an > IPFIX File, the specification proposes guidelines designed to ease > storage and retrieval of flow data using the format. Which format? > IPFIX Files contain only IPFIX Messages; any file metadata such as > checksums or export session details are stored using Options within > the IPFIX Message. This design has several advantages, including > complete compatibility with the IPFIX Protocol on the wire and free > manipulability of IPFIX Files through concatenation, appending, and > splitting (on IPFIX Message boundaries). A schematic of a typical > file is shown below: >=20 >=20 > +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+ > | IPFIX File | > | +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+ | > | | IPFIX Message | | > | | +-------------------------------+ | | > | | | Options Template Set | | | > | | | Options Template Record | | | > | | | . . . | | | > | | +-------------------------------+ | | > | | +-------------------------------+ | | > | | | Template Set | | | > | | | Template Record | | | > | | | . . . | | | > | | +-------------------------------+ | | > | +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+ | > | | IPFIX Message | | > | | +-------------------------------+ | | > | | | Data Set | | | > | | | Data Record | | | > | | | . . . | | | > | | +-------------------------------+ | | > | | +-------------------------------+ | | > | | | Data Set | | | > | | | Data Record | | | > | | | . . . | | | > | | +-------------------------------+ | | > | | . . . | | > | +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+ | > | . . . | > +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+ >=20 > Figure 1: Typical File Structure Looks a little bit as if the IPFIX File encapsulates the Messages with header and trailer. Consider: ^ +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+ : | IPFIX Message | : | +-------------------------------+ | : | | Options Template Set | | : | | Options Template Record | | : | | . . . | | : | +-------------------------------+ | : | +-------------------------------+ | : | | Template Set | | : | | Template Record | | : | | . . . | | : | +-------------------------------+ | : +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+ : | IPFIX Message | : | +-------------------------------+ | : | | Data Set | | : | | Data Record | | IPFIX File | | . . . | | : | +-------------------------------+ | : | +-------------------------------+ | : | | Data Set | | : | | Data Record | | : | | . . . | | : | +-------------------------------+ | : | . . . | : +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+ : . . . V > 4. Motivation > The variety of applications of flow data, and the variety of > presently deployed storage approaches, would seem to indicate the "would seem to indicate" =3D> indicate > need for a standard approach to flow storage with applicability > across the continuum of time scales over which flow data is stored. > A storage format based around flat files would best address the > variety of storage requirements. While much work has been done on > structured storage via RDBMS, relational database systems are not a > good basis for format standardization owing to the fact that their > internal data structures are generally private to a single > implementation and subject to change for internal reasons. Also, > there are a wide variety of operations available on flat files, and > external tools and standards can be leveraged to meet file-based flo= w > storage requirements. Further, flow data is often not very > semantically complicated, and is managed in very high volume; > therefore, an RDBMS-based flow storage system would not benefit much= > from the advantages of relational database technology. I would remove the last sentence. There is no need to argue against the usage of RDBMS. > Over the past decade XML markup has emerged as a new "universal" > representation format for structured data. It is intended to be > human-readable; indeed, that is one reason for its rapid adoption. > However XML has limited usefulness for representing network flow > data. Network flow data has a simple, repetitive, non-hierarchical > structure that does not benefit much from XML. An XML representatio= n > of flow data would be an essentially flat list of the attributes and= > their values for each flow record. >=20 > The XML approach to data encoding is very heavyweight when compared > to binary flow encoding. XML's use of start- and end-tags, and > plain-text encoding of the actual values, leads to significant > inefficiency in encoding size. Typical network flow datasets can > contain millions or billions of flows per hour of traffic > represented. Any increase in storage size per record can have > dramatic impact on flow data storage and transfer sizes. While data= > compression algorithms can partially remove the redundancy introduce= d > by XML encoding, they introduce additional overhead of their own. >=20 > A further problem is that XML processing tools require a full XML > parser. XML parsers are fully general and therefore complex, > resource-intensive and relatively slow, introducing significant > processing time overhead for large network-flow datasets. In > contrast, parsers for typical binary flow data encodings are simply > structured, since they only need to parse a very small header and > then have complete knowledge of all following fields for the > particular flow. These can then be read in a very efficient linear > fashion. I suggest shortening the text of the three paragraphs above. The argument th : | +-------------------------------+ | : +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+ : | IPFIX Message | : | +-------------------------------+ | : | | Data Set | | : | | Data Record | | IPFIX File | | . . . | | : | +-------------------------------+ | : | +-------------------------------+ | : | | Data Set | | : | | Data Record | | : | | . . . | | : | +-------------------------------+ | : | . . . | : +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+ : . . . V > 4. Motivation > The variety of applications of flow data, and the variety of > presently deployed storage approaches, would seem to indicate the "would seem to indicate" =3D> indicate > need for a standard approach to flow storage with applicability > across the continuum of time scales over which flow data is stored. > A storage format based around flat files would best address the > variety of storage requirements. While much work has been done on > structured storage via RDBMS, relational database systems are not a > good basis for format standardization owing to the fact that their > internal data structures are generally private to a single > implementation and subject to change for internal reasons. Also, > there are a wide variety of operations available on flat files, and > external tools and standards can be leveraged to meet file-based flo= w > storage requirements. Further, flow data is often not very > semantically complicated, and is managed in very high volume; > therefore, an RDBMS-based flow storage system would not benefit much= > from the advantages of relational database technology. I would remove the last sentence. There is no need to argue against the usage of RDBMS. > Over the past decade XML markup has emerged as a new "universal" > representation format for structured data. It is intended to be > human-readable; indeed, that is one reason for its rapid adoption. > However XML has limited usefulness for representing network flow > data. Network flow data has a simple, repetitive, non-hierarchical > structure that does not benefit much from XML. An XML representatio= n > of flow data would be an essentially flat list of the attributes and= > their values for each flow record. >=20 > The XML approach to data encoding is very heavyweight when compared > to binary flow encoding. XML's use of start- and end-tags, and > plain-text encoding of the actual values, leads to significant > inefficiency in encoding size. Typical network flow datasets can > contain millions or billions of flows per hour of traffic > represented. Any increase in storage size per record can have > dramatic impact on flow data storage and transfer sizes. While data= > compression algorithms can partially remove the redundancy introduce= d > by XML encoding, they introduce additional overhead of their own. >=20 > A further problem is that XML processing tools require a full XML > parser. XML parsers are fully general and therefore complex, > resource-intensive and relatively slow, introducing significant > processing time overhead for large network-flow datasets. In > contrast, parsers for typical binary flow data encodings are simply > structured, since they only need to parse a very small header and > then have complete knowledge of all following fields for the > particular flow. These can then be read in a very efficient linear > fashion. I suggest shortening the text of the three paragraphs above. The argument thatat parsing binary IPFIX Messages is simpler than parsing XML-encoded flow data is not convincing. A better argument is that parsers for IPFIX Messages exist anyway and can be reused. > This leads us to propose the IPFIX Message format as the basis for a= > new flow data file format. The IPFIX working group, in defining the= > IPFIX protocol, has already defined an information model and data > formatting rules for representation of flow data. Especially at > shorter time scales, when a file is a unit of data interchange, the > filesystem may be viewed as simply another IPFIX Message transport > between processes. This format is especially well suited to > representing flow data, as it was designed specifically for flow dat= a > export; it is easily extensible unlike ad-hoc serialization, and > compact unlike XML. In addition, IPFIX is an IETF standard for the > export and collection of flow data; using a common format for storag= e > and analysis at the collection side allows implementors to use > substantially the same information model and data formatting > implementation for transport as well as storage. > 5.2. Self Description > The IPFIX Message format is partially self-describing; that is, IPFI= X > Templates containing only IANA-assigned Information Elements can be > completely interpreted according to the IPFIX Information Model > without additional external data. I do not think that the IPFIX Message format is self-describing because a single IPFIX Message usually is not self-describing. You need all messages sequence of a Transport Session to get all the required Templates etc. > However, Templates containing private information elements lack > detailed type and semantic information; a Collecting Process > receiving data described by a template containing private Informatio= n > Elements it does not understand can only treat the data contained > within those Information Elements as octet arrays. To be fully self= - > describing, enterprise-specific Information Elements must be Unless this was a MUST, I would replace it by "can". > additionally described via IPFIX Options according to the Informatio= n > Element Type Options Template defined in "Exporting Type Information= > for IPFIX Information Elements" [I-D.ietf-ipfix-exporting-type]. > 5.4. Indexing and Searching > The IPFIX Message format has no direct support for indexing. Do you want so search within an IPFIX Message or within an IPFIX File (=3D= sequence of IPFIX Messages)? > 5.5. Data Integrity > Note that more advanced error correction is almost certainly best omit "almost certainly" > handled at a layer below that addressed by this document. Error > correction is a topic well addressed by the storage industry in > general (e.g. by RAID and other technologies), and by specifying a > flow storage format based upon files, we can leverage these features= > to meet this requirement. > 6.2. Storage of NetFlow V9-collected Flow Data >=20 > Although the IPFIX protocol is based on the Cisco Netflow Services, > Version 9 (NetFlow V9) protocol [RFC3954], the two have diverged I would not say that IPFIX is based on NetFlow V9. It's rather the successor. > since work began on IPFIX. However, since the NetFlow V9 informatio= n > model is a compatible subset of the IPFIX information model, it is > possible to use IPFIX files to store collected NetFlow V9 flow data.= > This approach may be particularly useful in multi-vendor, multi- > protocol collection infrastructures using both NetFlow V9 and IPFIX > to export flow data. > 7. Detailed Description >=20 > An IPFIX File, as introduced in Section 3 and elaborated below, is a= t > its core simply an IPFIX Message stream serialized to some > filesystem. Any valid serialized IPFIX Message stream MUST be > accepted by a File Reader as a valid IPFIX file. In this way, the IPFIX File > filesystem is simply treated as another IPFIX transport alo parsing binary IPFIX Messages is simpler than parsing XML-encoded flow data is not convincing. A better argument is that parsers for IPFIX Messages exist anyway and can be reused. > This leads us to propose the IPFIX Message format as the basis for a= > new flow data file format. The IPFIX working group, in defining the= > IPFIX protocol, has already defined an information model and data > formatting rules for representation of flow data. Especially at > shorter time scales, when a file is a unit of data interchange, the > filesystem may be viewed as simply another IPFIX Message transport > between processes. This format is especially well suited to > representing flow data, as it was designed specifically for flow dat= a > export; it is easily extensible unlike ad-hoc serialization, and > compact unlike XML. In addition, IPFIX is an IETF standard for the > export and collection of flow data; using a common format for storag= e > and analysis at the collection side allows implementors to use > substantially the same information model and data formatting > implementation for transport as well as storage. > 5.2. Self Description > The IPFIX Message format is partially self-describing; that is, IPFI= X > Templates containing only IANA-assigned Information Elements can be > completely interpreted according to the IPFIX Information Model > without additional external data. I do not think that the IPFIX Message format is self-describing because a single IPFIX Message usually is not self-describing. You need all messages sequence of a Transport Session to get all the required Templates etc. > However, Templates containing private information elements lack > detailed type and semantic information; a Collecting Process > receiving data described by a template containing private Informatio= n > Elements it does not understand can only treat the data contained > within those Information Elements as octet arrays. To be fully self= - > describing, enterprise-specific Information Elements must be Unless this was a MUST, I would replace it by "can". > additionally described via IPFIX Options according to the Informatio= n > Element Type Options Template defined in "Exporting Type Information= > for IPFIX Information Elements" [I-D.ietf-ipfix-exporting-type]. > 5.4. Indexing and Searching > The IPFIX Message format has no direct support for indexing. Do you want so search within an IPFIX Message or within an IPFIX File (=3D= sequence of IPFIX Messages)? > 5.5. Data Integrity > Note that more advanced error correction is almost certainly best omit "almost certainly" > handled at a layer below that addressed by this document. Error > correction is a topic well addressed by the storage industry in > general (e.g. by RAID and other technologies), and by specifying a > flow storage format based upon files, we can leverage these features= > to meet this requirement. > 6.2. Storage of NetFlow V9-collected Flow Data >=20 > Although the IPFIX protocol is based on the Cisco Netflow Services, > Version 9 (NetFlow V9) protocol [RFC3954], the two have diverged I would not say that IPFIX is based on NetFlow V9. It's rather the successor. > since work began on IPFIX. However, since the NetFlow V9 informatio= n > model is a compatible subset of the IPFIX information model, it is > possible to use IPFIX files to store collected NetFlow V9 flow data.= > This approach may be particularly useful in multi-vendor, multi- > protocol collection infrastructures using both NetFlow V9 and IPFIX > to export flow data. > 7. Detailed Description >=20 > An IPFIX File, as introduced in Section 3 and elaborated below, is a= t > its core simply an IPFIX Message stream serialized to some > filesystem. Any valid serialized IPFIX Message stream MUST be > accepted by a File Reader as a valid IPFIX file. In this way, the IPFIX File > filesystem is simply treated as another IPFIX transport alongngside > SCTP, TCP, and UDP. In contrast to normal IPFIX operation, the time= > between a File Writer writing an IPFIX Message stream to a File and = a > File Reader reading it can be extremely variable. In other words, > this notional file transport has unusually high latency, as the File= > Reader and File Writer do not necessarily run at the same time. >=20 > This section specifies the detailed actions of File Readers and File= > Writers in handling IPFIX Files, and further specifies actions of > File Writers in specific use cases. Unless otherwise specified > herein, where appropriate IPFIX File Writers MUST behave as IPFIX remove "where"? > Exporting Processes, and IPFIX File Readers MUST behave as IPFIX > Collecting Processes. > 7.2. File Writer Specification >=20 > While any valid serialized IPFIX Message stream is a valid IPFIX > File, the following recommendations will improve representation > simplicity and read performance in the general case, where possible.= >=20 > File Writers SHOULD emit each Template Set or Options Template Set t= o > appear in the file before any Data Set described by the Templates > within that Set, to ensure the File Reader can decode every Data Set= > without waiting to process subsequent Templates or Options Templates= =2E Is the File Reader allowed to wait for a Template anyway? In 7.1, you wrote that Templates can be redefined without withdrawal as it is the case in UDP sessions. In this case, decoding errors may occur if the Collector decodes previous Data Sets with the new Template. > File Writers SHOULD emit Data Records described by Options Templates= > to appear in the file before any Data Records which depend on the > scopes defined by those options. >=20 > File Writers SHOULD use Template Withdrawals to withdraw Templates i= f > template IDs need to be reused. In this case, the new Templates > reusing those IDs SHOULD appear directly in the file after the > Template Withdrawals making the IDs available for reuse. Template > Withdrawals SHOULD NOT be used unless necessary to reuse template Template > IDs. I do not understand the restrictions "SHOULD appear directly after" and "SHOULD NOT be used unless necessary". If there is no good reason, I would remove these restrictions. > Each IPFIX File is generally synonymous with a single Transport > Session. File Writers SHOULD store the Templates and Options > required to decode the data within the File in the File itself, and > File Readers SHOULD NOT use Templates or Options defined in one file= > to decode or interpret Data Sets in another. MUST the File Writer repeat Templates and Options in each File if the Transport Session is split into several files? >=20 > File Writers SHOULD write IPFIX Messages within an IPFIX File in > ascending Export Time order. >=20 > File Writers MAY write Data Records to an IPFIX File in any order. > However, File Writers that write flow records to an IPFIX File in > flowStartTime or flowEndTime order SHOULD be consistent in this > ordering within each File. The last sentence does not make sense to me since it is always true: If the IPFIX File Writer writes record ordered by time, then, of course, the order is consistent. > 7.3.1. Collocating a File Writer with a Collecting Process >=20 > When collocating a File Writer with an IPFIX Collecting Process for > archival storage of collected data in IPFIX Files as described in > Section 6.1, the following recommendations may improve the usefulnes= s > of the stored data. >=20 > The simplest way for a to store the data collected in a single "for a" what? > Transport Session is to simply write the incoming IPFIX Messages to > an IPFIX File as they are collected. However, the resulting files > will lack information about the IPFIX Transport Session used to > export them, such as the network addresses of the Exporting and > Collecting Processes and the protocols used to transport theside > SCTP, TCP, and UDP. In contrast to normal IPFIX operation, the time= > between a File Writer writing an IPFIX Message stream to a File and = a > File Reader reading it can be extremely variable. In other words, > this notional file transport has unusually high latency, as the File= > Reader and File Writer do not necessarily run at the same time. >=20 > This section specifies the detailed actions of File Readers and File= > Writers in handling IPFIX Files, and further specifies actions of > File Writers in specific use cases. Unless otherwise specified > herein, where appropriate IPFIX File Writers MUST behave as IPFIX remove "where"? > Exporting Processes, and IPFIX File Readers MUST behave as IPFIX > Collecting Processes. > 7.2. File Writer Specification >=20 > While any valid serialized IPFIX Message stream is a valid IPFIX > File, the following recommendations will improve representation > simplicity and read performance in the general case, where possible.= >=20 > File Writers SHOULD emit each Template Set or Options Template Set t= o > appear in the file before any Data Set described by the Templates > within that Set, to ensure the File Reader can decode every Data Set= > without waiting to process subsequent Templates or Options Templates= =2E Is the File Reader allowed to wait for a Template anyway? In 7.1, you wrote that Templates can be redefined without withdrawal as it is the case in UDP sessions. In this case, decoding errors may occur if the Collector decodes previous Data Sets with the new Template. > File Writers SHOULD emit Data Records described by Options Templates= > to appear in the file before any Data Records which depend on the > scopes defined by those options. >=20 > File Writers SHOULD use Template Withdrawals to withdraw Templates i= f > template IDs need to be reused. In this case, the new Templates > reusing those IDs SHOULD appear directly in the file after the > Template Withdrawals making the IDs available for reuse. Template > Withdrawals SHOULD NOT be used unless necessary to reuse template Template > IDs. I do not understand the restrictions "SHOULD appear directly after" and "SHOULD NOT be used unless necessary". If there is no good reason, I would remove these restrictions. > Each IPFIX File is generally synonymous with a single Transport > Session. File Writers SHOULD store the Templates and Options > required to decode the data within the File in the File itself, and > File Readers SHOULD NOT use Templates or Options defined in one file= > to decode or interpret Data Sets in another. MUST the File Writer repeat Templates and Options in each File if the Transport Session is split into several files? >=20 > File Writers SHOULD write IPFIX Messages within an IPFIX File in > ascending Export Time order. >=20 > File Writers MAY write Data Records to an IPFIX File in any order. > However, File Writers that write flow records to an IPFIX File in > flowStartTime or flowEndTime order SHOULD be consistent in this > ordering within each File. The last sentence does not make sense to me since it is always true: If the IPFIX File Writer writes record ordered by time, then, of course, the order is consistent. > 7.3.1. Collocating a File Writer with a Collecting Process >=20 > When collocating a File Writer with an IPFIX Collecting Process for > archival storage of collected data in IPFIX Files as described in > Section 6.1, the following recommendations may improve the usefulnes= s > of the stored data. >=20 > The simplest way for a to store the data collected in a single "for a" what? > Transport Session is to simply write the incoming IPFIX Messages to > an IPFIX File as they are collected. However, the resulting files > will lack information about the IPFIX Transport Session used to > export them, such as the network addresses of the Exporting and > Collecting Processes and the protocols used to transport them.m. In > this case, if information about the Transport Session is required, > the File Writer SHOULD store a single IPFIX Transport Session in an > IPFIX File and SHOULD record information about the Transport Session= > using the Export Session Details Options Template described in > Section 8.1.3. > 7.3.2. Collocating a File Writer with a Metering Process >=20 > Note that File Writers may also be collocated directly with IPFIX Remove "note that" > Metering Processes, for writing measured information directly to dis= k > without intermediate IPFIX Exporting or Collecting Processes. This > arrangement may be particularly useful when providing data to an > analysis environment with an IPFIX File based workflow, or when > testing Metering Processes during development. >=20 > When collocating a File Writer with a Metering Process, note that Remove "note that" > Information Elements associated with Exporting or Collecting > Processes are meaningless, and SHOULD NOT appear in the Export > Session Details Options Template described in Section 8.1.3 or the > Message Details Options Template described in Section 8.1.4. >=20 > When collocating a File Writer with an Exporting Process, the Export= Collocating File Writer and Exporting Process? Do you mean Metering Proce= ss? > Time of each Message SHOULD be the time at which the first Data > Record in the Message was received from the Exporting Process. Hm, I would rather say that the Export Time should be the time of writing the Message into the File. This is typically the time of the LAST Data Record in the Message. When exported by an Exporting Process, the Export Time is never before the time of the last Data Record either. > 7.3.3. Using IPFIX Files for Archival Storage >=20 > While in the general case File Writers should store one Transport > Session per IPFIX File, some applications storing large collections > of data over long periods of time may benefit from the ability to > treat a collection of IPFIX Files as a single Transport Session. A > File Writer MAY be configurable to write data from a single Transpor= t > Session into multiple IPFIX Files; however, File Writers supporting > such a configuration option MUST provide a configuration option to > support one-file-per-session behavior for interoperability purposes.= I do not think that a mandatory one-file-per-session option makes sense. Sometimes, it is not possible to enforce one-file-per-session, for example if the disk is full. What do you do then? > File Writers compressing or encrypting archival data and File Reader= s > reading compressed or encrypted archival data SHOULD follow the > recommendations in Section 9. >=20 > 7.3.4. Using IPFIX Files as Documents >=20 > When IPFIX Files are used as documents, to store a set of flows > relevant to query, investigation, or other common context, or for th= e > publication of flow data sets relevant to network research, each Fil= e > MUST be readable as a single Transport Session, self-contained and > making no reference to metadata stored in separate Files, in order t= o Maybe you should explain what you mean by metadata. > ensure interoperability. >=20 > When writing Files to be used as documents, File Writers may emit th= e > special Data Records described by Options Templates before any other= > Data Records in the File, in the following order, to ease the > inspection and use of documents by File Readers: >=20 > o Time Window records described by the File Time Window Options > Template as defined in Section 8.1.2 below; followed by >=20 > o commonPropertiesId definitions as described in "Reducing > Redundancy in IPFIX and PSAMP Reports" > [I-D.ietf-ipfix-reducing-redundancy]; followed by >=20 > o Information Element Type Records as described in "Exporting Type > Information for IPFIX Information Elements" > [I-D.ietf-ipfix-exporting-type]; followed by Exporting type information should In > this case, if information about the Transport Session is required, > the File Writer SHOULD store a single IPFIX Transport Session in an > IPFIX File and SHOULD record information about the Transport Session= > using the Export Session Details Options Template described in > Section 8.1.3. > 7.3.2. Collocating a File Writer with a Metering Process >=20 > Note that File Writers may also be collocated directly with IPFIX Remove "note that" > Metering Processes, for writing measured information directly to dis= k > without intermediate IPFIX Exporting or Collecting Processes. This > arrangement may be particularly useful when providing data to an > analysis environment with an IPFIX File based workflow, or when > testing Metering Processes during development. >=20 > When collocating a File Writer with a Metering Process, note that Remove "note that" > Information Elements associated with Exporting or Collecting > Processes are meaningless, and SHOULD NOT appear in the Export > Session Details Options Template described in Section 8.1.3 or the > Message Details Options Template described in Section 8.1.4. >=20 > When collocating a File Writer with an Exporting Process, the Export= Collocating File Writer and Exporting Process? Do you mean Metering Proce= ss? > Time of each Message SHOULD be the time at which the first Data > Record in the Message was received from the Exporting Process. Hm, I would rather say that the Export Time should be the time of writing the Message into the File. This is typically the time of the LAST Data Record in the Message. When exported by an Exporting Process, the Export Time is never before the time of the last Data Record either. > 7.3.3. Using IPFIX Files for Archival Storage >=20 > While in the general case File Writers should store one Transport > Session per IPFIX File, some applications storing large collections > of data over long periods of time may benefit from the ability to > treat a collection of IPFIX Files as a single Transport Session. A > File Writer MAY be configurable to write data from a single Transpor= t > Session into multiple IPFIX Files; however, File Writers supporting > such a configuration option MUST provide a configuration option to > support one-file-per-session behavior for interoperability purposes.= I do not think that a mandatory one-file-per-session option makes sense. Sometimes, it is not possible to enforce one-file-per-session, for example if the disk is full. What do you do then? > File Writers compressing or encrypting archival data and File Reader= s > reading compressed or encrypted archival data SHOULD follow the > recommendations in Section 9. >=20 > 7.3.4. Using IPFIX Files as Documents >=20 > When IPFIX Files are used as documents, to store a set of flows > relevant to query, investigation, or other common context, or for th= e > publication of flow data sets relevant to network research, each Fil= e > MUST be readable as a single Transport Session, self-contained and > making no reference to metadata stored in separate Files, in order t= o Maybe you should explain what you mean by metadata. > ensure interoperability. >=20 > When writing Files to be used as documents, File Writers may emit th= e > special Data Records described by Options Templates before any other= > Data Records in the File, in the following order, to ease the > inspection and use of documents by File Readers: >=20 > o Time Window records described by the File Time Window Options > Template as defined in Section 8.1.2 below; followed by >=20 > o commonPropertiesId definitions as described in "Reducing > Redundancy in IPFIX and PSAMP Reports" > [I-D.ietf-ipfix-reducing-redundancy]; followed by >=20 > o Information Element Type Records as described in "Exporting Type > Information for IPFIX Information Elements" > [I-D.ietf-ipfix-exporting-type]; followed by Exporting type information should bebe before reducing redundancy since common properties may contain enterprise specific IEs. > o Export Session details records described by the Export Session > Details Options Template as defined in Section 8.1.3 below. Do we really need to specify the order? I think it suffices to specify that the Time Window should appear at the beginning of the file. > The Export Time of each Message within a File used as a document > SHOULD be the time at which the Message was written by the File > Writer. Agreed, but this contradicts what is written in 7.3.2. > If an IPFIX File used as a document uses the technique described in > "Reducing Redundancy in IPFIX and PSAMP Reports" > [I-D.ietf-ipfix-reducing-redundancy] AND all of the non-Options > Templates in the File contain the commonPropertiesId Information > Element, a File Reader MAY assume the set of commonPropertiesId > definitions provides a complete table of contents for the File for > searching purposes. There is a problem: Templates may be redefined in the middle of the file. How does the File Reader now that all Templates ever defined in the middle of the File use the same set of common properties? > 7.3.5. Using IPFIX Files for Testing >=20 > IPFIX Files can be used for testing IPFIX Collecting Processes in tw= o > ways. First, IPFIX Files can be used to store specific flow data fo= r > regression and stress testing of Collectors; there are no special > considerations for IPFIX Files used in this way. >=20 > Second, IPFIX Files are useful for storing reference messages which > do not comply to the IPFIX Protocol in order to test the error > handling and recovery behavior of Collectors. Of course, IPFIX File= s > intended to be used in this application necessarily MAY violate any > of of the specifications in this document or in [RFC5101], and such duplicated "of" > files MUST NOT be transmitted to Collecting Processes or given as > input File Readers not under test. >=20 > Note that an extremely simple IPFIX Exporting Process may be crafted= > for testing purposes by simply reading an IPFIX File and transmittin= g > it directly to a Collecting Process. Similarly, an extremely simple= > Collecting Process may be crafted for testing purposes by simply > accepting connections and/or IPFIX Messages from Exporting Processes= > and writing the session's message stream to an IPFIX File. What is the reason for testing such a simple Collecting Process? It will write the invalid messages into a file. > 8.1. Recommended Options Templates for IPFIX Files >=20 > The following Options Templates allow IPFIX Message streams to meet > the requirements outlined above without extension to the message > format or protocol. They are defined in terms of existing > Information Elements defined in [RFC5102], the Information Elements > defined in "Exporting Type Information for IPFIX Information > Elements" [I-D.ietf-ipfix-exporting-type], as well as Information > Elements defined in Section 8.2. IPFIX File Readers and Writers > SHOULD support these options templates as defined below. Options Templates > In addition, IPFIX File Readers and Writers SHOULD support the > Options Templates defined in "Exporting Type Information for IPFIX > Information Elements" [I-D.ietf-ipfix-exporting-type] in order to > support self-description of enterprise-specific Information Elements= =2E I understand that a File Writer should add exporting type information if necessary and available. So, a writer should support this feature. But what about the File Reader? I do not think that supporting the reading of exporting type information options requires special capabilities that need to be supported. > 8.1.1. Message Checksum Options Template >=20 > The Message Checksum Options Template specifies the structure of a > Data Record for attaching an MD5 message checksum to an IPFIX > Message. An MD5 message checksum as described MAY be used if long- > term da before reducing redundancy since common properties may contain enterprise specific IEs. > o Export Session details records described by the Export Session > Details Options Template as defined in Section 8.1.3 below. Do we really need to specify the order? I think it suffices to specify that the Time Window should appear at the beginning of the file. > The Export Time of each Message within a File used as a document > SHOULD be the time at which the Message was written by the File > Writer. Agreed, but this contradicts what is written in 7.3.2. > If an IPFIX File used as a document uses the technique described in > "Reducing Redundancy in IPFIX and PSAMP Reports" > [I-D.ietf-ipfix-reducing-redundancy] AND all of the non-Options > Templates in the File contain the commonPropertiesId Information > Element, a File Reader MAY assume the set of commonPropertiesId > definitions provides a complete table of contents for the File for > searching purposes. There is a problem: Templates may be redefined in the middle of the file. How does the File Reader now that all Templates ever defined in the middle of the File use the same set of common properties? > 7.3.5. Using IPFIX Files for Testing >=20 > IPFIX Files can be used for testing IPFIX Collecting Processes in tw= o > ways. First, IPFIX Files can be used to store specific flow data fo= r > regression and stress testing of Collectors; there are no special > considerations for IPFIX Files used in this way. >=20 > Second, IPFIX Files are useful for storing reference messages which > do not comply to the IPFIX Protocol in order to test the error > handling and recovery behavior of Collectors. Of course, IPFIX File= s > intended to be used in this application necessarily MAY violate any > of of the specifications in this document or in [RFC5101], and such duplicated "of" > files MUST NOT be transmitted to Collecting Processes or given as > input File Readers not under test. >=20 > Note that an extremely simple IPFIX Exporting Process may be crafted= > for testing purposes by simply reading an IPFIX File and transmittin= g > it directly to a Collecting Process. Similarly, an extremely simple= > Collecting Process may be crafted for testing purposes by simply > accepting connections and/or IPFIX Messages from Exporting Processes= > and writing the session's message stream to an IPFIX File. What is the reason for testing such a simple Collecting Process? It will write the invalid messages into a file. > 8.1. Recommended Options Templates for IPFIX Files >=20 > The following Options Templates allow IPFIX Message streams to meet > the requirements outlined above without extension to the message > format or protocol. They are defined in terms of existing > Information Elements defined in [RFC5102], the Information Elements > defined in "Exporting Type Information for IPFIX Information > Elements" [I-D.ietf-ipfix-exporting-type], as well as Information > Elements defined in Section 8.2. IPFIX File Readers and Writers > SHOULD support these options templates as defined below. Options Templates > In addition, IPFIX File Readers and Writers SHOULD support the > Options Templates defined in "Exporting Type Information for IPFIX > Information Elements" [I-D.ietf-ipfix-exporting-type] in order to > support self-description of enterprise-specific Information Elements= =2E I understand that a File Writer should add exporting type information if necessary and available. So, a writer should support this feature. But what about the File Reader? I do not think that supporting the reading of exporting type information options requires special capabilities that need to be supported. > 8.1.1. Message Checksum Options Template >=20 > The Message Checksum Options Template specifies the structure of a > Data Record for attaching an MD5 message checksum to an IPFIX > Message. An MD5 message checksum as described MAY be used if long- > term datata integrity is important to the application. The described why only "long-term" data integrity? > Data Record MUST appear only once per IPFIX Message, but MAY appear > anywhere within the Message. > 8.1.2. File Time Window Options Template >=20 > The File Time Window Options Template specifies the structure of a > Data Record for attaching a time window to an IPFIX File; this Data > Record is referred to as a time window record. A time window record= > defines the earliest flow start time and the latest flow end time of= > the flow records within a File. One and only one time window record= > MAY appear within an IPFIX File if the time window information is > available; a File Writer MUST NOT write more than one time window > record to an IPFIX File. A File Writer that writes a time window > record to a File MUST NOT write any Flow with a start time before th= e > beginning of the window or an end time after the end of the window t= o > that File. Shouldn't the time window record be located at the beginning of the file? Yet this requires that we are able to prepend data to the file. Anyway, a SHOULD would be ok, in my opinion. > The template SHOULD contain the following Information Elements: >=20 > +---------------------+---------------------------------------------= + > | IE | Description = | > +---------------------+---------------------------------------------= + > | sessionScope | A marker denoting this Option applies to = | > | [scope] | the whole IPFIX Transport Session (i.e., = | > | | IPFIX File); content is ignored. This = | > | | Information Element MUST be defined as a = | > | | Scope Field. = | > | minFlowStartSeconds | The start time of the earliest flow in the = | > | | Transport Session (i.e., File) in epoch = | > | | seconds. = | > | maxFlowEndSeconds | The end time of the latest flow in the = | > | | Transport Session (i.e., File) in epoch = | > | | seconds. = | > +---------------------+---------------------------------------------= + sessionScope is ambiguous if the session is split into multiple files. I assume that in this case, the time window record describes the limits of a single file? Is there a reason for having seconds and not milliseconds as alternative?= What can we do if the flow start or end timestamp is missing in the recor= ds? In the case of PSAMP, we have observationTimeSeconds etc. and it makes sense to use this timestamp instead. > 8.1.3. Export Session Details Options Template >=20 > The Export Session Details Options Template specifies the structure > of a Data Record for recording the details of an IPFIX Transport > Session in an IPFIX File. It is intended for use in storing a singl= e > complete IPFIX Transport Session in a single IPFIX File. The Why not use it when a session is stored in multiple files as well? > described Data Record SHOULD appear only once in a given IPFIX File.= >=20 > 8.2. Recommended Information Elements for IPFIX Files >=20 > The following Information Elements are used by the options templates= > in Section 8.1 to allow IPFIX Message streams to meet the > requirements outlined above without extension of the message format > or protocol. IPFIX File Readers and Writers SHOULD support these > Information Elements as defined below. >=20 > In addition, IPFIX File Readers and Writers SHOULD support the > Information Elements defined in "Exporting Type Information for IPFI= X > Information Elements" [I-D.ietf-ipfix-exporting-type] in order to > support full self-description of Information Elements. I would remove this paragraph. You have already stated above that the writer integrity is important to the application. The described why only "long-term" data integrity? > Data Record MUST appear only once per IPFIX Message, but MAY appear > anywhere within the Message. > 8.1.2. File Time Window Options Template >=20 > The File Time Window Options Template specifies the structure of a > Data Record for attaching a time window to an IPFIX File; this Data > Record is referred to as a time window record. A time window record= > defines the earliest flow start time and the latest flow end time of= > the flow records within a File. One and only one time window record= > MAY appear within an IPFIX File if the time window information is > available; a File Writer MUST NOT write more than one time window > record to an IPFIX File. A File Writer that writes a time window > record to a File MUST NOT write any Flow with a start time before th= e > beginning of the window or an end time after the end of the window t= o > that File. Shouldn't the time window record be located at the beginning of the file? Yet this requires that we are able to prepend data to the file. Anyway, a SHOULD would be ok, in my opinion. > The template SHOULD contain the following Information Elements: >=20 > +---------------------+---------------------------------------------= + > | IE | Description = | > +---------------------+---------------------------------------------= + > | sessionScope | A marker denoting this Option applies to = | > | [scope] | the whole IPFIX Transport Session (i.e., = | > | | IPFIX File); content is ignored. This = | > | | Information Element MUST be defined as a = | > | | Scope Field. = | > | minFlowStartSeconds | The start time of the earliest flow in the = | > | | Transport Session (i.e., File) in epoch = | > | | seconds. = | > | maxFlowEndSeconds | The end time of the latest flow in the = | > | | Transport Session (i.e., File) in epoch = | > | | seconds. = | > +---------------------+---------------------------------------------= + sessionScope is ambiguous if the session is split into multiple files. I assume that in this case, the time window record describes the limits of a single file? Is there a reason for having seconds and not milliseconds as alternative?= What can we do if the flow start or end timestamp is missing in the recor= ds? In the case of PSAMP, we have observationTimeSeconds etc. and it makes sense to use this timestamp instead. > 8.1.3. Export Session Details Options Template >=20 > The Export Session Details Options Template specifies the structure > of a Data Record for recording the details of an IPFIX Transport > Session in an IPFIX File. It is intended for use in storing a singl= e > complete IPFIX Transport Session in a single IPFIX File. The Why not use it when a session is stored in multiple files as well? > described Data Record SHOULD appear only once in a given IPFIX File.= >=20 > 8.2. Recommended Information Elements for IPFIX Files >=20 > The following Information Elements are used by the options templates= > in Section 8.1 to allow IPFIX Message streams to meet the > requirements outlined above without extension of the message format > or protocol. IPFIX File Readers and Writers SHOULD support these > Information Elements as defined below. >=20 > In addition, IPFIX File Readers and Writers SHOULD support the > Information Elements defined in "Exporting Type Information for IPFI= X > Information Elements" [I-D.ietf-ipfix-exporting-type] in order to > support full self-description of Information Elements. I would remove this paragraph. You have already stated above that the writer s should support exporting type information options. This implies the support of the corresponding IEs. For other purposes, the exporting type information IEs are not required. > 8.2.3. maxExportSeconds >=20 > Description: The absolute Export Time of the latest IPFIX Message > within the scope containing this Information Element. This > Information Element SHOULD be bound to its containing IPFIX > Transport Session (i.e., File) via an options record and the > sessionScope Information Element, as defined below, and SHOULD > appear only once in a given IPFIX File. session !=3D file in the case of multiple files per session > 10.1. Encapsulation of Non-IPFIX Data in IPFIX Files >=20 > At times it may be useful to export or store non-IPFIX data inline i= n > an IPFIX File or Message stream. To do this cleanly, this data must= > be encapsulated into IPFIX Messages so that an IPFIX File Reader or > Collecting Process can handle it without any need to interpret it. > At the same time, this data must not be changed during transmission > or storage. The opaqueOctets Information Element as defined in > Section 8.2.9 is provided for this encapsulation. I wonder if the standardization of opaqueOctets makes sense at all. If a vendor wants to add such information, he can specify an enterprise-specific IE for this purpose with optional type information. With opaqueOctets, we never know which kind of data is contained in the field. And we lose one important advantage of IE which is the implicit semantics of any IE. > Processing the encapsulated non-IPFIX data is left to a separate > processing mechanisms that can identify encapsulated non-IPFIX data > in an IPFIX message stream, but need not have any other IPFIX > handling capability, except the ability to skip over all IPFIX > messages that do not encapsulate non-IPFIX data. >=20 > The Message Checksum Options Template, described in Section 8.1.1 ma= y > be used as a uniform mechanism to identify errors within encapsulate= d > data. >=20 > Note that this mechanism can only encapsulate data objects up to > 65,515 octets in length. If the space available in one IPFIX Messag= e > is not enough for the amount of data to be encapsulated, then the > data must be broken into smaller segments that are encapsulated into= > consecutive IPFIX Messages. Any additional structuring or semantics= > of the raw data is outside the scope of IPFIX and must be implemente= d > within the encapsulated binary data itself. Furthermore, the raw > encapsulated data cannot be assumed by an IPFIX File Reader to have > any specific format. >=20 > 10.2. Encapsulation of IPFIX Files within Other File Formats >=20 > Consequently, it may also be useful to reverse the encapsulation, > that is, to export or store IPFIX data inline within a non-IPFIX fil= e > or data stream. This makes sense when the other file format is not > compatible with the encapsulation described above in Section 10.1. > Generally speaking, the encapsulation here will be specific to the > format of the containing file. For example, IPFIX files may be > embedded in XML elements using hex or Base64 encoding, or in raw > binary files using start and end delimiters or some form of run- > length encoding. As there are as many potential encapsulations here= > as there are potential file formats, the specifics of each are out o= f > scope for this specification. I would remove the whole section. It is completely out of scope. > B.1.3. Template Format >=20 > Template FlowSets in NetFlow V9 support a subset of functionality of= > those in IPFIX. Specifically, NetFlow V9 does not have any support > for vendor-specific Information Elements as IPFIX does, so there is > no enterprise bit or facility for associating a private enterprise > number with an information element. NetFlow V9 does not support variable length fields. > Options Template FlowSets in NetFlow V9 are similar thould support exporting type information options. This implies the support of the corresponding IEs. For other purposes, the exporting type information IEs are not required. > 8.2.3. maxExportSeconds >=20 > Description: The absolute Export Time of the latest IPFIX Message > within the scope containing this Information Element. This > Information Element SHOULD be bound to its containing IPFIX > Transport Session (i.e., File) via an options record and the > sessionScope Information Element, as defined below, and SHOULD > appear only once in a given IPFIX File. session !=3D file in the case of multiple files per session > 10.1. Encapsulation of Non-IPFIX Data in IPFIX Files >=20 > At times it may be useful to export or store non-IPFIX data inline i= n > an IPFIX File or Message stream. To do this cleanly, this data must= > be encapsulated into IPFIX Messages so that an IPFIX File Reader or > Collecting Process can handle it without any need to interpret it. > At the same time, this data must not be changed during transmission > or storage. The opaqueOctets Information Element as defined in > Section 8.2.9 is provided for this encapsulation. I wonder if the standardization of opaqueOctets makes sense at all. If a vendor wants to add such information, he can specify an enterprise-specific IE for this purpose with optional type information. With opaqueOctets, we never know which kind of data is contained in the field. And we lose one important advantage of IE which is the implicit semantics of any IE. > Processing the encapsulated non-IPFIX data is left to a separate > processing mechanisms that can identify encapsulated non-IPFIX data > in an IPFIX message stream, but need not have any other IPFIX > handling capability, except the ability to skip over all IPFIX > messages that do not encapsulate non-IPFIX data. >=20 > The Message Checksum Options Template, described in Section 8.1.1 ma= y > be used as a uniform mechanism to identify errors within encapsulate= d > data. >=20 > Note that this mechanism can only encapsulate data objects up to > 65,515 octets in length. If the space available in one IPFIX Messag= e > is not enough for the amount of data to be encapsulated, then the > data must be broken into smaller segments that are encapsulated into= > consecutive IPFIX Messages. Any additional structuring or semantics= > of the raw data is outside the scope of IPFIX and must be implemente= d > within the encapsulated binary data itself. Furthermore, the raw > encapsulated data cannot be assumed by an IPFIX File Reader to have > any specific format. >=20 > 10.2. Encapsulation of IPFIX Files within Other File Formats >=20 > Consequently, it may also be useful to reverse the encapsulation, > that is, to export or store IPFIX data inline within a non-IPFIX fil= e > or data stream. This makes sense when the other file format is not > compatible with the encapsulation described above in Section 10.1. > Generally speaking, the encapsulation here will be specific to the > format of the containing file. For example, IPFIX files may be > embedded in XML elements using hex or Base64 encoding, or in raw > binary files using start and end delimiters or some form of run- > length encoding. As there are as many potential encapsulations here= > as there are potential file formats, the specifics of each are out o= f > scope for this specification. I would remove the whole section. It is completely out of scope. > B.1.3. Template Format >=20 > Template FlowSets in NetFlow V9 support a subset of functionality of= > those in IPFIX. Specifically, NetFlow V9 does not have any support > for vendor-specific Information Elements as IPFIX does, so there is > no enterprise bit or facility for associating a private enterprise > number with an information element. NetFlow V9 does not support variable length fields. > Options Template FlowSets in NetFlow V9 are similar to o Options > Template Sets in IPFIX in the same way. > B.1.4. Information Model >=20 > The NetFlow V9 field type definitions are a compatible subset of, an= d > have evolved in concert with, the IPFIX Information Model. IPFIX > Information Element numbers in the range 1-127 are defined by the RFC 5102 talks about IE IDs, not numbers. > IPFIX Information Model [RFC5102] to be compatible with the > corresponding NetFlow V9 field types. > B.2. A Method for Transforming NetFlow V9 messages to IPFIX >=20 > This appendix describes a method for transforming NetFlow V9 Packets= > into IPFIX Messages, which can be used to store NetFlow V9 data in > IPFIX Files. A process transforming NetFlow V9 Packets into IPFIX > Messages must handle the fact that NetFlow V9 Packets and IPFIX > Messages are framed differently, that sequence numbering works > differently, and that the NetFlow V9 field type definitions are only= > compatible with the IPFIX Information Model field and/or information= > element numbers below Information Element number 128. what is an Information Model field? Consider: ...are only compatible with the IPFIX Information Element IDs below 128. > For each incoming NetFlow V9 packet, the transformation process must= : >=20 > 1. Verify that the Version field of the packet header is 9. >=20 > 2. Verify that the Sequence Number field of the packet header is > valid. >=20 > 3. Scan the packet to: >=20 > 1. verify that it contains no Templates with field numbers field types > outside the range 1-127; >=20 > 2. verify that it contains no FlowSets with Set IDs between 2 > and 255 inclusive; >=20 > 3. verify that it contains the number of records in FlowSets, > Template FlowSets, and Options Template FlowSets declared in= > the Count field of the packet header; and >=20 > 4. count the number of records in FlowSets for calculating the only records of Data FlowSets must be counted!!! > IPFIX Sequence number. Number > 4. Calculate a Sequence Number for each IPFIX Observation Domain by= > storing the last Sequence Number sent for each Observation Domai= n > plus the count of records in FlowSets in the previous step to be= Data FlowSets! > sent as the Sequence Number for the IPFIX Message within that > Observation Domain following this one. "following this one" is confusing. I would move step 4 behind step 6 and write that the Sequence Number is increased by the number of records in Data FlowSets contained in the written Message. > 5. Generate a new IPFIX Message Header with: >=20 > 1. a Version field of 10; >=20 > 2. a Length field with the number of octets in the IPFIX > Message, generally available by subtracting 4 from the lengt= h > of the NetFlow V9 packet as returned from the transport laye= r > (accounting for the difference in message header lengths); >=20 > 3. the Sequence Number calculated for this message by the > Sequence Number calculation step; and >=20 > 4. Export Time and Observation Domain ID taken from the UNIX > secs and Source ID fields of the NetFlow V9 packet header, > respectively. >=20 > 6. Copy each FlowSet from the Netflow V9 packet to the IPFIX Messag= e > after the header. Replace Set ID 0 with Set ID 2 for Template > Sets, and Set ID 1 with Set ID 3 for Options Template Sets. >=20 > Note that this process loses system uptime information; if such > information is required, the transformation process will have to > export that information using IPFIX Options. This may require a mor= e > sophisticated transformation process structure. >=20 --=20 Dipl.-Ing. Gerhard M=FCnz Computer Networks and Internet Wilhelm Schickard Institute for Computer Science University of Tuebingen Sand 13 (Room B309), D-72076 Tuebingen, Germany Phone: +49 7071 29-70534 Fax: +49 7071 2Options > Template Sets in IPFIX in the same way. > B.1.4. Information Model >=20 > The NetFlow V9 field type definitions are a compatible subset of, an= d > have evolved in concert with, the IPFIX Information Model. IPFIX > Information Element numbers in the range 1-127 are defined by the RFC 5102 talks about IE IDs, not numbers. > IPFIX Information Model [RFC5102] to be compatible with the > corresponding NetFlow V9 field types. > B.2. A Method for Transforming NetFlow V9 messages to IPFIX >=20 > This appendix describes a method for transforming NetFlow V9 Packets= > into IPFIX Messages, which can be used to store NetFlow V9 data in > IPFIX Files. A process transforming NetFlow V9 Packets into IPFIX > Messages must handle the fact that NetFlow V9 Packets and IPFIX > Messages are framed differently, that sequence numbering works > differently, and that the NetFlow V9 field type definitions are only= > compatible with the IPFIX Information Model field and/or information= > element numbers below Information Element number 128. what is an Information Model field? Consider: ...are only compatible with the IPFIX Information Element IDs below 128. > For each incoming NetFlow V9 packet, the transformation process must= : >=20 > 1. Verify that the Version field of the packet header is 9. >=20 > 2. Verify that the Sequence Number field of the packet header is > valid. >=20 > 3. Scan the packet to: >=20 > 1. verify that it contains no Templates with field numbers field types > outside the range 1-127; >=20 > 2. verify that it contains no FlowSets with Set IDs between 2 > and 255 inclusive; >=20 > 3. verify that it contains the number of records in FlowSets, > Template FlowSets, and Options Template FlowSets declared in= > the Count field of the packet header; and >=20 > 4. count the number of records in FlowSets for calculating the only records of Data FlowSets must be counted!!! > IPFIX Sequence number. Number > 4. Calculate a Sequence Number for each IPFIX Observation Domain by= > storing the last Sequence Number sent for each Observation Domai= n > plus the count of records in FlowSets in the previous step to be= Data FlowSets! > sent as the Sequence Number for the IPFIX Message within that > Observation Domain following this one. "following this one" is confusing. I would move step 4 behind step 6 and write that the Sequence Number is increased by the number of records in Data FlowSets contained in the written Message. > 5. Generate a new IPFIX Message Header with: >=20 > 1. a Version field of 10; >=20 > 2. a Length field with the number of octets in the IPFIX > Message, generally available by subtracting 4 from the lengt= h > of the NetFlow V9 packet as returned from the transport laye= r > (accounting for the difference in message header lengths); >=20 > 3. the Sequence Number calculated for this message by the > Sequence Number calculation step; and >=20 > 4. Export Time and Observation Domain ID taken from the UNIX > secs and Source ID fields of the NetFlow V9 packet header, > respectively. >=20 > 6. Copy each FlowSet from the Netflow V9 packet to the IPFIX Messag= e > after the header. Replace Set ID 0 with Set ID 2 for Template > Sets, and Set ID 1 with Set ID 3 for Options Template Sets. >=20 > Note that this process loses system uptime information; if such > information is required, the transformation process will have to > export that information using IPFIX Options. This may require a mor= e > sophisticated transformation process structure. >=20 --=20 Dipl.-Ing. Gerhard M=FCnz Computer Networks and Internet Wilhelm Schickard Institute for Computer Science University of Tuebingen Sand 13 (Room B309), D-72076 Tuebingen, Germany Phone: +49 7071 29-70534 Fax: +49 7071 29-9-5220 E-mail: muenz@informatik.uni-tuebingen.de WWW: http://net.informatik.uni-tuebingen.de/~muenz --------------ms060301010901010606010100 Content-Type: application/x-pkcs7-signature; name="smime.p7s" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="smime.p7s" Content-Description: S/MIME Cryptographic Signature MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAQAAoIIJdTCC AxUwggJ+oAMCAQICED9aGsYWkMr+s4zmyODhB+IwDQYJKoZIhvcNAQEFBQAwYjELMAkGA1UE BhMCWkExJTAjBgNVBAoTHFRoYXd0ZSBDb25zdWx0aW5nIChQdHkpIEx0ZC4xLDAqBgNVBAMT I1RoYXd0ZSBQZXJzb25hbCBGcmVlbWFpbCBJc3N1aW5nIENBMB4XDTA4MDIxMzE1MTUxN1oX DTA5MDIxMjE1MTUxN1owbDEOMAwGA1UEBBMFTXVlbnoxEDAOBgNVBCoTB0dlcmhhcmQxFjAU BgNVBAMTDUdlcmhhcmQgTXVlbnoxMDAuBgkqhkiG9w0BCQEWIW11ZW56QGluZm9ybWF0aWsu dW5pLXR1ZWJpbmdlbi5kZTCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAMZex/Sq sAxkzTVvKP/YAgkaeXA+ngH59Aa0bbRPsKOWzAndGqty5EKcEzrnKqEJ27qHFvoF/pHp88U2 7SJI/xbqkgWeV2jRaldipZQYlnjYLQcmb4cewIFuGRRSVrm3BquzX38aYazuE4+DVH2Z3a8z n0FcdMXhA1NR2Ma1rh4G7SIeZ+hC7czbvNRPraBliGdQhs8J/6yP/iL8aNYAl9c7CL4ofRj8 Y9orMOV/4vtWTq76/VQUVdbhUMiv0D8aHqI1ZvGskhRRvmITgQRVbbn8N8WTpZ0UCgMDjxPP 9i5IhLfp6oBtsKl4OZ0RXvSLZrbJTkBX3vnEutcyxDvyNgMCAwEAAaM+MDwwLAYDVR0RBCUw I4EhbXVlbnpAaW5mb3JtYXRpay51bmktdHVlYmluZ2VuLmRlMAwGA1UdEwEB/wQCMAAwDQYJ KoZIhvcNAQEFBQADgYEAX5SiD6epJePwBjJumOsTF6wzeuZRDLYlN+fOpXwd2C0Yx6i8iIZ9 l/J/nGaE1YpJPfX5oJDE+tOk1vYh2E9ThLOj9kJ3buZmgOCdVu90qtCWhfhli7RCYcJ+G9M3 FCnqbrzI/waPPXGB8/DY1HKgPj5G+oKPUK+GD2aE1Q3PYGowggMVMIICfqADAgECAhA/WhrG FpDK/rOM5sjg4QfiMA0GCSqGSIb3DQEBBQUAMGIxCzAJBgNVBAYTAlpBMSUwIwYDVQQKExxU aGF3dGUgQ29uc3VsdGluZyAoUHR5KSBMdGQuMSwwKgYDVQQDEyNUaGF3dGUgUGVyc29uYWwg RnJlZW1haWwgSXNzdWluZyBDQTAeFw0wODAyMTMxNTE1MTdaFw0wOTAyMTIxNTE1MTdaMGwx DjAMBgNVBAQTBU11ZW56MRAwDgYDVQQqEwdHZXJoYXJkMRYwFAYDVQQDEw1HZXJoYXJkIE11 ZW56MTAwLgYJKoZIhvcNAQkBFiFtdWVuekBpbmZvcm1hdGlrLnVuaS10dWViaW5nZW4uZGUw ggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQDGXsf0qrAMZM01byj/2AIJGnlwPp4B +fQGtG20T7CjlswJ3RqrcuRCnBM65yqhCdu6hxb6Bf6R6fPFNu0iSP8W6pIFnldo0WpXYqWU GJZ42C0HJm+HHsCBbhkUUla5twars19/GmGs7hOPg1R9md2vM59BXHTF4QNTUdjGta4eBu0i HmfoQu3M27zUT62gZYhnUIbPCf+sj/4i/GjWAJfXOwi+KH0Y/GPaKzDlf+L7Vk6u+v1UFFXW 4VDIr9A/Gh6iNWbxrJIUUb5iE4EEVW25/DfFk6WdFAoDA48Tz/YuSIS36eqAbbCpeDmdEV70 i2a2yU5AV975xLrXMsQ78jYDAgMBAAGjPjA8MCwGA1UdEQQlMCOBIW11ZW56QGluZm9ybWF0 aWsudW5pLXR1ZWJpbmdlbi5kZTAMBgNVHRMBAf8EAjAAMA0GCSqGSIb3DQEBBQUAA4GBAF+U og+nqSXj8AYybpjrExesM3rmUQy2JTfnzqV8HdgtGMeovIiGfZfyf5xmhNWKST31+aCQxPrT pNb2IdhPU4Szo/ZCd27mZoDgnVbvdKrQloX4ZYu0QmHCfhvTNxQp6m68yP8Gjz1xgfPw2NRy oD4+RvqCj1Cvhg9mhNUNz2BqMIIDPzCCAqigAwIBAgIBDTANBgkqhkiG9w0BAQUFADCB0TEL MAkGA1UEBhMCWkExFTATBgNVBAgTDFdlc3Rlcm4gQ2FwZTESMBAGA1UEBxMJQ2FwZSBUb3du MRowGAYDVQQKExFUaGF3dGUgQ29uc3VsdGluZzEoMCYGA1UECxMfQ2VydGlmaWNhdGlvbiBT ZXJ2aWNlcyBEaXZpc2lvbjEkMCIGA1UEAxMbVGhhd3RlIFBlcnNvbmFsIEZyZWVtYWlsIENB MSswKQYJKoZIhvcNAQkBFhxwZXJzb25hbC1mcmVlbWFpbEB0aGF3dGUuY29tMB4XDTAzMDcx NzAwMDAwMFoXDTEzMDcxNjIzNTk1OVowYjELMAkGA1UEBhMCWkExJTAjBgNVBAoTHFRoYXd0 ZSBDb25zdWx0aW5nIChQdHkpIEx0ZC4xLDAqBgNVBAMTI1RoYXd0ZSBQZXJzb25hbCBGcmVl bWFpbCBJc3N1aW5nIENBMIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQDEpjxVc1X7TrnK mVoeaMB1BHCd3+n/ox7svc31W/Iadr1/DDph8r9RzgHU5VAKMNcCY1osiRVwjt3J8CuFWqo/ cVbLrzwLB+fxH5E2JCoTzyvV84J3PQO+K/67GD4Hv0CAAmTXp6a7n2XRxSpUhQ9IBH+nttE8 YQRAHmQZcmC3+wIDAQABo4GUMIGRMBIGA1UdEwEB/wQIMAYBAf8CAQAwQwYDVR0fBDwwOjA4 oDagNIYyaHR0cDovL2NybC50aGF3dGUuY29tL1RoYXd0ZVBlcnNvbmFsRnJlZW1haWxDQS5j cmwwCwYDVR0PBAQDAgEGMCkGA1UdEQQiMCCkHjAcMRowGAYDVQQDExFQcml2YXRlTGFiZWwy LTEzODANBgkqhkiG9w0BAQUFAAOBgQBIjNFQg+oLLswNo2asZw9/r6y+whehQ5aUnX9MIbj4 Nh+qLZ82L8D0HFAgk3A8/a3hYWLD2ToZfoSxmRsAxRoLgnSeJVCUYsfbJ3FXJY3dqZw5jowg T2Vfldr394fWxghOrvbqNOUQGls1TXfjViF4gtwhGTXeJLHTHUb/XV9lTzGCA2QwggNgAgEB MHYwYjELMAkGA1UEBhMCWkExJTAjBgNVBAoTHFRoYXd0ZSBDb25zdWx0aW5nIChQdHkpIEx0 ZC4xLDAqBgNVBAMTI1RoYXd0ZSBQZXJzb25hbCBGcmVlbWFpbCBJc3N1aW5nIENBAhA/WhrG FpDK/rOM5sjg4QfiMAkGBSsOAwIaBQCgggHDMBgGCSqGSIb3DQEJAzELBgkqhkiG9w0BBwEw HAYJKoZIhvcNAQkFMQ8XDTA4MDgxNTE2NTkyNVowIwYJKoZIhvcNAQkEMRYEFGkjcIK/kkYb dIKVaJhoueeKbdoJMFIGCSqGSIb3DQEJDzFFMEMwCgYIKoZIhvcNAwcwDgYIKoZIhvcNAwIC AgCAMA0GCCqGSIb3DQMCAgFA5220 E-mail: muenz@informatik.uni-tuebingen.de WWW: http://net.informatik.uni-tuebingen.de/~muenz --------------ms060301010901010606010100 Content-Type: application/x-pkcs7-signature; name="smime.p7s" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="smime.p7s" Content-Description: S/MIME Cryptographic Signature MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAQAAoIIJdTCC AxUwggJ+oAMCAQICED9aGsYWkMr+s4zmyODhB+IwDQYJKoZIhvcNAQEFBQAwYjELMAkGA1UE BhMCWkExJTAjBgNVBAoTHFRoYXd0ZSBDb25zdWx0aW5nIChQdHkpIEx0ZC4xLDAqBgNVBAMT I1RoYXd0ZSBQZXJzb25hbCBGcmVlbWFpbCBJc3N1aW5nIENBMB4XDTA4MDIxMzE1MTUxN1oX DTA5MDIxMjE1MTUxN1owbDEOMAwGA1UEBBMFTXVlbnoxEDAOBgNVBCoTB0dlcmhhcmQxFjAU BgNVBAMTDUdlcmhhcmQgTXVlbnoxMDAuBgkqhkiG9w0BCQEWIW11ZW56QGluZm9ybWF0aWsu dW5pLXR1ZWJpbmdlbi5kZTCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAMZex/Sq sAxkzTVvKP/YAgkaeXA+ngH59Aa0bbRPsKOWzAndGqty5EKcEzrnKqEJ27qHFvoF/pHp88U2 7SJI/xbqkgWeV2jRaldipZQYlnjYLQcmb4cewIFuGRRSVrm3BquzX38aYazuE4+DVH2Z3a8z n0FcdMXhA1NR2Ma1rh4G7SIeZ+hC7czbvNRPraBliGdQhs8J/6yP/iL8aNYAl9c7CL4ofRj8 Y9orMOV/4vtWTq76/VQUVdbhUMiv0D8aHqI1ZvGskhRRvmITgQRVbbn8N8WTpZ0UCgMDjxPP 9i5IhLfp6oBtsKl4OZ0RXvSLZrbJTkBX3vnEutcyxDvyNgMCAwEAAaM+MDwwLAYDVR0RBCUw I4EhbXVlbnpAaW5mb3JtYXRpay51bmktdHVlYmluZ2VuLmRlMAwGA1UdEwEB/wQCMAAwDQYJ KoZIhvcNAQEFBQADgYEAX5SiD6epJePwBjJumOsTF6wzeuZRDLYlN+fOpXwd2C0Yx6i8iIZ9 l/J/nGaE1YpJPfX5oJDE+tOk1vYh2E9ThLOj9kJ3buZmgOCdVu90qtCWhfhli7RCYcJ+G9M3 FCnqbrzI/waPPXGB8/DY1HKgPj5G+oKPUK+GD2aE1Q3PYGowggMVMIICfqADAgECAhA/WhrG FpDK/rOM5sjg4QfiMA0GCSqGSIb3DQEBBQUAMGIxCzAJBgNVBAYTAlpBMSUwIwYDVQQKExxU aGF3dGUgQ29uc3VsdGluZyAoUHR5KSBMdGQuMSwwKgYDVQQDEyNUaGF3dGUgUGVyc29uYWwg RnJlZW1haWwgSXNzdWluZyBDQTAeFw0wODAyMTMxNTE1MTdaFw0wOTAyMTIxNTE1MTdaMGwx DjAMBgNVBAQTBU11ZW56MRAwDgYDVQQqEwdHZXJoYXJkMRYwFAYDVQQDEw1HZXJoYXJkIE11 ZW56MTAwLgYJKoZIhvcNAQkBFiFtdWVuekBpbmZvcm1hdGlrLnVuaS10dWViaW5nZW4uZGUw ggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQDGXsf0qrAMZM01byj/2AIJGnlwPp4B +fQGtG20T7CjlswJ3RqrcuRCnBM65yqhCdu6hxb6Bf6R6fPFNu0iSP8W6pIFnldo0WpXYqWU GJZ42C0HJm+HHsCBbhkUUla5twars19/GmGs7hOPg1R9md2vM59BXHTF4QNTUdjGta4eBu0i HmfoQu3M27zUT62gZYhnUIbPCf+sj/4i/GjWAJfXOwi+KH0Y/GPaKzDlf+L7Vk6u+v1UFFXW 4VDIr9A/Gh6iNWbxrJIUUb5iE4EEVW25/DfFk6WdFAoDA48Tz/YuSIS36eqAbbCpeDmdEV70 i2a2yU5AV975xLrXMsQ78jYDAgMBAAGjPjA8MCwGA1UdEQQlMCOBIW11ZW56QGluZm9ybWF0 aWsudW5pLXR1ZWJpbmdlbi5kZTAMBgNVHRMBAf8EAjAAMA0GCSqGSIb3DQEBBQUAA4GBAF+U og+nqSXj8AYybpjrExesM3rmUQy2JTfnzqV8HdgtGMeovIiGfZfyf5xmhNWKST31+aCQxPrT pNb2IdhPU4Szo/ZCd27mZoDgnVbvdKrQloX4ZYu0QmHCfhvTNxQp6m68yP8Gjz1xgfPw2NRy oD4+RvqCj1Cvhg9mhNUNz2BqMIIDPzCCAqigAwIBAgIBDTANBgkqhkiG9w0BAQUFADCB0TEL MAkGA1UEBhMCWkExFTATBgNVBAgTDFdlc3Rlcm4gQ2FwZTESMBAGA1UEBxMJQ2FwZSBUb3du MRowGAYDVQQKExFUaGF3dGUgQ29uc3VsdGluZzEoMCYGA1UECxMfQ2VydGlmaWNhdGlvbiBT ZXJ2aWNlcyBEaXZpc2lvbjEkMCIGA1UEAxMbVGhhd3RlIFBlcnNvbmFsIEZyZWVtYWlsIENB MSswKQYJKoZIhvcNAQkBFhxwZXJzb25hbC1mcmVlbWFpbEB0aGF3dGUuY29tMB4XDTAzMDcx NzAwMDAwMFoXDTEzMDcxNjIzNTk1OVowYjELMAkGA1UEBhMCWkExJTAjBgNVBAoTHFRoYXd0 ZSBDb25zdWx0aW5nIChQdHkpIEx0ZC4xLDAqBgNVBAMTI1RoYXd0ZSBQZXJzb25hbCBGcmVl bWFpbCBJc3N1aW5nIENBMIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQDEpjxVc1X7TrnK mVoeaMB1BHCd3+n/ox7svc31W/Iadr1/DDph8r9RzgHU5VAKMNcCY1osiRVwjt3J8CuFWqo/ cVbLrzwLB+fxH5E2JCoTzyvV84J3PQO+K/67GD4Hv0CAAmTXp6a7n2XRxSpUhQ9IBH+nttE8 YQRAHmQZcmC3+wIDAQABo4GUMIGRMBIGA1UdEwEB/wQIMAYBAf8CAQAwQwYDVR0fBDwwOjA4 oDagNIYyaHR0cDovL2NybC50aGF3dGUuY29tL1RoYXd0ZVBlcnNvbmFsRnJlZW1haWxDQS5j cmwwCwYDVR0PBAQDAgEGMCkGA1UdEQQiMCCkHjAcMRowGAYDVQQDExFQcml2YXRlTGFiZWwy LTEzODANBgkqhkiG9w0BAQUFAAOBgQBIjNFQg+oLLswNo2asZw9/r6y+whehQ5aUnX9MIbj4 Nh+qLZ82L8D0HFAgk3A8/a3hYWLD2ToZfoSxmRsAxRoLgnSeJVCUYsfbJ3FXJY3dqZw5jowg T2Vfldr394fWxghOrvbqNOUQGls1TXfjViF4gtwhGTXeJLHTHUb/XV9lTzGCA2QwggNgAgEB MHYwYjELMAkGA1UEBhMCWkExJTAjBgNVBAoTHFRoYXd0ZSBDb25zdWx0aW5nIChQdHkpIEx0 ZC4xLDAqBgNVBAMTI1RoYXd0ZSBQZXJzb25hbCBGcmVlbWFpbCBJc3N1aW5nIENBAhA/WhrG FpDK/rOM5sjg4QfiMAkGBSsOAwIaBQCgggHDMBgGCSqGSIb3DQEJAzELBgkqhkiG9w0BBwEw HAYJKoZIhvcNAQkFMQ8XDTA4MDgxNTE2NTkyNVowIwYJKoZIhvcNAQkEMRYEFGkjcIK/kkYb dIKVaJhoueeKbdoJMFIGCSqGSIb3DQEJDzFFMEMwCgYIKoZIhvcNAwcwDgYIKoZIhvcNAwIC AgCAMA0GCCqGSIb3DQMCAgFAMAMAcGBSsOAwIHMA0GCCqGSIb3DQMCAgEoMIGFBgkrBgEEAYI3 EAQxeDB2MGIxCzAJBgNVBAYTAlpBMSUwIwYDVQQKExxUaGF3dGUgQ29uc3VsdGluZyAoUHR5 KSBMdGQuMSwwKgYDVQQDEyNUaGF3dGUgUGVyc29uYWwgRnJlZW1haWwgSXNzdWluZyBDQQIQ P1oaxhaQyv6zjObI4OEH4jCBhwYLKoZIhvcNAQkQAgsxeKB2MGIxCzAJBgNVBAYTAlpBMSUw IwYDVQQKExxUaGF3dGUgQ29uc3VsdGluZyAoUHR5KSBMdGQuMSwwKgYDVQQDEyNUaGF3dGUg UGVyc29uYWwgRnJlZW1haWwgSXNzdWluZyBDQQIQP1oaxhaQyv6zjObI4OEH4jANBgkqhkiG 9w0BAQEFAASCAQBNOszIkVj1zI/3XtSQO5rkCHQl485Y+Gzv9W42FWG+gept3ixoMvhAVHsG Ed03htgRLdVAYjkdeVbT7yhttBzcLipHZZGM85p6hKtffZw5HV9NII3Ka5F0Sfqy9ZcmWy2i AZtPO4mRGaeddlj3hs2DFcxPKK9NCnHJtxMNAwcwp43i/PFLBcEBCubLbYtvfwFLechy9kZy /JjIPTqbqoRgssRIV1tS61TTe6LUJrFg+MIw0HEAOdnkrIgkwwwfmY1iPzJON9vlvRMVk+uz Ke/ITmc4Bc6zjjuHJIz2VAUgG3qgBJ9AK5HkRl7trMn4v4695LDpsFE+0p9ajmYxZj+mAAAA AAAA --------------ms060301010901010606010100-- --===============0692567637== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ IPFIX mailing list IPFIX@ietf.org https://www.ietf.org/mailman/listinfo/ipfix --===============0692567637==-- cGBSsOAwIHMA0GCCqGSIb3DQMCAgEoMIGFBgkrBgEEAYI3 EAQxeDB2MGIxCzAJBgNVBAYTAlpBMSUwIwYDVQQKExxUaGF3dGUgQ29uc3VsdGluZyAoUHR5 KSBMdGQuMSwwKgYDVQQDEyNUaGF3dGUgUGVyc29uYWwgRnJlZW1haWwgSXNzdWluZyBDQQIQ P1oaxhaQyv6zjObI4OEH4jCBhwYLKoZIhvcNAQkQAgsxeKB2MGIxCzAJBgNVBAYTAlpBMSUw IwYDVQQKExxUaGF3dGUgQ29uc3VsdGluZyAoUHR5KSBMdGQuMSwwKgYDVQQDEyNUaGF3dGUg UGVyc29uYWwgRnJlZW1haWwgSXNzdWluZyBDQQIQP1oaxhaQyv6zjObI4OEH4jANBgkqhkiG 9w0BAQEFAASCAQBNOszIkVj1zI/3XtSQO5rkCHQl485Y+Gzv9W42FWG+gept3ixoMvhAVHsG Ed03htgRLdVAYjkdeVbT7yhttBzcLipHZZGM85p6hKtffZw5HV9NII3Ka5F0Sfqy9ZcmWy2i AZtPO4mRGaeddlj3hs2DFcxPKK9NCnHJtxMNAwcwp43i/PFLBcEBCubLbYtvfwFLechy9kZy /JjIPTqbqoRgssRIV1tS61TTe6LUJrFg+MIw0HEAOdnkrIgkwwwfmY1iPzJON9vlvRMVk+uz Ke/ITmc4Bc6zjjuHJIz2VAUgG3qgBJ9AK5HkRl7trMn4v4695LDpsFE+0p9ajmYxZj+mAAAA AAAA --------------ms060301010901010606010100-- --===============0692567637== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ IPFIX mailing list IPFIX@ietf.org https://www.ietf.org/mailman/listinfo/ipfix --===============0692567637==-- From ipfix-bounces@ietf.org Wed Aug 20 00:12:15 2008 Return-Path: X-Original-To: ipfix-archive@lists.ietf.org Delivered-To: ietfarch-ipfix-archive@core3.amsl.com Received: from [127.0.0.1] (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 0E2E63A695B; Wed, 20 Aug 2008 00:12:15 -0700 (PDT) X-Original-To: ipfix@core3.amsl.com Delivered-To: ipfix@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 365283A69A3 for ; Wed, 20 Aug 2008 00:12:14 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -0.147 X-Spam-Level: X-Spam-Status: No, score=-0.147 tagged_above=-999 required=5 tests=[AWL=-0.312, BAYES_40=-0.185, HELO_EQ_DE=0.35] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id vvk4EQQ5Gmk1 for ; Wed, 20 Aug 2008 00:12:13 -0700 (PDT) Received: from hermes.jacobs-university.de (hermes.jacobs-university.de [212.201.44.23]) by core3.amsl.com (Postfix) with ESMTP id B18793A6841 for ; Wed, 20 Aug 2008 00:12:12 -0700 (PDT) Received: from localhost (demetrius2.jacobs-university.de [212.201.44.47]) by hermes.jacobs-university.de (Postfix) with ESMTP id B32E9C0072; Wed, 20 Aug 2008 09:12:20 +0200 (CEST) X-Virus-Scanned: amavisd-new at jacobs-university.de Received: from hermes.jacobs-university.de ([212.201.44.23]) by localhost (demetrius2.jacobs-university.de [212.201.44.32]) (amavisd-new, port 10024) with ESMTP id UvGqSRwrGS1J; Wed, 20 Aug 2008 09:12:13 +0200 (CEST) Received: from elstar.local (elstar.iuhb02.iu-bremen.de [10.50.231.133]) by hermes.jacobs-university.de (Postfix) with ESMTP id 8AEA1C006A; Wed, 20 Aug 2008 09:12:13 +0200 (CEST) Received: by elstar.local (Postfix, from userid 501) id BF101697409; Wed, 20 Aug 2008 09:12:12 +0200 (CEST) Date: Wed, 20 Aug 2008 09:12:12 +0200 From: Juergen Schoenwaelder To: ipfix@ietf.org Message-ID: <20080820071212.GC17541@elstar.local> Mail-Followup-To: ipfix@ietf.org, nmrg@ibr.cs.tu-bs.de MIME-Version: 1.0 Content-Disposition: inline User-Agent: Mutt/1.5.18 (2008-05-17) Cc: nmrg@ibr.cs.tu-bs.de Subject: [IPFIX] Netflow/IPFIX workshop: information and request for feedback X-BeenThere: ipfix@ietf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: j.schoenwaelder@jacobs-university.de List-Id: IPFIX WG discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: ipfix-bounces@ietf.org Errors-To: ipfix-bounces@ietf.org Hi, the following Netflow/IPFIX workshop/meeting proposal was sent to the Network Management Research Group (NMRG) mailing list on July 31st. I like to know whether there is support for such an event form the IPFIX community. Please drop a note if you plan to contribute so that I can gauge interest. If you have questions concerning the scope and organization, please contact Ramin Sadre. If the question is of general interest, please keep the NMRG list on CC for now. /js ----- Forwarded message from Ramin Sadre ----- Date: Thu, 31 Jul 2008 17:37:12 +0200 From: Ramin Sadre User-Agent: Thunderbird 2.0.0.16 (Windows/20080708) To: nmrg@ibr.cs.tu-bs.de Subject: [nmrg] Netflow/IPFIX workshop: information and request for feedback Dear list members, (we apologize for multiple postings) as already announced some days ago, the EMANICS network of excellence is currently preparing the EMANICS Workshop on Netflow/IPFIX usage in network management http://www.simpleweb.org/netflow/ The worFrom ipfix-bounces@ietf.org Wed Aug 20 00:12:15 2008 Return-Path: X-Original-To: ipfix-archive@optimus.ietf.org Delivered-To: ietfarch-ipfix-archive@core3.amsl.com Received: from [127.0.0.1] (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 0E2E63A695B; Wed, 20 Aug 2008 00:12:15 -0700 (PDT) X-Original-To: ipfix@core3.amsl.com Delivered-To: ipfix@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 365283A69A3 for ; Wed, 20 Aug 2008 00:12:14 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -0.147 X-Spam-Level: X-Spam-Status: No, score=-0.147 tagged_above=-999 required=5 tests=[AWL=-0.312, BAYES_40=-0.185, HELO_EQ_DE=0.35] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id vvk4EQQ5Gmk1 for ; Wed, 20 Aug 2008 00:12:13 -0700 (PDT) Received: from hermes.jacobs-university.de (hermes.jacobs-university.de [212.201.44.23]) by core3.amsl.com (Postfix) with ESMTP id B18793A6841 for ; Wed, 20 Aug 2008 00:12:12 -0700 (PDT) Received: from localhost (demetrius2.jacobs-university.de [212.201.44.47]) by hermes.jacobs-university.de (Postfix) with ESMTP id B32E9C0072; Wed, 20 Aug 2008 09:12:20 +0200 (CEST) X-Virus-Scanned: amavisd-new at jacobs-university.de Received: from hermes.jacobs-university.de ([212.201.44.23]) by localhost (demetrius2.jacobs-university.de [212.201.44.32]) (amavisd-new, port 10024) with ESMTP id UvGqSRwrGS1J; Wed, 20 Aug 2008 09:12:13 +0200 (CEST) Received: from elstar.local (elstar.iuhb02.iu-bremen.de [10.50.231.133]) by hermes.jacobs-university.de (Postfix) with ESMTP id 8AEA1C006A; Wed, 20 Aug 2008 09:12:13 +0200 (CEST) Received: by elstar.local (Postfix, from userid 501) id BF101697409; Wed, 20 Aug 2008 09:12:12 +0200 (CEST) Date: Wed, 20 Aug 2008 09:12:12 +0200 From: Juergen Schoenwaelder To: ipfix@ietf.org Message-ID: <20080820071212.GC17541@elstar.local> Mail-Followup-To: ipfix@ietf.org, nmrg@ibr.cs.tu-bs.de MIME-Version: 1.0 Content-Disposition: inline User-Agent: Mutt/1.5.18 (2008-05-17) Cc: nmrg@ibr.cs.tu-bs.de Subject: [IPFIX] Netflow/IPFIX workshop: information and request for feedback X-BeenThere: ipfix@ietf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: j.schoenwaelder@jacobs-university.de List-Id: IPFIX WG discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: ipfix-bounces@ietf.org Errors-To: ipfix-bounces@ietf.org Hi, the following Netflow/IPFIX workshop/meeting proposal was sent to the Network Management Research Group (NMRG) mailing list on July 31st. I like to know whether there is support for such an event form the IPFIX community. Please drop a note if you plan to contribute so that I can gauge interest. If you have questions concerning the scope and organization, please contact Ramin Sadre. If the question is of general interest, please keep the NMRG list on CC for now. /js ----- Forwarded message from Ramin Sadre ----- Date: Thu, 31 Jul 2008 17:37:12 +0200 From: Ramin Sadre User-Agent: Thunderbird 2.0.0.16 (Windows/20080708) To: nmrg@ibr.cs.tu-bs.de Subject: [nmrg] Netflow/IPFIX workshop: information and request for feedback Dear list members, (we apologize for multiple postings) as already announced some days ago, the EMANICS network of excellence is currently preparing the EMANICS Workshop on Netflow/IPFIX usage in network management http://www.simpleweb.org/netflow/ The wkshop will take place October 30, at the LRZ Munich, Germany. We would highly appreciate your feedback and, of course, your participation. In addition, we kindly ask you to express your opinion on making the workshop a joint Emanics&IRTF-NMRG event. We want the workshop to be an opportunity for people to exchange and discuss their experiences and ideas. To structure the discussions the workshop will be organized into several discussion topics. For each discussion topic, there will be short presentations (5-10 minutes) introducing the topic, followed by a discussion round. Currently, the following topics are planned: 1. "What technologies are available to analyze flow data?" Presentations: * Flow record query languages. * Using SQL databases for flow processing. 2. "What can be expected from distributed analysis techniques?" Presentations: * A distributed architecture for IP flow analysis. 3. "Should we have a standard format for the annotation of Netflow/IPFIX traces?" Presentations: * Labeling flow traces for intrusion detection evaluation. 4. "For what kind of applications can Netflow/IPFIX be used?" Presentations: * Network anomaly characterization using flows. * Application level dependency discovery with flows. * Management of light paths in optical networks using flows. The above list is still preliminary and we welcome your contributions and suggestions. Regards, Ramin Sadre -- !! This message is brought to you via the `nmrg' mailing list. !! Please do not reply to this message to unsubscribe. To unsubscribe or adjust !! your settings, send a mail message to !! or look at https://mail.ibr.cs.tu-bs.de/mailman/listinfo/nmrg. ----- End forwarded message ----- _______________________________________________ IPFIX mailing list IPFIX@ietf.org https://www.ietf.org/mailman/listinfo/ipfix orkshop will take place October 30, at the LRZ Munich, Germany. We would highly appreciate your feedback and, of course, your participation. In addition, we kindly ask you to express your opinion on making the workshop a joint Emanics&IRTF-NMRG event. We want the workshop to be an opportunity for people to exchange and discuss their experiences and ideas. To structure the discussions the workshop will be organized into several discussion topics. For each discussion topic, there will be short presentations (5-10 minutes) introducing the topic, followed by a discussion round. Currently, the following topics are planned: 1. "What technologies are available to analyze flow data?" Presentations: * Flow record query languages. * Using SQL databases for flow processing. 2. "What can be expected from distributed analysis techniques?" Presentations: * A distributed architecture for IP flow analysis. 3. "Should we have a standard format for the annotation of Netflow/IPFIX traces?" Presentations: * Labeling flow traces for intrusion detection evaluation. 4. "For what kind of applications can Netflow/IPFIX be used?" Presentations: * Network anomaly characterization using flows. * Application level dependency discovery with flows. * Management of light paths in optical networks using flows. The above list is still preliminary and we welcome your contributions and suggestions. Regards, Ramin Sadre -- !! This message is brought to you via the `nmrg' mailing list. !! Please do not reply to this message to unsubscribe. To unsubscribe or adjust !! your settings, send a mail message to !! or look at https://mail.ibr.cs.tu-bs.de/mailman/listinfo/nmrg. ----- End forwarded message ----- _______________________________________________ IPFIX mailing list IPFIX@ietf.org https://www.ietf.org/mailman/listinfo/ipfix From ipfix-bounces@ietf.org Wed Aug 20 02:00:06 2008 Return-Path: X-Original-To: ipfix-archive@optimus.ietf.org Delivered-To: ietfarch-ipfix-archive@core3.amsl.com Received: from [127.0.0.1] (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 136633A6B3D; Wed, 20 Aug 2008 02:00:06 -0700 (PDT) X-Original-To: ipfix@core3.amsl.com Delivered-To: ipfix@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id E657F3A6B2D for ; Wed, 20 Aug 2008 02:00:04 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.227 X-Spam-Level: X-Spam-Status: No, score=-1.227 tagged_above=-999 required=5 tests=[AWL=1.022, BAYES_00=-2.599, HELO_EQ_DE=0.35] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id aC8tHOffOePV for ; Wed, 20 Aug 2008 02:00:03 -0700 (PDT) Received: from smtp.cs.uni-tuebingen.de (u-173-c156.cs.uni-tuebingen.de [134.2.173.156]) by core3.amsl.com (Postfix) with ESMTP id 5EC5F3A6AB3 for ; Wed, 20 Aug 2008 02:00:03 -0700 (PDT) Received: from u-172-c138.cs.uni-tuebingen.de ([134.2.172.138]) by smtp.cs.uni-tuebingen.de with esmtpsa (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.69) (envelope-from ) id 1KVjY2-0003R4-Gy; Wed, 20 Aug 2008 11:00:10 +0200 Message-ID: <48ABDD4E.3050307@informatik.uni-tuebingen.de> Date: Wed, 20 Aug 2008 11:01:02 +0200 From: Gerhard Muenz User-Agent: Thunderbird 2.0.0.16 (Windows/20080708) MIME-Version: 1.0 To: ipfix@ietf.org, nmrg@ibr.cs.tu-bs.de References: <20080820071212.GC17541@elstar.local> In-Reply-To: <20080820071212.GC17541@elstar.local> Subject: Re: [IPFIX] Netflow/IPFIX workshop: information and request for feedback X-BeenThere: ipfix@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: IPFIX WG discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============0694312984==" Sender: ipfix-bounces@ietf.org Errors-To: ipfix-bounces@ietf.org This is a cryptographically signed message in MIME format. --===============0694312984== Content-Type: multipart/signed; protocol="application/x-pkcs7-signature"; micalg=sha1; boundary="------------ms050705060205010503030406" This is a cryptographically signed message in MIME format. --------------ms050705060205010503030406 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Hi J=FCrgen, Thank you for the information. My group is interested in the event and will participate (and possibly contribute) in the workshop. Gerhard Juergen Schoenwaelder wrote: > Hi, >=20 > the following Netflow/IPFIX workshop/meeting proposal was sent to the > Network Management Research Group (NMRG) mailing list on July 31st. I > like to know whether there is support for such an event form the IPFIX > community. Please drop a note if you plan to contribute so that I can > gauge interest. >=20 > If you have questions concerning the scope and organization, please > contact Ramin Sadre. If the question is of general interest, please > keep the NMRG list on CC for now. >=20 > /js >=20 > ----- Forwarded message from Ramin Sadre ----- >=20 > Date: Thu, 31 Jul 2008 17:37:12 +0200 > From: Ramin Sadre > User-Agent: Thunderbird 2.0.0.16 (Windows/20080708) > To: nmrg@ibr.cs.tu-bs.de > Subject: [nmrg] Netflow/IPFIX workshop: information and request for fee= dback >=20 > Dear list members, >=20 > (we apologize for multiple postings) >=20 > as already announced some dayFrom ipfix-bounces@ietf.org Wed Aug 20 02:00:06 2008 Return-Path: X-Original-To: ipfix-archive@lists.ietf.org Delivered-To: ietfarch-ipfix-archive@core3.amsl.com Received: from [127.0.0.1] (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 136633A6B3D; Wed, 20 Aug 2008 02:00:06 -0700 (PDT) X-Original-To: ipfix@core3.amsl.com Delivered-To: ipfix@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id E657F3A6B2D for ; Wed, 20 Aug 2008 02:00:04 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.227 X-Spam-Level: X-Spam-Status: No, score=-1.227 tagged_above=-999 required=5 tests=[AWL=1.022, BAYES_00=-2.599, HELO_EQ_DE=0.35] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id aC8tHOffOePV for ; Wed, 20 Aug 2008 02:00:03 -0700 (PDT) Received: from smtp.cs.uni-tuebingen.de (u-173-c156.cs.uni-tuebingen.de [134.2.173.156]) by core3.amsl.com (Postfix) with ESMTP id 5EC5F3A6AB3 for ; Wed, 20 Aug 2008 02:00:03 -0700 (PDT) Received: from u-172-c138.cs.uni-tuebingen.de ([134.2.172.138]) by smtp.cs.uni-tuebingen.de with esmtpsa (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.69) (envelope-from ) id 1KVjY2-0003R4-Gy; Wed, 20 Aug 2008 11:00:10 +0200 Message-ID: <48ABDD4E.3050307@informatik.uni-tuebingen.de> Date: Wed, 20 Aug 2008 11:01:02 +0200 From: Gerhard Muenz User-Agent: Thunderbird 2.0.0.16 (Windows/20080708) MIME-Version: 1.0 To: ipfix@ietf.org, nmrg@ibr.cs.tu-bs.de References: <20080820071212.GC17541@elstar.local> In-Reply-To: <20080820071212.GC17541@elstar.local> Subject: Re: [IPFIX] Netflow/IPFIX workshop: information and request for feedback X-BeenThere: ipfix@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: IPFIX WG discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============0694312984==" Sender: ipfix-bounces@ietf.org Errors-To: ipfix-bounces@ietf.org This is a cryptographically signed message in MIME format. --===============0694312984== Content-Type: multipart/signed; protocol="application/x-pkcs7-signature"; micalg=sha1; boundary="------------ms050705060205010503030406" This is a cryptographically signed message in MIME format. --------------ms050705060205010503030406 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Hi J=FCrgen, Thank you for the information. My group is interested in the event and will participate (and possibly contribute) in the workshop. Gerhard Juergen Schoenwaelder wrote: > Hi, >=20 > the following Netflow/IPFIX workshop/meeting proposal was sent to the > Network Management Research Group (NMRG) mailing list on July 31st. I > like to know whether there is support for such an event form the IPFIX > community. Please drop a note if you plan to contribute so that I can > gauge interest. >=20 > If you have questions concerning the scope and organization, please > contact Ramin Sadre. If the question is of general interest, please > keep the NMRG list on CC for now. >=20 > /js >=20 > ----- Forwarded message from Ramin Sadre ----- >=20 > Date: Thu, 31 Jul 2008 17:37:12 +0200 > From: Ramin Sadre > User-Agent: Thunderbird 2.0.0.16 (Windows/20080708) > To: nmrg@ibr.cs.tu-bs.de > Subject: [nmrg] Netflow/IPFIX workshop: information and request for fee= dback >=20 > Dear list members, >=20 > (we apologize for multiple postings) >=20 > as already announced some days s ago, the EMANICS network of excellence > is currently preparing the > EMANICS Workshop on Netflow/IPFIX usage in network management > http://www.simpleweb.org/netflow/ > The workshop will take place October 30, at the LRZ Munich, Germany. >=20 > We would highly appreciate your feedback and, of course, your > participation. In addition, we kindly ask you to express your opinion > on making the workshop a joint Emanics&IRTF-NMRG event. >=20 > We want the workshop to be an opportunity for people to exchange and > discuss their experiences and ideas. To structure the discussions > the workshop will be organized into several discussion topics. > For each discussion topic, there will be short presentations (5-10 minu= tes) > introducing the topic, followed by a discussion round. >=20 > Currently, the following topics are planned: > 1. "What technologies are available to analyze flow data?" > Presentations: > * Flow record query languages. > * Using SQL databases for flow processing. > 2. "What can be expected from distributed analysis techniques?" > Presentations: > * A distributed architecture for IP flow analysis. > 3. "Should we have a standard format for the annotation of Netflow/I= PFIX traces?" > Presentations: > * Labeling flow traces for intrusion detection evaluation. > 4. "For what kind of applications can Netflow/IPFIX be used?" > Presentations: > * Network anomaly characterization using flows. > * Application level dependency discovery with flows. > * Management of light paths in optical networks using flows. >=20 > The above list is still preliminary and we welcome your contributions > and suggestions. >=20 > Regards, > Ramin Sadre >=20 --=20 Dipl.-Ing. Gerhard M=FCnz Computer Networks and Internet Wilhelm Schickard Institute for Computer Science University of Tuebingen Sand 13 (Room B309), D-72076 Tuebingen, Germany Phone: +49 7071 29-70534 Fax: +49 7071 29-5220 E-mail: muenz@informatik.uni-tuebingen.de WWW: http://net.informatik.uni-tuebingen.de/~muenz --------------ms050705060205010503030406 Content-Type: application/x-pkcs7-signature; name="smime.p7s" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="smime.p7s" Content-Description: S/MIME Cryptographic Signature MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAQAAoIIJdTCC AxUwggJ+oAMCAQICED9aGsYWkMr+s4zmyODhB+IwDQYJKoZIhvcNAQEFBQAwYjELMAkGA1UE BhMCWkExJTAjBgNVBAoTHFRoYXd0ZSBDb25zdWx0aW5nIChQdHkpIEx0ZC4xLDAqBgNVBAMT I1RoYXd0ZSBQZXJzb25hbCBGcmVlbWFpbCBJc3N1aW5nIENBMB4XDTA4MDIxMzE1MTUxN1oX DTA5MDIxMjE1MTUxN1owbDEOMAwGA1UEBBMFTXVlbnoxEDAOBgNVBCoTB0dlcmhhcmQxFjAU BgNVBAMTDUdlcmhhcmQgTXVlbnoxMDAuBgkqhkiG9w0BCQEWIW11ZW56QGluZm9ybWF0aWsu dW5pLXR1ZWJpbmdlbi5kZTCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAMZex/Sq sAxkzTVvKP/YAgkaeXA+ngH59Aa0bbRPsKOWzAndGqty5EKcEzrnKqEJ27qHFvoF/pHp88U2 7SJI/xbqkgWeV2jRaldipZQYlnjYLQcmb4cewIFuGRRSVrm3BquzX38aYazuE4+DVH2Z3a8z n0FcdMXhA1NR2Ma1rh4G7SIeZ+hC7czbvNRPraBliGdQhs8J/6yP/iL8aNYAl9c7CL4ofRj8 Y9orMOV/4vtWTq76/VQUVdbhUMiv0D8aHqI1ZvGskhRRvmITgQRVbbn8N8WTpZ0UCgMDjxPP 9i5IhLfp6oBtsKl4OZ0RXvSLZrbJTkBX3vnEutcyxDvyNgMCAwEAAaM+MDwwLAYDVR0RBCUw I4EhbXVlbnpAaW5mb3JtYXRpay51bmktdHVlYmluZ2VuLmRlMAwGA1UdEwEB/wQCMAAwDQYJ KoZIhvcNAQEFBQADgYEAX5SiD6epJePwBjJumOsTF6wzeuZRDLYlN+fOpXwd2C0Yx6i8iIZ9 l/J/nGaE1YpJPfX5oJDE+tOk1vYh2E9ThLOj9kJ3buZmgOCdVu90qtCWhfhli7RCYcJ+G9M3 FCnqbrzI/waPPXGB8/DY1HKgPj5G+oKPUK+GD2aE1Q3PYGowggMVMIICfqADAgECAhA/WhrG FpDK/rOM5sjg4QfiMA0GCSqGSIb3DQEBBQUAMGIxCzAJBgNVBAYTAlpBMSUwIwYDVQQKExxU aGF3dGUgQ29uc3VsdGluZyAoUHR5KSBMdGQuMSwwKgYDVQQDEyNUaGF3dGUgUGVyc29uYWwg RnJlZW1haWwgSXNzdWluZyBDQTAeFw0wODAyMTMxNTE1MTdaFw0wOTAyMTIxNTE1MTdaMGwx DjAMBgNVBAQTBU11ZW56MRAwDgYDVQQqEwdHZXJoYXJkMRYwFAYDVQQDEw1HZXJoYXJkIE11 ZW56MTAwLgYJKoZIhvcNAQkBFiFtdWVuekBpbmZvcm1hdGlrLnVuaS10dWViaW5nZW4uZGUw ggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQDGXsf0qrAMZM01byj/2AIJGnlwPp4B +fQGtG20T7CjlswJ3RqrcuRCnBM65yqhCdu6hxb6Bf6R6fPFNu0iSP8W6pIFnldo0WpXYqWU GJZ42C0HJm+HHsCBbhkUUla5twars19/GmGs7hOPg1R9md2vM5ago, the EMANICS network of excellence > is currently preparing the > EMANICS Workshop on Netflow/IPFIX usage in network management > http://www.simpleweb.org/netflow/ > The workshop will take place October 30, at the LRZ Munich, Germany. >=20 > We would highly appreciate your feedback and, of course, your > participation. In addition, we kindly ask you to express your opinion > on making the workshop a joint Emanics&IRTF-NMRG event. >=20 > We want the workshop to be an opportunity for people to exchange and > discuss their experiences and ideas. To structure the discussions > the workshop will be organized into several discussion topics. > For each discussion topic, there will be short presentations (5-10 minu= tes) > introducing the topic, followed by a discussion round. >=20 > Currently, the following topics are planned: > 1. "What technologies are available to analyze flow data?" > Presentations: > * Flow record query languages. > * Using SQL databases for flow processing. > 2. "What can be expected from distributed analysis techniques?" > Presentations: > * A distributed architecture for IP flow analysis. > 3. "Should we have a standard format for the annotation of Netflow/I= PFIX traces?" > Presentations: > * Labeling flow traces for intrusion detection evaluation. > 4. "For what kind of applications can Netflow/IPFIX be used?" > Presentations: > * Network anomaly characterization using flows. > * Application level dependency discovery with flows. > * Management of light paths in optical networks using flows. >=20 > The above list is still preliminary and we welcome your contributions > and suggestions. >=20 > Regards, > Ramin Sadre >=20 --=20 Dipl.-Ing. Gerhard M=FCnz Computer Networks and Internet Wilhelm Schickard Institute for Computer Science University of Tuebingen Sand 13 (Room B309), D-72076 Tuebingen, Germany Phone: +49 7071 29-70534 Fax: +49 7071 29-5220 E-mail: muenz@informatik.uni-tuebingen.de WWW: http://net.informatik.uni-tuebingen.de/~muenz --------------ms050705060205010503030406 Content-Type: application/x-pkcs7-signature; name="smime.p7s" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="smime.p7s" Content-Description: S/MIME Cryptographic Signature MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAQAAoIIJdTCC AxUwggJ+oAMCAQICED9aGsYWkMr+s4zmyODhB+IwDQYJKoZIhvcNAQEFBQAwYjELMAkGA1UE BhMCWkExJTAjBgNVBAoTHFRoYXd0ZSBDb25zdWx0aW5nIChQdHkpIEx0ZC4xLDAqBgNVBAMT I1RoYXd0ZSBQZXJzb25hbCBGcmVlbWFpbCBJc3N1aW5nIENBMB4XDTA4MDIxMzE1MTUxN1oX DTA5MDIxMjE1MTUxN1owbDEOMAwGA1UEBBMFTXVlbnoxEDAOBgNVBCoTB0dlcmhhcmQxFjAU BgNVBAMTDUdlcmhhcmQgTXVlbnoxMDAuBgkqhkiG9w0BCQEWIW11ZW56QGluZm9ybWF0aWsu dW5pLXR1ZWJpbmdlbi5kZTCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAMZex/Sq sAxkzTVvKP/YAgkaeXA+ngH59Aa0bbRPsKOWzAndGqty5EKcEzrnKqEJ27qHFvoF/pHp88U2 7SJI/xbqkgWeV2jRaldipZQYlnjYLQcmb4cewIFuGRRSVrm3BquzX38aYazuE4+DVH2Z3a8z n0FcdMXhA1NR2Ma1rh4G7SIeZ+hC7czbvNRPraBliGdQhs8J/6yP/iL8aNYAl9c7CL4ofRj8 Y9orMOV/4vtWTq76/VQUVdbhUMiv0D8aHqI1ZvGskhRRvmITgQRVbbn8N8WTpZ0UCgMDjxPP 9i5IhLfp6oBtsKl4OZ0RXvSLZrbJTkBX3vnEutcyxDvyNgMCAwEAAaM+MDwwLAYDVR0RBCUw I4EhbXVlbnpAaW5mb3JtYXRpay51bmktdHVlYmluZ2VuLmRlMAwGA1UdEwEB/wQCMAAwDQYJ KoZIhvcNAQEFBQADgYEAX5SiD6epJePwBjJumOsTF6wzeuZRDLYlN+fOpXwd2C0Yx6i8iIZ9 l/J/nGaE1YpJPfX5oJDE+tOk1vYh2E9ThLOj9kJ3buZmgOCdVu90qtCWhfhli7RCYcJ+G9M3 FCnqbrzI/waPPXGB8/DY1HKgPj5G+oKPUK+GD2aE1Q3PYGowggMVMIICfqADAgECAhA/WhrG FpDK/rOM5sjg4QfiMA0GCSqGSIb3DQEBBQUAMGIxCzAJBgNVBAYTAlpBMSUwIwYDVQQKExxU aGF3dGUgQ29uc3VsdGluZyAoUHR5KSBMdGQuMSwwKgYDVQQDEyNUaGF3dGUgUGVyc29uYWwg RnJlZW1haWwgSXNzdWluZyBDQTAeFw0wODAyMTMxNTE1MTdaFw0wOTAyMTIxNTE1MTdaMGwx DjAMBgNVBAQTBU11ZW56MRAwDgYDVQQqEwdHZXJoYXJkMRYwFAYDVQQDEw1HZXJoYXJkIE11 ZW56MTAwLgYJKoZIhvcNAQkBFiFtdWVuekBpbmZvcm1hdGlrLnVuaS10dWViaW5nZW4uZGUw ggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQDGXsf0qrAMZM01byj/2AIJGnlwPp4B +fQGtG20T7CjlswJ3RqrcuRCnBM65yqhCdu6hxb6Bf6R6fPFNu0iSP8W6pIFnldo0WpXYqWU GJZ42C0HJm+HHsCBbhkUUla5twars19/GmGs7hOPg1R9md2vM59BXHTF4QNTUdjGta4eBu0i HmfoQu3M27zUT62gZYhnUIbPCf+sj/4i/GjWAJfXOwi+KH0Y/GPaKzDlf+L7Vk6u+v1UFFXW 4VDIr9A/Gh6iNWbxrJIUUb5iE4EEVW25/DfFk6WdFAoDA48Tz/YuSIS36eqAbbCpeDmdEV70 i2a2yU5AV975xLrXMsQ78jYDAgMBAAGjPjA8MCwGA1UdEQQlMCOBIW11ZW56QGluZm9ybWF0 aWsudW5pLXR1ZWJpbmdlbi5kZTAMBgNVHRMBAf8EAjAAMA0GCSqGSIb3DQEBBQUAA4GBAF+U og+nqSXj8AYybpjrExesM3rmUQy2JTfnzqV8HdgtGMeovIiGfZfyf5xmhNWKST31+aCQxPrT pNb2IdhPU4Szo/ZCd27mZoDgnVbvdKrQloX4ZYu0QmHCfhvTNxQp6m68yP8Gjz1xgfPw2NRy oD4+RvqCj1Cvhg9mhNUNz2BqMIIDPzCCAqigAwIBAgIBDTANBgkqhkiG9w0BAQUFADCB0TEL MAkGA1UEBhMCWkExFTATBgNVBAgTDFdlc3Rlcm4gQ2FwZTESMBAGA1UEBxMJQ2FwZSBUb3du MRowGAYDVQQKExFUaGF3dGUgQ29uc3VsdGluZzEoMCYGA1UECxMfQ2VydGlmaWNhdGlvbiBT ZXJ2aWNlcyBEaXZpc2lvbjEkMCIGA1UEAxMbVGhhd3RlIFBlcnNvbmFsIEZyZWVtYWlsIENB MSswKQYJKoZIhvcNAQkBFhxwZXJzb25hbC1mcmVlbWFpbEB0aGF3dGUuY29tMB4XDTAzMDcx NzAwMDAwMFoXDTEzMDcxNjIzNTk1OVowYjELMAkGA1UEBhMCWkExJTAjBgNVBAoTHFRoYXd0 ZSBDb25zdWx0aW5nIChQdHkpIEx0ZC4xLDAqBgNVBAMTI1RoYXd0ZSBQZXJzb25hbCBGcmVl bWFpbCBJc3N1aW5nIENBMIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQDEpjxVc1X7TrnK mVoeaMB1BHCd3+n/ox7svc31W/Iadr1/DDph8r9RzgHU5VAKMNcCY1osiRVwjt3J8CuFWqo/ cVbLrzwLB+fxH5E2JCoTzyvV84J3PQO+K/67GD4Hv0CAAmTXp6a7n2XRxSpUhQ9IBH+nttE8 YQRAHmQZcmC3+wIDAQABo4GUMIGRMBIGA1UdEwEB/wQIMAYBAf8CAQAwQwYDVR0fBDwwOjA4 oDagNIYyaHR0cDovL2NybC50aGF3dGUuY29tL1RoYXd0ZVBlcnNvbmFsRnJlZW1haWxDQS5j cmwwCwYDVR0PBAQDAgEGMCkGA1UdEQQiMCCkHjAcMRowGAYDVQQDExFQcml2YXRlTGFiZWwy LTEzODANBgkqhkiG9w0BAQUFAAOBgQBIjNFQg+oLLswNo2asZw9/r6y+whehQ5aUnX9MIbj4 Nh+qLZ82L8D0HFAgk3A8/a3hYWLD2ToZfoSxmRsAxRoLgnSeJVCUYsfbJ3FXJY3dqZw5jowg T2Vfldr394fWxghOrvbqNOUQGls1TXfjViF4gtwhGTXeJLHTHUb/XV9lTzGCA2QwggNgAgEB MHYwYjELMAkGA1UEBhMCWkExJTAjBgNVBAoTHFRoYXd0ZSBDb25zdWx0aW5nIChQdHkpIEx0 ZC4xLDAqBgNVBAMTI1RoYXd0ZSBQZXJzb25hbCBGcmVlbWFpbCBJc3N1aW5nIENBAhA/WhrG FpDK/rOM5sjg4QfiMAkGBSsOAwIaBQCgggHDMBgGCSqGSIb3DQEJAzELBgkqhkiG9w0BBwEw HAYJKoZIhvcNAQkFMQ8XDTA4MDgyMDA5MDEwMlowIwYJKoZIhvcNAQkEMRYEFLLHi7oLRsG0 z3K0IchvKucHywWkMFIGCSqGSIb3DQEJDzFFMEMwCgYIKoZIhvcNAwcwDgYIKoZIhvcNAwIC AgCAMA0GCCqGSIb3DQMCAgFAMAcGBSsOAwIHMA0GCCqGSIb3DQMCAgEoMIGFBgkrBgEEAYI3 EAQxeDB2MGIxCzAJBgNVBAYTAlpBMSUwIwYDVQQKExxUaGF3dGUgQ29uc3VsdGluZyAoUHR5 KSBMdGQuMSwwKgYDVQQDEyNUaGF3dGUgUGVyc29uYWwgRnJlZW1haWwgSXNzdWluZyBDQQIQ P1oaxhaQyv6zjObI4OEH4jCBhwYLKoZIhvcNAQkQAgsxeKB2MGIxCzAJBgNVBAYTAlpBMSUw IwYDVQQKExxUaGF3dGUgQ29uc3VsdGluZyAoUHR5KSBMdGQuMSwwKgYDVQQDEyNUaGF3dGUg UGVyc29uYWwgRnJlZW1haWwgSXNzdWluZyBDQQIQP1oaxhaQyv6zjObI4OEH4jANBgkqhkiG 9w0BAQEFAASCAQAc6BrsrGVjPCrQML47h/Dy2sxHyJ0eJkl6tTsilShOXvmdW6l5BpQeS169 qEcPepnYbsMARHpIQIifZTUxWvOBr27oOlmjTRGCb1CKVp3a5NLFTAhydGj1o3oQTOr6V/Ic l1dGy9ygM/pkQ/tVUL6twLT5tZ0OP9J8hOVskUqdwvkU/EvZJoIBwpsKe6tt8+ehnalx9KME 2O/z13zmImXx3fbuvqKwuJNVPD3B+/LmX4ghiFkbiFSQBbJusWadurSenxpz4yCX9ZY0l5b0 Pjb7e9ahZT6WkgRydlRbXrSBOy4v8hJfQY1oirHn0ZIFzVoLmnhB9I/fKmSxhlryk3s/AAAA AAAA --------------ms050705060205010503030406-- --===============0694312984== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ IPFIX mailing list IPFIX@ietf.org https://www.ietf.org/mailman/listinfo/ipfix --===============0694312984==-- 9BXHTF4QNTUdjGta4eBu0i HmfoQu3M27zUT62gZYhnUIbPCf+sj/4i/GjWAJfXOwi+KH0Y/GPaKzDlf+L7Vk6u+v1UFFXW 4VDIr9A/Gh6iNWbxrJIUUb5iE4EEVW25/DfFk6WdFAoDA48Tz/YuSIS36eqAbbCpeDmdEV70 i2a2yU5AV975xLrXMsQ78jYDAgMBAAGjPjA8MCwGA1UdEQQlMCOBIW11ZW56QGluZm9ybWF0 aWsudW5pLXR1ZWJpbmdlbi5kZTAMBgNVHRMBAf8EAjAAMA0GCSqGSIb3DQEBBQUAA4GBAF+U og+nqSXj8AYybpjrExesM3rmUQy2JTfnzqV8HdgtGMeovIiGfZfyf5xmhNWKST31+aCQxPrT pNb2IdhPU4Szo/ZCd27mZoDgnVbvdKrQloX4ZYu0QmHCfhvTNxQp6m68yP8Gjz1xgfPw2NRy oD4+RvqCj1Cvhg9mhNUNz2BqMIIDPzCCAqigAwIBAgIBDTANBgkqhkiG9w0BAQUFADCB0TEL MAkGA1UEBhMCWkExFTATBgNVBAgTDFdlc3Rlcm4gQ2FwZTESMBAGA1UEBxMJQ2FwZSBUb3du MRowGAYDVQQKExFUaGF3dGUgQ29uc3VsdGluZzEoMCYGA1UECxMfQ2VydGlmaWNhdGlvbiBT ZXJ2aWNlcyBEaXZpc2lvbjEkMCIGA1UEAxMbVGhhd3RlIFBlcnNvbmFsIEZyZWVtYWlsIENB MSswKQYJKoZIhvcNAQkBFhxwZXJzb25hbC1mcmVlbWFpbEB0aGF3dGUuY29tMB4XDTAzMDcx NzAwMDAwMFoXDTEzMDcxNjIzNTk1OVowYjELMAkGA1UEBhMCWkExJTAjBgNVBAoTHFRoYXd0 ZSBDb25zdWx0aW5nIChQdHkpIEx0ZC4xLDAqBgNVBAMTI1RoYXd0ZSBQZXJzb25hbCBGcmVl bWFpbCBJc3N1aW5nIENBMIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQDEpjxVc1X7TrnK mVoeaMB1BHCd3+n/ox7svc31W/Iadr1/DDph8r9RzgHU5VAKMNcCY1osiRVwjt3J8CuFWqo/ cVbLrzwLB+fxH5E2JCoTzyvV84J3PQO+K/67GD4Hv0CAAmTXp6a7n2XRxSpUhQ9IBH+nttE8 YQRAHmQZcmC3+wIDAQABo4GUMIGRMBIGA1UdEwEB/wQIMAYBAf8CAQAwQwYDVR0fBDwwOjA4 oDagNIYyaHR0cDovL2NybC50aGF3dGUuY29tL1RoYXd0ZVBlcnNvbmFsRnJlZW1haWxDQS5j cmwwCwYDVR0PBAQDAgEGMCkGA1UdEQQiMCCkHjAcMRowGAYDVQQDExFQcml2YXRlTGFiZWwy LTEzODANBgkqhkiG9w0BAQUFAAOBgQBIjNFQg+oLLswNo2asZw9/r6y+whehQ5aUnX9MIbj4 Nh+qLZ82L8D0HFAgk3A8/a3hYWLD2ToZfoSxmRsAxRoLgnSeJVCUYsfbJ3FXJY3dqZw5jowg T2Vfldr394fWxghOrvbqNOUQGls1TXfjViF4gtwhGTXeJLHTHUb/XV9lTzGCA2QwggNgAgEB MHYwYjELMAkGA1UEBhMCWkExJTAjBgNVBAoTHFRoYXd0ZSBDb25zdWx0aW5nIChQdHkpIEx0 ZC4xLDAqBgNVBAMTI1RoYXd0ZSBQZXJzb25hbCBGcmVlbWFpbCBJc3N1aW5nIENBAhA/WhrG FpDK/rOM5sjg4QfiMAkGBSsOAwIaBQCgggHDMBgGCSqGSIb3DQEJAzELBgkqhkiG9w0BBwEw HAYJKoZIhvcNAQkFMQ8XDTA4MDgyMDA5MDEwMlowIwYJKoZIhvcNAQkEMRYEFLLHi7oLRsG0 z3K0IchvKucHywWkMFIGCSqGSIb3DQEJDzFFMEMwCgYIKoZIhvcNAwcwDgYIKoZIhvcNAwIC AgCAMA0GCCqGSIb3DQMCAgFAMAcGBSsOAwIHMA0GCCqGSIb3DQMCAgEoMIGFBgkrBgEEAYI3 EAQxeDB2MGIxCzAJBgNVBAYTAlpBMSUwIwYDVQQKExxUaGF3dGUgQ29uc3VsdGluZyAoUHR5 KSBMdGQuMSwwKgYDVQQDEyNUaGF3dGUgUGVyc29uYWwgRnJlZW1haWwgSXNzdWluZyBDQQIQ P1oaxhaQyv6zjObI4OEH4jCBhwYLKoZIhvcNAQkQAgsxeKB2MGIxCzAJBgNVBAYTAlpBMSUw IwYDVQQKExxUaGF3dGUgQ29uc3VsdGluZyAoUHR5KSBMdGQuMSwwKgYDVQQDEyNUaGF3dGUg UGVyc29uYWwgRnJlZW1haWwgSXNzdWluZyBDQQIQP1oaxhaQyv6zjObI4OEH4jANBgkqhkiG 9w0BAQEFAASCAQAc6BrsrGVjPCrQML47h/Dy2sxHyJ0eJkl6tTsilShOXvmdW6l5BpQeS169 qEcPepnYbsMARHpIQIifZTUxWvOBr27oOlmjTRGCb1CKVp3a5NLFTAhydGj1o3oQTOr6V/Ic l1dGy9ygM/pkQ/tVUL6twLT5tZ0OP9J8hOVskUqdwvkU/EvZJoIBwpsKe6tt8+ehnalx9KME 2O/z13zmImXx3fbuvqKwuJNVPD3B+/LmX4ghiFkbiFSQBbJusWadurSenxpz4yCX9ZY0l5b0 Pjb7e9ahZT6WkgRydlRbXrSBOy4v8hJfQY1oirHn0ZIFzVoLmnhB9I/fKmSxhlryk3s/AAAA AAAA --------------ms050705060205010503030406-- --===============0694312984== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ IPFIX mailing list IPFIX@ietf.org https://www.ietf.org/mailman/listinfo/ipfix --===============0694312984==-- From ipfix-bounces@ietf.org Wed Aug 20 03:47:38 2008 Return-Path: X-Original-To: ipfix-archive@lists.ietf.org Delivered-To: ietfarch-ipfix-archive@core3.amsl.com Received: from [127.0.0.1] (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 33EA23A6BCC; Wed, 20 Aug 2008 03:47:38 -0700 (PDT) X-Original-To: ipfix@core3.amsl.com Delivered-To: ipfix@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 003BA3A6BCC for ; Wed, 20 Aug 2008 03:47:37 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.6 X-Spam-Level: X-Spam-Status: No, score=-2.6 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, NO_RELAYS=-0.001] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id zA+5Xd45+UjS for ; Wed, 20 Aug 2008 03:47:32 -0700 (PDT) Received: from mail.nttv6.net (mail.nttv6.net [IPv6:2001:fa8::25]) by core3.amsl.com (Postfix) with ESMTP id 0C9DE3A695B for ; Wed, 20 Aug 2008 03:47:31 -0700 (PDT) Received: from localhost ([IPv6:2001:fa8:1000:0:e909:51aa:c18a:d513]) by mail.nttv6.net (8.14.3/8.14.3) with ESMTP id m7KAlK1u000410; Wed, 20 Aug 2008 19:47:20 +0900 (JST) (envelope-from akoba@nttv6.net) Date: Wed, 20 Aug 2008 19:44:06 +0900 From: Atsushi Kobayashi To: ipfix@ietf.org, nmrg@ibr.cs.tu-bs.de In-Reply-To: <20080820071212.GC17541@elstar.local> References: <20080820071212.GC17541@elstar.local> Message-Id: <20080820190430.2B60.17391CF2@nttv6.net> MIME-Version: 1.0 X-Mailer: Becky! ver. 2.46 [ja] X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.0 (mail.nttv6.net [IPv6:2001:fa8::25]); Wed, 20 Aug 2008 19:47:24 +0900 (JST) Subject: Re: [IPFIX] Netflow/IPFIX workshop: information and request for feedback X-BeenThere: ipfix@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: IPFIX WG discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: ipfix-bounces@ietf.org Errors-To: ipfix-bounces@ietf.org Hi, I am interested in the workshop. I have one question. Does scope include the new feature of IPFIX/PSAMP? I think that NetFlow usage is limited to anomaly traffic detection, accounting system or traffic engineering. Definitely, they are main scope of NetFlow/IPFIX. But, I believe that new features of IPFIX/PSAMP can apply a passive measurement of network performance, and so on. I feel that many network operators haven't recognized new feature of IPFIX/PSAMP. Although VoIP and IPTV services are expanding, it is troublesome to introduce the passive measurement system as before. To cope with it and accelerate an implementation of IPFIX/PSAMP, I hope that case studies using IPFIX/PSAMP are considered more and more in several workshops and communities. Regards, Atsushi KOBAYASHI On Wed, 20 Aug 2008 09:12:12 +0200 Juergen Schoenwaelder wrote: > Hi, > > the following Netflow/IPFIX workshop/meeting proposal was sent to the > Network Management Research Group (NMRG) mailing list on July 31st. I > like to know whether there is support for such an event form the IPFIX > community. Please drop a note if you plan to contribute so that I can > gauge interest. > > If you have questions concerning the scope and organization, please > contact Ramin Sadre. If the question is of general interest, please > keep the NMRG list on CC for now. > > /js > > ----- Forwarded message from Ramin Sadre ----- > > Date: Thu, 31 Jul 2008 17:37:12 +0200 > From: Ramin Sadre > User-Agent: Thunderbird 2.0.0.16 (Windows/20080708) > To: nmrg@ibr.cs.tu-bs.de > Subject: [nmrg] Netflow/IPFIX workshop: information and request for feedback > > Dear list members, > > (we apologize for multiple postings) > > as already announced some days ago, the EMANICS network of excellence > is currently preparing the > EMANICS Workshop on Netflow/IPFIX usage in network management > http://www.simpleweb.org/netflow/ > The workshop will take place October 30, at the LRZ Munich, Germany. > > We would highly appreciate your feedback and, of course, your > participation. In addition, we kindly ask you to express your opinion > on making the workshop a joint Emanics&IRTF-NMRG event. > > We want the workshop to be an opportunity for people to exchange and > discuss their experiences and ideas. To structure the discussions > the workshop will be organized into several discussion topics. > For each discussion topic, there will be short presentations (5-10 minutes) > introducing the topic, followed by a discussion round. > > Currently, the following topics are planned: > 1. "What technologies are available to analyze flow data?" > Presentations: > * Flow record query languages. > * Using SQL databases for flow processing. > 2. "What can be expected from distributed analysis techniques?" > Presentations: > * A distributed architecture for IP flow analysis. > 3. "Should we have a standard format for the annotation of Netflow/IPFIX traces?" > Presentations: > * Labeling flow traces for intrusion detection evaluation. > 4. "For what kind of applications can Netflow/IPFIX be used?" > Presentations: > * Network anomaly characterization using flows. > * Application level dependency discovery with flows. > * Management of light paths in optical networks using flows. > > The above list is still preliminary and we welcome your contributions > and suggestions. > > Regards, > Ramin Sadre > > -- > !! This message is brought to you via the `nmrg' mailing list. > !! Please do not reply to this message to unsubscribe. To unsubscribe or adjust > !! your settings, send a mail message to > !! or look at https://mail.ibr.cs.tu-bs.de/mailman/listinfo/nmrg. > > ----- End forwarded message ----- > _______________________________________________ > IPFIX mailing list > IPFIX@ietf.org > https://www.ietf.org/mailman/listinfo/ipfix --- Atsushi KOBAYASHI NTT Information Sharing Platform Lab. tel:+81-(0)422-59-3978 fax:+81-(0)422-59-5637 _______________________________________________ IPFIX mailing list IPFIX@ietf.org https://www.ietf.org/mailman/listinfo/ipfix From ipfix-bounces@ietf.org Wed Aug 20 03:47:38 2008 Return-Path: X-Original-To: ipfix-archive@optimus.ietf.org Delivered-To: ietfarch-ipfix-archive@core3.amsl.com Received: from [127.0.0.1] (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 33EA23A6BCC; Wed, 20 Aug 2008 03:47:38 -0700 (PDT) X-Original-To: ipfix@core3.amsl.com Delivered-To: ipfix@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 003BA3A6BCC for ; Wed, 20 Aug 2008 03:47:37 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.6 X-Spam-Level: X-Spam-Status: No, score=-2.6 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, NO_RELAYS=-0.001] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id zA+5Xd45+UjS for ; Wed, 20 Aug 2008 03:47:32 -0700 (PDT) Received: from mail.nttv6.net (mail.nttv6.net [IPv6:2001:fa8::25]) by core3.amsl.com (Postfix) with ESMTP id 0C9DE3A695B for ; Wed, 20 Aug 2008 03:47:31 -0700 (PDT) Received: from localhost ([IPv6:2001:fa8:1000:0:e909:51aa:c18a:d513]) by mail.nttv6.net (8.14.3/8.14.3) with ESMTP id m7KAlK1u000410; Wed, 20 Aug 2008 19:47:20 +0900 (JST) (envelope-from akoba@nttv6.net) Date: Wed, 20 Aug 2008 19:44:06 +0900 From: Atsushi Kobayashi To: ipfix@ietf.org, nmrg@ibr.cs.tu-bs.de In-Reply-To: <20080820071212.GC17541@elstar.local> References: <20080820071212.GC17541@elstar.local> Message-Id: <20080820190430.2B60.17391CF2@nttv6.net> MIME-Version: 1.0 X-Mailer: Becky! ver. 2.46 [ja] X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.0 (mail.nttv6.net [IPv6:2001:fa8::25]); Wed, 20 Aug 2008 19:47:24 +0900 (JST) Subject: Re: [IPFIX] Netflow/IPFIX workshop: information and request for feedback X-BeenThere: ipfix@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: IPFIX WG discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: ipfix-bounces@ietf.org Errors-To: ipfix-bounces@ietf.org Hi, I am interested in the workshop. I have one question. Does scope include the new feature of IPFIX/PSAMP? I think that NetFlow usage is limited to anomaly traffic detection, accounting system or traffic engineering. Definitely, they are main scope of NetFlow/IPFIX. But, I believe that new features of IPFIX/PSAMP can apply a passive measurement of network performance, and so on. I feel that many network operators haven't recognized new feature of IPFIX/PSAMP. Although VoIP and IPTV services are expanding, it is troublesome to introduce the passive measurement system as before. To cope with it and accelerate an implementation of IPFIX/PSAMP, I hope that case studies using IPFIX/PSAMP are considered more and more in several workshops and communities. Regards, Atsushi KOBAYASHI On Wed, 20 Aug 2008 09:12:12 +0200 Juergen Schoenwaelder wrote: > Hi, > > the following Netflow/IPFIX workshop/meeting proposal was sent to the > Network Management Research Group (NMRG) mailing list on July 31st. I > like to know whether there is support for such an event form the IPFIX > community. Please drop a note if you plan to contribute so that I can > gauge interest. > > If you have questions concerning the scope and organization, please > contact Ramin Sadre. If the question is of general interest, please > keep the NMRG list on CC for now. > > /js > > ----- Forwarded message from Ramin Sadre ----- > > Date: Thu, 31 Jul 2008 17:37:12 +0200 > From: Ramin Sadre > User-Agent: Thunderbird 2.0.0.16 (Windows/20080708) > To: nmrg@ibr.cs.tu-bs.de > Subject: [nmrg] Netflow/IPFIX workshop: information and request for feedback > > Dear list members, > > (we apologize for multiple postings) > > as already announced some days ago, the EMANICS network of excellence > is currently preparing the > EMANICS Workshop on Netflow/IPFIX usage in network management > http://www.simpleweb.org/netflow/ > The workshop will take place October 30, at the LRZ Munich, Germany. > > We would highly appreciate your feedback and, of course, your > participation. In addition, we kindly ask you to express your opinion > on making the workshop a joint Emanics&IRTF-NMRG event. > > We want the workshop to be an opportunity for people to exchange and > discuss their experiences and ideas. To structure the discussions > the workshop will be organized into several discussion topics. > For each discussion topic, there will be short presentations (5-10 minutes) > introducing the topic, followed by a discussion round. > > Currently, the following topics are planned: > 1. "What technologies are available to analyze flow data?" > Presentations: > * Flow record query languages. > * Using SQL databases for flow processing. > 2. "What can be expected from distributed analysis techniques?" > Presentations: > * A distributed architecture for IP flow analysis. > 3. "Should we have a standard format for the annotation of Netflow/IPFIX traces?" > Presentations: > * Labeling flow traces for intrusion detection evaluation. > 4. "For what kind of applications can Netflow/IPFIX be used?" > Presentations: > * Network anomaly characterization using flows. > * Application level dependency discovery with flows. > * Management of light paths in optical networks using flows. > > The above list is still preliminary and we welcome your contributions > and suggestions. > > Regards, > Ramin Sadre > > -- > !! This message is brought to you via the `nmrg' mailing list. > !! Please do not reply to this message to unsubscribe. To unsubscribe or adjust > !! your settings, send a mail message to > !! or look at https://mail.ibr.cs.tu-bs.de/mailman/listinfo/nmrg. > > ----- End forwarded message ----- > _______________________________________________ > IPFIX mailing list > IPFIX@ietf.org > https://www.ietf.org/mailman/listinfo/ipfix --- Atsushi KOBAYASHI NTT Information Sharing Platform Lab. tel:+81-(0)422-59-3978 fax:+81-(0)422-59-5637 _______________________________________________ IPFIX mailing list IPFIX@ietf.org https://www.ietf.org/mailman/listinfo/ipfix From ipfix-bounces@ietf.org Wed Aug 20 07:44:20 2008 Return-Path: X-Original-To: ipfix-archive@lists.ietf.org Delivered-To: ietfarch-ipfix-archive@core3.amsl.com Received: from [127.0.0.1] (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id A634D3A6C14; Wed, 20 Aug 2008 07:44:20 -0700 (PDT) X-Original-To: ipfix@core3.amsl.com Delivered-To: ipfix@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id DC7CB3A6C14 for ; Wed, 20 Aug 2008 07:44:19 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -0.39 X-Spam-Level: X-Spam-Status: No, score=-0.39 tagged_above=-999 required=5 tests=[BAYES_20=-0.74, HELO_EQ_DE=0.35] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id TmZbEdnMBQPF for ; Wed, 20 Aug 2008 07:44:16 -0700 (PDT) Received: from faui45.informatik.uni-erlangen.de (faui45.informatik.uni-erlangen.de [131.188.34.45]) by core3.amsl.com (Postfix) with ESMTP id 0A6673A6BF6 for ; Wed, 20 Aug 2008 07:44:16 -0700 (PDT) Received: from faui7s0.informatik.uni-erlangen.de (faui7s0.informatik.uni-erlangen.de [131.188.37.210]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by faui45.informatik.uni-erlangen.de (Postfix) with ESMTPS id C2FEA4828; Wed, 20 Aug 2008 16:43:57 +0200 (MEST) Received: from [131.188.37.96] (faui7m16 [131.188.37.96]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by faui7s0.informatik.uni-erlangen.de (Postfix) with ESMTP id A231BB0038; Wed, 20 Aug 2008 16:43:57 +0200 (CEST) Message-ID: <48AC2DAF.20605@informatik.uni-erlangen.de> Date: Wed, 20 Aug 2008 16:43:59 +0200 From: Christoph Sommer User-Agent: Thunderbird 2.0.0.16 (X11/20080724) MIME-Version: 1.0 To: ipfix@ietf.org, nmrg@ibr.cs.tu-bs.de, j.schoenwaelder@jacobs-university.de References: <20080820071212.GC17541@elstar.local> In-Reply-To: <20080820071212.GC17541@elstar.local> X-Enigmail-Version: 0.95.0 Subject: Re: [IPFIX] Netflow/IPFIX workshop: information and request for feedback X-BeenThere: ipfix@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: IPFIX WG discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: ipfix-bounces@ietf.org Errors-To: ipfix-bounces@ietf.org Juergen Schoenwaelder wrote: > Please drop a note if you plan to contribute so that I can > gauge interest. Sounds like a good opportunity to discuss some open issues, esp. regarding hierachical flow aggregation! I just sent an E-Mail to arrange for a timeslot. Kind regards, -- Christoph Sommer Computer Networks and Communication Systems University of Erlangen-Nuremberg, Germany Phone: +49 9131 85-27993 / Fax: +49 9131 85-27409 http://www7.informatik.uni-erlangen.de/~sommer/ _______________________________________________ IPFIX mailing list IPFIX@ietf.org https://www.ietf.org/mailman/listinfo/ipfix From ipfix-bounces@ietf.org Wed Aug 20 07:44:20 2008 Return-Path: X-Original-To: ipfix-archive@optimus.ietf.org Delivered-To: ietfarch-ipfix-archive@core3.amsl.com Received: from [127.0.0.1] (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id A634D3A6C14; Wed, 20 Aug 2008 07:44:20 -0700 (PDT) X-Original-To: ipfix@core3.amsl.com Delivered-To: ipfix@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id DC7CB3A6C14 for ; Wed, 20 Aug 2008 07:44:19 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -0.39 X-Spam-Level: X-Spam-Status: No, score=-0.39 tagged_above=-999 required=5 tests=[BAYES_20=-0.74, HELO_EQ_DE=0.35] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id TmZbEdnMBQPF for ; Wed, 20 Aug 2008 07:44:16 -0700 (PDT) Received: from faui45.informatik.uni-erlangen.de (faui45.informatik.uni-erlangen.de [131.188.34.45]) by core3.amsl.com (Postfix) with ESMTP id 0A6673A6BF6 for ; Wed, 20 Aug 2008 07:44:16 -0700 (PDT) Received: from faui7s0.informatik.uni-erlangen.de (faui7s0.informatik.uni-erlangen.de [131.188.37.210]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by faui45.informatik.uni-erlangen.de (Postfix) with ESMTPS id C2FEA4828; Wed, 20 Aug 2008 16:43:57 +0200 (MEST) Received: from [131.188.37.96] (faui7m16 [131.188.37.96]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by faui7s0.informatik.uni-erlangen.de (Postfix) with ESMTP id A231BB0038; Wed, 20 Aug 2008 16:43:57 +0200 (CEST) Message-ID: <48AC2DAF.20605@informatik.uni-erlangen.de> Date: Wed, 20 Aug 2008 16:43:59 +0200 From: Christoph Sommer User-Agent: Thunderbird 2.0.0.16 (X11/20080724) MIME-Version: 1.0 To: ipfix@ietf.org, nmrg@ibr.cs.tu-bs.de, j.schoenwaelder@jacobs-university.de References: <20080820071212.GC17541@elstar.local> In-Reply-To: <20080820071212.GC17541@elstar.local> X-Enigmail-Version: 0.95.0 Subject: Re: [IPFIX] Netflow/IPFIX workshop: information and request for feedback X-BeenThere: ipfix@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: IPFIX WG discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: ipfix-bounces@ietf.org Errors-To: ipfix-bounces@ietf.org Juergen Schoenwaelder wrote: > Please drop a note if you plan to contribute so that I can > gauge interest. Sounds like a good opportunity to discuss some open issues, esp. regarding hierachical flow aggregation! I just sent an E-Mail to arrange for a timeslot. Kind regards, -- Christoph Sommer Computer Networks and Communication Systems University of Erlangen-Nuremberg, Germany Phone: +49 9131 85-27993 / Fax: +49 9131 85-27409 http://www7.informatik.uni-erlangen.de/~sommer/ _______________________________________________ IPFIX mailing list IPFIX@ietf.org https://www.ietf.org/mailman/listinfo/ipfix From ipfix-bounces@ietf.org Wed Aug 20 09:12:22 2008 Return-Path: X-Original-To: ipfix-archive@lists.ietf.org Delivered-To: ietfarch-ipfix-archive@core3.amsl.com Received: from [127.0.0.1] (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 668B83A6C25; Wed, 20 Aug 2008 09:12:22 -0700 (PDT) X-Original-To: ipfix@core3.amsl.com Delivered-To: ipfix@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 4B3283A6841 for ; Wed, 20 Aug 2008 09:12:21 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.338 X-Spam-Level: X-Spam-Status: No, score=-1.338 tagged_above=-999 required=5 tests=[AWL=0.911, BAYES_00=-2.599, HELO_EQ_DE=0.35] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id lri8askos5xN for ; Wed, 20 Aug 2008 09:12:20 -0700 (PDT) Received: from hermes.jacobs-university.de (hermes.jacobs-university.de [212.201.44.23]) by core3.amsl.com (Postfix) with ESMTP id 535553A6C25 for ; Wed, 20 Aug 2008 09:12:20 -0700 (PDT) Received: from localhost (demetrius3.jacobs-university.de [212.201.44.48]) by hermes.jacobs-university.de (Postfix) with ESMTP id 3150CC0083; Wed, 20 Aug 2008 18:11:27 +0200 (CEST) X-Virus-Scanned: amavisd-new at jacobs-university.de Received: from hermes.jacobs-university.de ([212.201.44.23]) by localhost (demetrius3.jacobs-university.de [212.201.44.32]) (amavisd-new, port 10024) with ESMTP id lgZ0En8+H4wT; Wed, 20 Aug 2008 18:11:21 +0200 (CEST) Received: from elstar.local (elstar.iuhb02.iu-bremen.de [10.50.231.133]) by hermes.jacobs-university.de (Postfix) with ESMTP id 7D60FC0079; Wed, 20 Aug 2008 18:11:20 +0200 (CEST) Received: by elstar.local (Postfix, from userid 501) id BDBEF699040; Wed, 20 Aug 2008 18:11:19 +0200 (CEST) Date: Wed, 20 Aug 2008 18:11:19 +0200 From: Juergen Schoenwaelder To: Atsushi Kobayashi Message-ID: <20080820161119.GA19573@elstar.local> Mail-Followup-To: Atsushi Kobayashi , ipfix@ietf.org, nmrg@ibr.cs.tu-bs.de References: <20080820071212.GC17541@elstar.local> <20080820190430.2B60.17391CF2@nttv6.net> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20080820190430.2B60.17391CF2@nttv6.net> User-Agent: Mutt/1.5.18 (2008-05-17) Cc: nmrg@ibr.cs.tu-bs.de, ipfix@ietf.org Subject: Re: [IPFIX] Netflow/IPFIX workshop: information and request for feedback X-BeenThere: ipfix@ietf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: j.schoenwaelder@jacobs-university.de List-Id: IPFIX WG discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: ipfix-bounces@ietf.org Errors-To: ipfix-bounces@ietf.org On Wed, Aug 20, 2008 at 07:44:06PM +0900, Atsushi Kobayashi wrote: > I am interested in the workshop. I have one question. > > Does scope include the new feature of IPFIX/PSAMP? Ramin is the organizer of the workshop and he should provide an authoritative answer. However, I would be surprised if he would rule out discussion of new features or usage scenarios of IPFIX/PSAMP. /js -- Juergen Schoenwaelder Jacobs University Bremen gGmbH Phone: +49 421 200 3587 Campus Ring 1, 28759 Bremen, Germany Fax: +49 421 200 3103 _______________________________________________ IPFIX mailing list IPFIX@ietf.org https://www.ietf.org/mailman/listinfo/ipfix From ipfix-bounces@ietf.org Wed Aug 20 09:12:22 2008 Return-Path: X-Original-To: ipfix-archive@optimus.ietf.org Delivered-To: ietfarch-ipfix-archive@core3.amsl.com Received: from [127.0.0.1] (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 668B83A6C25; Wed, 20 Aug 2008 09:12:22 -0700 (PDT) X-Original-To: ipfix@core3.amsl.com Delivered-To: ipfix@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 4B3283A6841 for ; Wed, 20 Aug 2008 09:12:21 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.338 X-Spam-Level: X-Spam-Status: No, score=-1.338 tagged_above=-999 required=5 tests=[AWL=0.911, BAYES_00=-2.599, HELO_EQ_DE=0.35] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id lri8askos5xN for ; Wed, 20 Aug 2008 09:12:20 -0700 (PDT) Received: from hermes.jacobs-university.de (hermes.jacobs-university.de [212.201.44.23]) by core3.amsl.com (Postfix) with ESMTP id 535553A6C25 for ; Wed, 20 Aug 2008 09:12:20 -0700 (PDT) Received: from localhost (demetrius3.jacobs-university.de [212.201.44.48]) by hermes.jacobs-university.de (Postfix) with ESMTP id 3150CC0083; Wed, 20 Aug 2008 18:11:27 +0200 (CEST) X-Virus-Scanned: amavisd-new at jacobs-university.de Received: from hermes.jacobs-university.de ([212.201.44.23]) by localhost (demetrius3.jacobs-university.de [212.201.44.32]) (amavisd-new, port 10024) with ESMTP id lgZ0En8+H4wT; Wed, 20 Aug 2008 18:11:21 +0200 (CEST) Received: from elstar.local (elstar.iuhb02.iu-bremen.de [10.50.231.133]) by hermes.jacobs-university.de (Postfix) with ESMTP id 7D60FC0079; Wed, 20 Aug 2008 18:11:20 +0200 (CEST) Received: by elstar.local (Postfix, from userid 501) id BDBEF699040; Wed, 20 Aug 2008 18:11:19 +0200 (CEST) Date: Wed, 20 Aug 2008 18:11:19 +0200 From: Juergen Schoenwaelder To: Atsushi Kobayashi Message-ID: <20080820161119.GA19573@elstar.local> Mail-Followup-To: Atsushi Kobayashi , ipfix@ietf.org, nmrg@ibr.cs.tu-bs.de References: <20080820071212.GC17541@elstar.local> <20080820190430.2B60.17391CF2@nttv6.net> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20080820190430.2B60.17391CF2@nttv6.net> User-Agent: Mutt/1.5.18 (2008-05-17) Cc: nmrg@ibr.cs.tu-bs.de, ipfix@ietf.org Subject: Re: [IPFIX] Netflow/IPFIX workshop: information and request for feedback X-BeenThere: ipfix@ietf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: j.schoenwaelder@jacobs-university.de List-Id: IPFIX WG discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: ipfix-bounces@ietf.org Errors-To: ipfix-bounces@ietf.org On Wed, Aug 20, 2008 at 07:44:06PM +0900, Atsushi Kobayashi wrote: > I am interested in the workshop. I have one question. > > Does scope include the new feature of IPFIX/PSAMP? Ramin is the organizer of the workshop and he should provide an authoritative answer. However, I would be surprised if he would rule out discussion of new features or usage scenarios of IPFIX/PSAMP. /js -- Juergen Schoenwaelder Jacobs University Bremen gGmbH Phone: +49 421 200 3587 Campus Ring 1, 28759 Bremen, Germany Fax: +49 421 200 3103 _______________________________________________ IPFIX mailing list IPFIX@ietf.org https://www.ietf.org/mailman/listinfo/ipfix From ipfix-bounces@ietf.org Sun Aug 24 13:53:27 2008 Return-Path: X-Original-To: ipfix-archive@lists.ietf.org Delivered-To: ietfarch-ipfix-archive@core3.amsl.com Received: from [127.0.0.1] (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id BCDE03A68F0; Sun, 24 Aug 2008 13:53:27 -0700 (PDT) X-Original-To: ipfix@core3.amsl.com Delivered-To: ipfix@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id C960D3A6994 for ; Thu, 21 Aug 2008 03:13:07 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: 0.985 X-Spam-Level: X-Spam-Status: No, score=0.985 tagged_above=-999 required=5 tests=[BAYES_05=-1.11, HELO_EQ_NL=0.55, HOST_EQ_NL=1.545] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 0SxsQl-djZWc for ; Thu, 21 Aug 2008 03:13:07 -0700 (PDT) Received: from rotterdam.ewi.utwente.nl (rotterdam.ewi.utwente.nl [130.89.10.5]) by core3.amsl.com (Postfix) with ESMTP id C40FE3A68AC for ; Thu, 21 Aug 2008 03:13:06 -0700 (PDT) Received: from [127.0.0.1] (ewi1009.ewi.utwente.nl [130.89.11.34]) by rotterdam.ewi.utwente.nl (8.13.6/8.13.6) with ESMTP id m7LABfST008325; Thu, 21 Aug 2008 12:11:46 +0200 (MEST) Message-ID: <48AD3F58.203@ewi.utwente.nl> Date: Thu, 21 Aug 2008 12:11:36 +0200 From: Ramin Sadre User-Agent: Thunderbird 2.0.0.16 (Windows/20080708) MIME-Version: 1.0 To: Atsushi Kobayashi , ipfix@ietf.org, nmrg@ibr.cs.tu-bs.de References: <20080820071212.GC17541@elstar.local> <20080820190430.2B60.17391CF2@nttv6.net> <20080820161119.GA19573@elstar.local> In-Reply-To: <20080820161119.GA19573@elstar.local> X-Scanned-By: MIMEDefang 2.52 on 130.89.10.5 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-3.0rc3 (rotterdam.ewi.utwente.nl [130.89.10.5]); Thu, 21 Aug 2008 12:11:47 +0200 (MEST) X-Mailman-Approved-At: Sun, 24 Aug 2008 13:53:26 -0700 Subject: Re: [IPFIX] [nmrg] Re: Netflow/IPFIX workshop: information and request for feedback X-BeenThere: ipfix@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: IPFIX WG discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: ipfix-bounces@ietf.org Errors-To: ipfix-bounces@ietf.org >> I am interested in the workshop. I have one question. >> Does scope include the new feature of IPFIX/PSAMP? > Ramin is the organizer of the workshop and he should provide an > authoritative answer. However, I would be surprised if he would rule > out discussion of new features or usage scenarios of IPFIX/PSAMP. Indeed, discussion of features/usage of IPFIX/PSAMP is welcome. In this context, please note that we have recently added the topic "How do sampling and aggregation affect the volume and accuracy of data collection and analysis?" to the list of topics. Please also see http://www.simpleweb.org/netflow/ Regards, Ramin _______________________________________________ IPFIX mailing list IPFIX@ietf.org https://www.ietf.org/mailman/listinfo/ipfix From ipfix-bounces@ietf.org Sun Aug 24 13:53:27 2008 Return-Path: X-Original-To: ipfix-archive@optimus.ietf.org Delivered-To: ietfarch-ipfix-archive@core3.amsl.com Received: from [127.0.0.1] (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id BCDE03A68F0; Sun, 24 Aug 2008 13:53:27 -0700 (PDT) X-Original-To: ipfix@core3.amsl.com Delivered-To: ipfix@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id C960D3A6994 for ; Thu, 21 Aug 2008 03:13:07 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: 0.985 X-Spam-Level: X-Spam-Status: No, score=0.985 tagged_above=-999 required=5 tests=[BAYES_05=-1.11, HELO_EQ_NL=0.55, HOST_EQ_NL=1.545] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 0SxsQl-djZWc for ; Thu, 21 Aug 2008 03:13:07 -0700 (PDT) Received: from rotterdam.ewi.utwente.nl (rotterdam.ewi.utwente.nl [130.89.10.5]) by core3.amsl.com (Postfix) with ESMTP id C40FE3A68AC for ; Thu, 21 Aug 2008 03:13:06 -0700 (PDT) Received: from [127.0.0.1] (ewi1009.ewi.utwente.nl [130.89.11.34]) by rotterdam.ewi.utwente.nl (8.13.6/8.13.6) with ESMTP id m7LABfST008325; Thu, 21 Aug 2008 12:11:46 +0200 (MEST) Message-ID: <48AD3F58.203@ewi.utwente.nl> Date: Thu, 21 Aug 2008 12:11:36 +0200 From: Ramin Sadre User-Agent: Thunderbird 2.0.0.16 (Windows/20080708) MIME-Version: 1.0 To: Atsushi Kobayashi , ipfix@ietf.org, nmrg@ibr.cs.tu-bs.de References: <20080820071212.GC17541@elstar.local> <20080820190430.2B60.17391CF2@nttv6.net> <20080820161119.GA19573@elstar.local> In-Reply-To: <20080820161119.GA19573@elstar.local> X-Scanned-By: MIMEDefang 2.52 on 130.89.10.5 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-3.0rc3 (rotterdam.ewi.utwente.nl [130.89.10.5]); Thu, 21 Aug 2008 12:11:47 +0200 (MEST) X-Mailman-Approved-At: Sun, 24 Aug 2008 13:53:26 -0700 Subject: Re: [IPFIX] [nmrg] Re: Netflow/IPFIX workshop: information and request for feedback X-BeenThere: ipfix@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: IPFIX WG discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: ipfix-bounces@ietf.org Errors-To: ipfix-bounces@ietf.org >> I am interested in the workshop. I have one question. >> Does scope include the new feature of IPFIX/PSAMP? > Ramin is the organizer of the workshop and he should provide an > authoritative answer. However, I would be surprised if he would rule > out discussion of new features or usage scenarios of IPFIX/PSAMP. Indeed, discussion of features/usage of IPFIX/PSAMP is welcome. In this context, please note that we have recently added the topic "How do sampling and aggregation affect the volume and accuracy of data collection and analysis?" to the list of topics. Please also see http://www.simpleweb.org/netflow/ Regards, Ramin _______________________________________________ IPFIX mailing list IPFIX@ietf.org https://www.ietf.org/mailman/listinfo/ipfix From ipfix-bounces@ietf.org Tue Aug 26 03:36:05 2008 Return-Path: X-Original-To: ipfix-archive@lists.ietf.org Delivered-To: ietfarch-ipfix-archive@core3.amsl.com Received: from [127.0.0.1] (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id D890E3A6AD4; Tue, 26 Aug 2008 03:36:05 -0700 (PDT) X-Original-To: ipfix@core3.amsl.com Delivered-To: ipfix@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 96D4D3A6AD4 for ; Tue, 26 Aug 2008 03:36:05 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: 0.001 X-Spam-Level: X-Spam-Status: No, score=0.001 tagged_above=-999 required=5 tests=[BAYES_50=0.001] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id MlxuJjCyDLrf for ; Tue, 26 Aug 2008 03:36:04 -0700 (PDT) Received: from av-tac-bru.cisco.com (odd-brew.cisco.com [144.254.15.119]) by core3.amsl.com (Postfix) with ESMTP id 3F6A03A6AD1 for ; Tue, 26 Aug 2008 03:36:04 -0700 (PDT) X-TACSUNS: Virus Scanned Received: from strange-brew.cisco.com (localhost [127.0.0.1]) by av-tac-bru.cisco.com (8.11.7p3+Sun/8.11.7) with ESMTP id m7QATvd20716; Tue, 26 Aug 2008 12:29:58 +0200 (CEST) Received: from [10.61.80.219] (ams3-vpn-dhcp4316.cisco.com [10.61.80.219]) by strange-brew.cisco.com (8.11.7p3+Sun/8.11.7) with ESMTP id m7QATaA05727; Tue, 26 Aug 2008 12:29:46 +0200 (CEST) Message-ID: <48B3DB10.8090500@cisco.com> Date: Tue, 26 Aug 2008 12:29:36 +0200 From: Benoit Claise User-Agent: Thunderbird 1.5.0.12 (Windows/20070509) MIME-Version: 1.0 To: iana-prot-param@icann.org, Michelle Cotton , "ipfix@ietf.org" References: <48874293.4010705@cisco.com> <488CBA82.3060004@cisco.com> <48909E80.40604@cisco.com> In-Reply-To: Cc: Juergen Quittek , n.brownlee@auckland.ac.nz Subject: Re: [IPFIX] [IANA #144572] request for new IPFIX information elements - It's start to be really ridiculous... X-BeenThere: ipfix@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: IPFIX WG discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: ipfix-bounces@ietf.org Errors-To: ipfix-bounces@ietf.org Michelle, It's start to be really ridiculous... We understood that this IPFIX registry might be a little special and difficult due to - the history of NetFlow version 9, already implemented for years. - the fact that RFC5102 doesn't have a clear procedure for IANA ("MUST be published using a well-established and persistent publication medium") So, during IETF 71, we (Dan Romascanu, Nevil Brownlee, Brian Trammell, and I) came to you in person to explain everything. And again, during IETF 72, we (Juergen Quittek, Paul Aitken, Brian Trammel, and I) came to you in person to explain everything. It was supposed to be fixed within a week. And still no news. I'm not sure if you realize the mess in which my company is: we invented NetFlow, pushed through standardization via IPFIX, published all our information elements to the world, and now we're paying a big price for that. Because, obviously we kept implementing new Information Elements since we started this IPFIX effort at the IETF in 2001... So we have many groups within the company wanteFrom ipfix-bounces@ietf.org Tue Aug 26 03:36:05 2008 Return-Path: X-Original-To: ipfix-archive@optimus.ietf.org Delivered-To: ietfarch-ipfix-archive@core3.amsl.com Received: from [127.0.0.1] (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id D890E3A6AD4; Tue, 26 Aug 2008 03:36:05 -0700 (PDT) X-Original-To: ipfix@core3.amsl.com Delivered-To: ipfix@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 96D4D3A6AD4 for ; Tue, 26 Aug 2008 03:36:05 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: 0.001 X-Spam-Level: X-Spam-Status: No, score=0.001 tagged_above=-999 required=5 tests=[BAYES_50=0.001] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id MlxuJjCyDLrf for ; Tue, 26 Aug 2008 03:36:04 -0700 (PDT) Received: from av-tac-bru.cisco.com (odd-brew.cisco.com [144.254.15.119]) by core3.amsl.com (Postfix) with ESMTP id 3F6A03A6AD1 for ; Tue, 26 Aug 2008 03:36:04 -0700 (PDT) X-TACSUNS: Virus Scanned Received: from strange-brew.cisco.com (localhost [127.0.0.1]) by av-tac-bru.cisco.com (8.11.7p3+Sun/8.11.7) with ESMTP id m7QATvd20716; Tue, 26 Aug 2008 12:29:58 +0200 (CEST) Received: from [10.61.80.219] (ams3-vpn-dhcp4316.cisco.com [10.61.80.219]) by strange-brew.cisco.com (8.11.7p3+Sun/8.11.7) with ESMTP id m7QATaA05727; Tue, 26 Aug 2008 12:29:46 +0200 (CEST) Message-ID: <48B3DB10.8090500@cisco.com> Date: Tue, 26 Aug 2008 12:29:36 +0200 From: Benoit Claise User-Agent: Thunderbird 1.5.0.12 (Windows/20070509) MIME-Version: 1.0 To: iana-prot-param@icann.org, Michelle Cotton , "ipfix@ietf.org" References: <48874293.4010705@cisco.com> <488CBA82.3060004@cisco.com> <48909E80.40604@cisco.com> In-Reply-To: Cc: Juergen Quittek , n.brownlee@auckland.ac.nz Subject: Re: [IPFIX] [IANA #144572] request for new IPFIX information elements - It's start to be really ridiculous... X-BeenThere: ipfix@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: IPFIX WG discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: ipfix-bounces@ietf.org Errors-To: ipfix-bounces@ietf.org Michelle, It's start to be really ridiculous... We understood that this IPFIX registry might be a little special and difficult due to - the history of NetFlow version 9, already implemented for years. - the fact that RFC5102 doesn't have a clear procedure for IANA ("MUST be published using a well-established and persistent publication medium") So, during IETF 71, we (Dan Romascanu, Nevil Brownlee, Brian Trammell, and I) came to you in person to explain everything. And again, during IETF 72, we (Juergen Quittek, Paul Aitken, Brian Trammel, and I) came to you in person to explain everything. It was supposed to be fixed within a week. And still no news. I'm not sure if you realize the mess in which my company is: we invented NetFlow, pushed through standardization via IPFIX, published all our information elements to the world, and now we're paying a big price for that. Because, obviously we kept implementing new Information Elements since we started this IPFIX effort at the IETF in 2001... So we have many groups within the company wand new information element assignments. What should we tell them? - the IANA procedure is almost in place? We've been telling that for too long now... - assign the next one available but that may change? Great from an interoperability point of view... - wait? we can't anymore... The last time you assign an I.E. (239, biflowDirection), that forced the IPFIX WG to change the already agreed upon I.E. assignments ... which is the reason why we had to produce http://tools.ietf.org/id/draft-aitken-ipfix-new-infos-03.txt Regards, Benoit. > Benoit, > > It was great meeting with you all at this IETF. > > I have spoken with my XML folks. We are already working on it. > It should be done by the end of the week but I can't promise. > I'll send you all something as soon as it is ready as well as send you questions > if anything comes up. > > Patience :) > > This is one of my highest priorities.... > > Thanks, > > Michelle > > > > On Wed Jul 30 17:07:16 2008, bclaise@cisco.com wrote: > >> Michelle, >> >> Good talking to you the other day. >> Have you received some feedback from your XML expert? >> Do you have something to show us while we're still at the IETF? >> >> Regards, Benoit. >> >>>> Dear all, >>>> >>>> Tuesday 11h30 is fine for Paul and myself. >>>> What about you? >>>> >>>> Regards, Benoit. >>>> >>>>> Dear Michelle, >>>>> >>>>> I will be in Dublin from Monday to Wednesday morning. >>>>> When would be a good time for you to meet? >>>>> >>>>> Thanks, >>>>> >>>>> Juergen > _______________________________________________ IPFIX mailing list IPFIX@ietf.org https://www.ietf.org/mailman/listinfo/ipfix ted new information element assignments. What should we tell them? - the IANA procedure is almost in place? We've been telling that for too long now... - assign the next one available but that may change? Great from an interoperability point of view... - wait? we can't anymore... The last time you assign an I.E. (239, biflowDirection), that forced the IPFIX WG to change the already agreed upon I.E. assignments ... which is the reason why we had to produce http://tools.ietf.org/id/draft-aitken-ipfix-new-infos-03.txt Regards, Benoit. > Benoit, > > It was great meeting with you all at this IETF. > > I have spoken with my XML folks. We are already working on it. > It should be done by the end of the week but I can't promise. > I'll send you all something as soon as it is ready as well as send you questions > if anything comes up. > > Patience :) > > This is one of my highest priorities.... > > Thanks, > > Michelle > > > > On Wed Jul 30 17:07:16 2008, bclaise@cisco.com wrote: > >> Michelle, >> >> Good talking to you the other day. >> Have you received some feedback from your XML expert? >> Do you have something to show us while we're still at the IETF? >> >> Regards, Benoit. >> >>>> Dear all, >>>> >>>> Tuesday 11h30 is fine for Paul and myself. >>>> What about you? >>>> >>>> Regards, Benoit. >>>> >>>>> Dear Michelle, >>>>> >>>>> I will be in Dublin from Monday to Wednesday morning. >>>>> When would be a good time for you to meet? >>>>> >>>>> Thanks, >>>>> >>>>> Juergen > _______________________________________________ IPFIX mailing list IPFIX@ietf.org https://www.ietf.org/mailman/listinfo/ipfix From ipfix-bounces@ietf.org Tue Aug 26 10:35:08 2008 Return-Path: X-Original-To: ipfix-archive@optimus.ietf.org Delivered-To: ietfarch-ipfix-archive@core3.amsl.com Received: from [127.0.0.1] (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id E757C3A6A19; Tue, 26 Aug 2008 10:35:08 -0700 (PDT) X-Original-To: ipfix@core3.amsl.com Delivered-To: ipfix@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 53E813A6ADA for ; Tue, 26 Aug 2008 08:06:04 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.189 X-Spam-Level: X-Spam-Status: No, score=-6.189 tagged_above=-999 required=5 tests=[AWL=0.409, BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id TK7NPSYQtiHn for ; Tue, 26 Aug 2008 08:05:53 -0700 (PDT) Received: from EXPFE100-1.exc.icann.org (expfe100-1.exc.icann.org [64.78.22.236]) by core3.amsl.com (Postfix) with ESMTP id D12923A6B57 for ; Tue, 26 Aug 2008 08:05:53 -0700 (PDT) Received: from EXVPMBX100-1.exc.icann.org ([64.78.22.233]) by EXPFE100-1.exc.icann.org ([64.78.22.236]) with mapi; Tue, 26 Aug 2008 08:05:51 -0700 From: Michelle Cotton To: Benoit Claise , "iana-prot-param@icann.org" , "ipfix@ietf.org" Date: Tue, 26 Aug 2008 08:05:44 -0700 Thread-Topic: [IANA #144572] request for new IPFIX information elements - It's start to be really ridiculous... Thread-Index: AckHauhDxNKjryw4SeOO9oxrUDBsogAIk4R1 Message-ID: In-Reply-To: <48B3DB10.8090500@cisco.com> Accept-Language: en-US Content-Language: en X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US MIME-Version: 1.0 X-Mailman-Approved-At: Tue, 26 Aug 2008 10:35:07 -0700 Cc: "n.brownlee@auckland.ac.nz" , Juergen Quittek Subject: Re: [IPFIX] [IANA #144572] request for new IPFIX information elements - It's start to be really ridiculous... X-BeenThere: ipfix@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: IPFIX WG discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============1644031243==" Sender: ipfix-bounces@ietf.org Errors-To: ipfix-bounces@ietf.org --===============1644031243== Content-Language: en Content-Type: multipart/alternative; boundary="_000_C4D969D81DA9Bmichellecottonicannorg_" --_000_C4D969D81DA9Bmichellecottonicannorg_ Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable We have the XML registry, I'm going to send you all something to look at an= d make sure that it is sufficient. Look for it later today. Apologies, I have been on some unexpected personal travel in the last 2 wee= ks but it is now finished. Michelle On 8/26/08 3:29 AM, "Benoit Claise" wrote: Michelle, It's start to be really ridiculous... We understood that this IPFIX registry might be a little special and difficult due to - the history of NetFlow version 9, already implemented for years. - the fact that RFC5102 doesn't have a clear procedure for IANA ("MUST be published using a well-established and persistent publication medium") So, during IETF 71, we (Dan Romascanu, Nevil Brownlee, Brian Trammell, and I) came to you in person to explain everything. And again, during IETF 72, we (Juergen Quittek, Paul Aitken, Brian Trammel, and I) came to you in person to explain everything. It was supFrom ipfix-bounces@ietf.org Tue Aug 26 10:35:08 2008 Return-Path: X-Original-To: ipfix-archive@lists.ietf.org Delivered-To: ietfarch-ipfix-archive@core3.amsl.com Received: from [127.0.0.1] (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id E757C3A6A19; Tue, 26 Aug 2008 10:35:08 -0700 (PDT) X-Original-To: ipfix@core3.amsl.com Delivered-To: ipfix@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 53E813A6ADA for ; Tue, 26 Aug 2008 08:06:04 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.189 X-Spam-Level: X-Spam-Status: No, score=-6.189 tagged_above=-999 required=5 tests=[AWL=0.409, BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id TK7NPSYQtiHn for ; Tue, 26 Aug 2008 08:05:53 -0700 (PDT) Received: from EXPFE100-1.exc.icann.org (expfe100-1.exc.icann.org [64.78.22.236]) by core3.amsl.com (Postfix) with ESMTP id D12923A6B57 for ; Tue, 26 Aug 2008 08:05:53 -0700 (PDT) Received: from EXVPMBX100-1.exc.icann.org ([64.78.22.233]) by EXPFE100-1.exc.icann.org ([64.78.22.236]) with mapi; Tue, 26 Aug 2008 08:05:51 -0700 From: Michelle Cotton To: Benoit Claise , "iana-prot-param@icann.org" , "ipfix@ietf.org" Date: Tue, 26 Aug 2008 08:05:44 -0700 Thread-Topic: [IANA #144572] request for new IPFIX information elements - It's start to be really ridiculous... Thread-Index: AckHauhDxNKjryw4SeOO9oxrUDBsogAIk4R1 Message-ID: In-Reply-To: <48B3DB10.8090500@cisco.com> Accept-Language: en-US Content-Language: en X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US MIME-Version: 1.0 X-Mailman-Approved-At: Tue, 26 Aug 2008 10:35:07 -0700 Cc: "n.brownlee@auckland.ac.nz" , Juergen Quittek Subject: Re: [IPFIX] [IANA #144572] request for new IPFIX information elements - It's start to be really ridiculous... X-BeenThere: ipfix@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: IPFIX WG discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============1644031243==" Sender: ipfix-bounces@ietf.org Errors-To: ipfix-bounces@ietf.org --===============1644031243== Content-Language: en Content-Type: multipart/alternative; boundary="_000_C4D969D81DA9Bmichellecottonicannorg_" --_000_C4D969D81DA9Bmichellecottonicannorg_ Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable We have the XML registry, I'm going to send you all something to look at an= d make sure that it is sufficient. Look for it later today. Apologies, I have been on some unexpected personal travel in the last 2 wee= ks but it is now finished. Michelle On 8/26/08 3:29 AM, "Benoit Claise" wrote: Michelle, It's start to be really ridiculous... We understood that this IPFIX registry might be a little special and difficult due to - the history of NetFlow version 9, already implemented for years. - the fact that RFC5102 doesn't have a clear procedure for IANA ("MUST be published using a well-established and persistent publication medium") So, during IETF 71, we (Dan Romascanu, Nevil Brownlee, Brian Trammell, and I) came to you in person to explain everything. And again, during IETF 72, we (Juergen Quittek, Paul Aitken, Brian Trammel, and I) came to you in person to explain everything. It was suppoposed to be fixed within a week. And still no news. I'm not sure if you realize the mess in which my company is: we invented NetFlow, pushed through standardization via IPFIX, published all our information elements to the world, and now we're paying a big price for that. Because, obviously we kept implementing new Information Elements since we started this IPFIX effort at the IETF in 2001... So we have many groups within the company wanted new information element assignments. What should we tell them? - the IANA procedure is almost in place? We've been telling that for too long now... - assign the next one available but that may change? Great from an interoperability point of view... - wait? we can't anymore... The last time you assign an I.E. (239, biflowDirection), that forced the IPFIX WG to change the already agreed upon I.E. assignments ... which is the reason why we had to produce http://tools.ietf.org/id/draft-aitken-ipfix-new-infos-03.txt Regards, Benoit. > Benoit, > > It was great meeting with you all at this IETF. > > I have spoken with my XML folks. We are already working on it. > It should be done by the end of the week but I can't promise. > I'll send you all something as soon as it is ready as well as send you qu= estions > if anything comes up. > > Patience :) > > This is one of my highest priorities.... > > Thanks, > > Michelle > > > > On Wed Jul 30 17:07:16 2008, bclaise@cisco.com wrote: > >> Michelle, >> >> Good talking to you the other day. >> Have you received some feedback from your XML expert? >> Do you have something to show us while we're still at the IETF? >> >> Regards, Benoit. >> >>>> Dear all, >>>> >>>> Tuesday 11h30 is fine for Paul and myself. >>>> What about you? >>>> >>>> Regards, Benoit. >>>> >>>>> Dear Michelle, >>>>> >>>>> I will be in Dublin from Monday to Wednesday morning. >>>>> When would be a good time for you to meet? >>>>> >>>>> Thanks, >>>>> >>>>> Juergen > --_000_C4D969D81DA9Bmichellecottonicannorg_ Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Re: [IANA #144572] request for new IPFIX information elements - It's= start to be really ridiculous... We have the XML registry, I’m going to send yo= u all something to look at and make sure that it is sufficient.
Look for it later today.

Apologies, I have been on some unexpected personal travel in the last 2 wee= ks but it is now finished.

Michelle


On 8/26/08 3:29 AM, "Benoit Claise" <bclaise@cisco.com> wro= te:

Michelle,

It's start to be really ridiculous...

We understood that this IPFIX registry might be a little special and
difficult due to
    - the history of NetFlow version 9, already impleme= nted for years.
    - the fact that RFC5102 doesn't have a clear proced= ure for IANA
("MUST be published using a well-established and persistent publicatio= n
medium")

So, during IETF 71, we (Dan Romascanu, Nevil Brownlee, Brian Trammell,
and I) came to you in person to explain everything.
And again, during IETF 72, we (Juergen Quittek, Paul Aitken, Brian
Trammel, and I) came to you in person to explain everything.
It was supposed to be fixed within a week. And still no news.

I'm not sure if you realize the mess in which my company is: we invented NetFlow, pushed through standardization via IPFIX, published all our
information elements to the world, and now we're paying a big price for
that. Because, obviously we kept implementing new Information Elements
since we started this IPFIX effort at the IETF in 2001... So we have
many groups within the company wanted new information element
assignments. Wsed to be fixed within a week. And still no news. I'm not sure if you realize the mess in which my company is: we invented NetFlow, pushed through standardization via IPFIX, published all our information elements to the world, and now we're paying a big price for that. Because, obviously we kept implementing new Information Elements since we started this IPFIX effort at the IETF in 2001... So we have many groups within the company wanted new information element assignments. What should we tell them? - the IANA procedure is almost in place? We've been telling that for too long now... - assign the next one available but that may change? Great from an interoperability point of view... - wait? we can't anymore... The last time you assign an I.E. (239, biflowDirection), that forced the IPFIX WG to change the already agreed upon I.E. assignments ... which is the reason why we had to produce http://tools.ietf.org/id/draft-aitken-ipfix-new-infos-03.txt Regards, Benoit. > Benoit, > > It was great meeting with you all at this IETF. > > I have spoken with my XML folks. We are already working on it. > It should be done by the end of the week but I can't promise. > I'll send you all something as soon as it is ready as well as send you qu= estions > if anything comes up. > > Patience :) > > This is one of my highest priorities.... > > Thanks, > > Michelle > > > > On Wed Jul 30 17:07:16 2008, bclaise@cisco.com wrote: > >> Michelle, >> >> Good talking to you the other day. >> Have you received some feedback from your XML expert? >> Do you have something to show us while we're still at the IETF? >> >> Regards, Benoit. >> >>>> Dear all, >>>> >>>> Tuesday 11h30 is fine for Paul and myself. >>>> What about you? >>>> >>>> Regards, Benoit. >>>> >>>>> Dear Michelle, >>>>> >>>>> I will be in Dublin from Monday to Wednesday morning. >>>>> When would be a good time for you to meet? >>>>> >>>>> Thanks, >>>>> >>>>> Juergen > --_000_C4D969D81DA9Bmichellecottonicannorg_ Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Re: [IANA #144572] request for new IPFIX information elements - It's= start to be really ridiculous... We have the XML registry, I’m going to send yo= u all something to look at and make sure that it is sufficient.
Look for it later today.

Apologies, I have been on some unexpected personal travel in the last 2 wee= ks but it is now finished.

Michelle


On 8/26/08 3:29 AM, "Benoit Claise" <bclaise@cisco.com> wro= te:

Michelle,

It's start to be really ridiculous...

We understood that this IPFIX registry might be a little special and
difficult due to
    - the history of NetFlow version 9, already impleme= nted for years.
    - the fact that RFC5102 doesn't have a clear proced= ure for IANA
("MUST be published using a well-established and persistent publicatio= n
medium")

So, during IETF 71, we (Dan Romascanu, Nevil Brownlee, Brian Trammell,
and I) came to you in person to explain everything.
And again, during IETF 72, we (Juergen Quittek, Paul Aitken, Brian
Trammel, and I) came to you in person to explain everything.
It was supposed to be fixed within a week. And still no news.

I'm not sure if you realize the mess in which my company is: we invented NetFlow, pushed through standardization via IPFIX, published all our
information elements to the world, and now we're paying a big price for
that. Because, obviously we kept implementing new Information Elements
since we started this IPFIX effort at the IETF in 2001... So we have
many groups within the company wanted new information element
assignments. Whahat should we tell them?
 - the IANA procedure is almost in place? We've been telling that for<= BR> too long now...
 - assign the next one available but that may change? Great from an interoperability point of view...
 - wait? we can't anymore... The last time you assign an I.E. (239, biflowDirection), that forced the IPFIX WG to change the already agreed
upon I.E. assignments ... which is the reason why we had to produce
ht= tp://tools.ietf.org/id/draft-aitken-ipfix-new-infos-03.txt

Regards, Benoit.

> Benoit,
>
> It was great meeting with you all at this IETF.
>
> I have spoken with my XML folks.  We are already working on it. > It should be done by the end of the week but I can't promise.
> I'll send you all something as soon as it is ready as well as send you= questions
> if anything comes up.
>
> Patience :)
>
> This is one of my highest priorities....
>
> Thanks,
>
> Michelle
>
>
>
> On Wed Jul 30 17:07:16 2008, bclaise@cisco.com wrote:
>
>> Michelle,
>>
>> Good talking to you the other day.
>> Have you received some feedback from your XML expert?
>> Do you have something to show us while we're still at the IETF? >>
>> Regards, Benoit.
>>
>>>> Dear all,
>>>>
>>>> Tuesday 11h30 is fine for Paul and myself.
>>>> What about you?
>>>>
>>>> Regards, Benoit.
>>>>
>>>>> Dear Michelle,
>>>>>
>>>>> I will be in Dublin from Monday to Wednesday morning.<= BR> >>>>> When would be a good time for you to meet?
>>>>>
>>>>> Thanks,
>>>>>
>>>>>     Juergen
>


--_000_C4D969D81DA9Bmichellecottonicannorg_-- --===============1644031243== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ IPFIX mailing list IPFIX@ietf.org https://www.ietf.org/mailman/listinfo/ipfix --===============1644031243==-- t should we tell them?
 - the IANA procedure is almost in place? We've been telling that for<= BR> too long now...
 - assign the next one available but that may change? Great from an interoperability point of view...
 - wait? we can't anymore... The last time you assign an I.E. (239, biflowDirection), that forced the IPFIX WG to change the already agreed
upon I.E. assignments ... which is the reason why we had to produce
ht= tp://tools.ietf.org/id/draft-aitken-ipfix-new-infos-03.txt

Regards, Benoit.

> Benoit,
>
> It was great meeting with you all at this IETF.
>
> I have spoken with my XML folks.  We are already working on it. > It should be done by the end of the week but I can't promise.
> I'll send you all something as soon as it is ready as well as send you= questions
> if anything comes up.
>
> Patience :)
>
> This is one of my highest priorities....
>
> Thanks,
>
> Michelle
>
>
>
> On Wed Jul 30 17:07:16 2008, bclaise@cisco.com wrote:
>
>> Michelle,
>>
>> Good talking to you the other day.
>> Have you received some feedback from your XML expert?
>> Do you have something to show us while we're still at the IETF? >>
>> Regards, Benoit.
>>
>>>> Dear all,
>>>>
>>>> Tuesday 11h30 is fine for Paul and myself.
>>>> What about you?
>>>>
>>>> Regards, Benoit.
>>>>
>>>>> Dear Michelle,
>>>>>
>>>>> I will be in Dublin from Monday to Wednesday morning.<= BR> >>>>> When would be a good time for you to meet?
>>>>>
>>>>> Thanks,
>>>>>
>>>>>     Juergen
>


--_000_C4D969D81DA9Bmichellecottonicannorg_-- --===============1644031243== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ IPFIX mailing list IPFIX@ietf.org https://www.ietf.org/mailman/listinfo/ipfix --===============1644031243==-- From ipfix-bounces@ietf.org Wed Aug 27 01:57:30 2008 Return-Path: X-Original-To: ipfix-archive@lists.ietf.org Delivered-To: ietfarch-ipfix-archive@core3.amsl.com Received: from [127.0.0.1] (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id C34583A6867; Wed, 27 Aug 2008 01:57:30 -0700 (PDT) X-Original-To: ipfix@core3.amsl.com Delivered-To: ipfix@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id BAC9E3A6842 for ; Wed, 27 Aug 2008 01:57:28 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: 0 X-Spam-Level: X-Spam-Status: No, score=x tagged_above=-999 required=5 tests=[] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id HruTjljDulgA for ; Wed, 27 Aug 2008 01:57:28 -0700 (PDT) Received: from av-tac-bru.cisco.com (odd-brew.cisco.com [144.254.15.119]) by core3.amsl.com (Postfix) with ESMTP id 12AAB3A6B02 for ; Wed, 27 Aug 2008 01:57:26 -0700 (PDT) X-TACSUNS: Virus Scanned Received: from strange-brew.cisco.com (localhost [127.0.0.1]) by av-tac-bru.cisco.com (8.11.7p3+Sun/8.11.7) with ESMTP id m7R8vSQ18461 for ; Wed, 27 Aug 2008 10:57:28 +0200 (CEST) Received: from [10.61.96.112] (dhcp-10-61-96-112.cisco.com [10.61.96.112]) by strange-brew.cisco.com (8.11.7p3+Sun/8.11.7) with ESMTP id m7R8vLA08621 for ; Wed, 27 Aug 2008 10:57:21 +0200 (CEST) Message-ID: <48B516F1.3020105@cisco.com> Date: Wed, 27 Aug 2008 10:57:21 +0200 From: Benoit Claise User-Agent: Thunderbird 1.5.0.12 (Windows/20070509) MIME-Version: 1.0 To: "ipfix@ietf.org" References: <48B3D434.9030309@cisco.com> In-Reply-To: <48B3D434.9030309@cisco.com> Subject: [IPFIX] Feedback on the IPFIX file last-call X-BeenThere: ipfix@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: IPFIX WG discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============1415078296==" Sender: ipfix-bounces@ietf.org Errors-To: ipfix-bounces@ietf.org This is a multi-part message in MIME format. --===============1415078296== Content-Type: multipart/alternative; boundary="------------050305070204080904090002" This is a multi-part message in MIME format. --------------050305070204080904090002 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Dear all, Here is my feedback. In preparation for this review, I read first the email from Paul and Gerhard. I will try not to repeat their feedback. Regards, Benoit. > > > > ------------------------------------------------------------------------ > > > > > IPFIX Working Group B. Trammell > Internet-Draft E. Boschi > Intended status: Standards Track Hitachi Europe > Expires: January 15, 2009 L. Mark > Fraunhofer IFAM > T. Zseby > Fraunhofer FOKUS > A. Wagner > ETH Zurich > July 14, 2008 > > > An IPFFrom ipfix-bounces@ietf.org Wed Aug 27 01:57:30 2008 Return-Path: X-Original-To: ipfix-archive@optimus.ietf.org Delivered-To: ietfarch-ipfix-archive@core3.amsl.com Received: from [127.0.0.1] (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id C34583A6867; Wed, 27 Aug 2008 01:57:30 -0700 (PDT) X-Original-To: ipfix@core3.amsl.com Delivered-To: ipfix@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id BAC9E3A6842 for ; Wed, 27 Aug 2008 01:57:28 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: 0 X-Spam-Level: X-Spam-Status: No, score=x tagged_above=-999 required=5 tests=[] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id HruTjljDulgA for ; Wed, 27 Aug 2008 01:57:28 -0700 (PDT) Received: from av-tac-bru.cisco.com (odd-brew.cisco.com [144.254.15.119]) by core3.amsl.com (Postfix) with ESMTP id 12AAB3A6B02 for ; Wed, 27 Aug 2008 01:57:26 -0700 (PDT) X-TACSUNS: Virus Scanned Received: from strange-brew.cisco.com (localhost [127.0.0.1]) by av-tac-bru.cisco.com (8.11.7p3+Sun/8.11.7) with ESMTP id m7R8vSQ18461 for ; Wed, 27 Aug 2008 10:57:28 +0200 (CEST) Received: from [10.61.96.112] (dhcp-10-61-96-112.cisco.com [10.61.96.112]) by strange-brew.cisco.com (8.11.7p3+Sun/8.11.7) with ESMTP id m7R8vLA08621 for ; Wed, 27 Aug 2008 10:57:21 +0200 (CEST) Message-ID: <48B516F1.3020105@cisco.com> Date: Wed, 27 Aug 2008 10:57:21 +0200 From: Benoit Claise User-Agent: Thunderbird 1.5.0.12 (Windows/20070509) MIME-Version: 1.0 To: "ipfix@ietf.org" References: <48B3D434.9030309@cisco.com> In-Reply-To: <48B3D434.9030309@cisco.com> Subject: [IPFIX] Feedback on the IPFIX file last-call X-BeenThere: ipfix@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: IPFIX WG discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============1415078296==" Sender: ipfix-bounces@ietf.org Errors-To: ipfix-bounces@ietf.org This is a multi-part message in MIME format. --===============1415078296== Content-Type: multipart/alternative; boundary="------------050305070204080904090002" This is a multi-part message in MIME format. --------------050305070204080904090002 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Dear all, Here is my feedback. In preparation for this review, I read first the email from Paul and Gerhard. I will try not to repeat their feedback. Regards, Benoit. > > > > ------------------------------------------------------------------------ > > > > > IPFIX Working Group B. Trammell > Internet-Draft E. Boschi > Intended status: Standards Track Hitachi Europe > Expires: January 15, 2009 L. Mark > Fraunhofer IFAM > T. Zseby > Fraunhofer FOKUS > A. Wagner > ETH Zurich > July 14, 2008 > > > An IIX-Based File Format > Add "Specifications" ? > draft-ietf-ipfix-file-02.txt > > Status of this Memo > > By submitting this Internet-Draft, each author represents that any > applicable patent or other IPR claims of which he or she is aware > have been or will be disclosed, and any of which he or she becomes > aware will be disclosed, in accordance with Section 6 of BCP 79. > > Internet-Drafts are working documents of the Internet Engineering > Task Force (IETF), its areas, and its working groups. Note that > other groups may also distribute working documents as Internet- > Drafts. > > Internet-Drafts are draft documents valid for a maximum of six months > and may be updated, replaced, or obsoleted by other documents at any > time. It is inappropriate to use Internet-Drafts as reference > material or to cite them other than as "work in progress." > > The list of current Internet-Drafts can be accessed at > http://www.ietf.org/ietf/1id-abstracts.txt. > > The list of Internet-Draft Shadow Directories can be accessed at > http://www.ietf.org/shadow.html. > > This Internet-Draft will expire on January 15, 2009. > > Abstract > > This document describes a file format for the storage of flow data > based upon the IPFIX Message format. It proposes a set of > requirements for flat-file, binary flow data file formats, then > applies the IPFIX message format to these requirements to build a new > Message > file format. This IPFIX-based file format is designed to facilitate > interoperability and reusability among a wide variety of flow > > > > Trammell, et al. Expires January 15, 2009 [Page 1] > > Internet-Draft IPFIX Files July 2008 > > > storage, processing, and analysis tools. > > > Table of Contents > > 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 4 > 1.1. IPFIX Documents Overview . . . . . . . . . . . . . . . . . 4 > 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 5 > 3. Design Overview . . . . . . . . . . . . . . . . . . . . . . . 6 > 4. Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . 7 > 5. Requirements . . . . . . . . . . . . . . . . . . . . . . . . . 9 > 5.1. Record Format Flexibility . . . . . . . . . . . . . . . . 10 > 5.2. Self Description . . . . . . . . . . . . . . . . . . . . . 10 > 5.3. Data Compression . . . . . . . . . . . . . . . . . . . . . 11 > 5.4. Indexing and Searching . . . . . . . . . . . . . . . . . . 11 > 5.5. Data Integrity . . . . . . . . . . . . . . . . . . . . . . 12 > 5.6. Creator Authentication and Confidentiality . . . . . . . . 12 > 5.7. Anonymization and Obfuscation . . . . . . . . . . . . . . 13 > 5.8. Session Auditability and Replayability . . . . . . . . . . 13 > 5.9. Performance Characteristics . . . . . . . . . . . . . . . 14 > 6. Applicability . . . . . . . . . . . . . . . . . . . . . . . . 14 > 6.1. Storage of IPFIX-collected Flow Data . . . . . . . . . . . 14 > 6.2. Storage of NetFlow V9-collected Flow Data . . . . . . . . 15 > 6.3. Testing IPFIX Collecting Processes . . . . . . . . . . . . 15 > 6.4. IPFIX Device Diagnostics . . . . . . . . . . . . . . . . . 15 > 7. Detailed Description . . . . . . . . . . . . . . . . . . . . . 16 > Specifications? > 7.1. File Reader Specification . . . . . . . . . . . . . . . . 16 > 7.2. File Writer Specification . . . . . . . . . . . . . . . . 17 > 7.3. Specific File Writer Use Cases . . . . . . . . . . . . . . 18 > 7.3.1. Collocating a File Writer with a Collecting Process . 18 > 7.3.2. Collocating a File Writer with a Metering Process . . 19 > 7.3.3. Using IPFIX Files for Archival Storage . . . . . . . . 19 > 7.3.4. Using IPFIX Files as Documents . . . . . . . . . . . . 19 > 7.3.5. Using IPFIX Files for Testing . . . . . . . . . . . . 20 > 7.3.PFIX-Based File Format > Add "Specifications" ? > draft-ietf-ipfix-file-02.txt > > Status of this Memo > > By submitting this Internet-Draft, each author represents that any > applicable patent or other IPR claims of which he or she is aware > have been or will be disclosed, and any of which he or she becomes > aware will be disclosed, in accordance with Section 6 of BCP 79. > > Internet-Drafts are working documents of the Internet Engineering > Task Force (IETF), its areas, and its working groups. Note that > other groups may also distribute working documents as Internet- > Drafts. > > Internet-Drafts are draft documents valid for a maximum of six months > and may be updated, replaced, or obsoleted by other documents at any > time. It is inappropriate to use Internet-Drafts as reference > material or to cite them other than as "work in progress." > > The list of current Internet-Drafts can be accessed at > http://www.ietf.org/ietf/1id-abstracts.txt. > > The list of Internet-Draft Shadow Directories can be accessed at > http://www.ietf.org/shadow.html. > > This Internet-Draft will expire on January 15, 2009. > > Abstract > > This document describes a file format for the storage of flow data > based upon the IPFIX Message format. It proposes a set of > requirements for flat-file, binary flow data file formats, then > applies the IPFIX message format to these requirements to build a new > Message > file format. This IPFIX-based file format is designed to facilitate > interoperability and reusability among a wide variety of flow > > > > Trammell, et al. Expires January 15, 2009 [Page 1] > > Internet-Draft IPFIX Files July 2008 > > > storage, processing, and analysis tools. > > > Table of Contents > > 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 4 > 1.1. IPFIX Documents Overview . . . . . . . . . . . . . . . . . 4 > 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 5 > 3. Design Overview . . . . . . . . . . . . . . . . . . . . . . . 6 > 4. Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . 7 > 5. Requirements . . . . . . . . . . . . . . . . . . . . . . . . . 9 > 5.1. Record Format Flexibility . . . . . . . . . . . . . . . . 10 > 5.2. Self Description . . . . . . . . . . . . . . . . . . . . . 10 > 5.3. Data Compression . . . . . . . . . . . . . . . . . . . . . 11 > 5.4. Indexing and Searching . . . . . . . . . . . . . . . . . . 11 > 5.5. Data Integrity . . . . . . . . . . . . . . . . . . . . . . 12 > 5.6. Creator Authentication and Confidentiality . . . . . . . . 12 > 5.7. Anonymization and Obfuscation . . . . . . . . . . . . . . 13 > 5.8. Session Auditability and Replayability . . . . . . . . . . 13 > 5.9. Performance Characteristics . . . . . . . . . . . . . . . 14 > 6. Applicability . . . . . . . . . . . . . . . . . . . . . . . . 14 > 6.1. Storage of IPFIX-collected Flow Data . . . . . . . . . . . 14 > 6.2. Storage of NetFlow V9-collected Flow Data . . . . . . . . 15 > 6.3. Testing IPFIX Collecting Processes . . . . . . . . . . . . 15 > 6.4. IPFIX Device Diagnostics . . . . . . . . . . . . . . . . . 15 > 7. Detailed Description . . . . . . . . . . . . . . . . . . . . . 16 > Specifications? > 7.1. File Reader Specification . . . . . . . . . . . . . . . . 16 > 7.2. File Writer Specification . . . . . . . . . . . . . . . . 17 > 7.3. Specific File Writer Use Cases . . . . . . . . . . . . . . 18 > 7.3.1. Collocating a File Writer with a Collecting Process . 18 > 7.3.2. Collocating a File Writer with a Metering Process . . 19 > 7.3.3. Using IPFIX Files for Archival Storage . . . . . . . . 19 > 7.3.4. Using IPFIX Files as Documents . . . . . . . . . . . . 19 > 7.3.5. Using IPFIX Files for Testing . . . . . . . . . . . . 20 > 7.6. Writing IPFIX Files for Device Diagnostics . . . . . . 21 > 8. File Format Metadata Specification . . . . . . . . . . . . . . 21 > > 8.1. Recommended Options Templates for IPFIX Files . . . . . . 21 > 8.1.1. Message Checksum Options Template . . . . . . . . . . 21 > 8.1.2. File Time Window Options Template . . . . . . . . . . 22 > 8.1.3. Export Session Details Options Template . . . . . . . 22 > 8.1.4. Message Details Options Template . . . . . . . . . . . 24 > 8.2. Recommended Information Elements for IPFIX Files . . . . . 26 > 8.2.1. collectionTimeMilliseconds . . . . . . . . . . . . . . 26 > 8.2.2. exportSctpStreamId . . . . . . . . . . . . . . . . . . 27 > 8.2.3. maxExportSeconds . . . . . . . . . . . . . . . . . . . 27 > 8.2.4. maxFlowEndSeconds . . . . . . . . . . . . . . . . . . 28 > 8.2.5. messageMD5Checksum . . . . . . . . . . . . . . . . . . 28 > 8.2.6. messageScope . . . . . . . . . . . . . . . . . . . . . 28 > > > > Trammell, et al. Expires January 15, 2009 [Page 2] > > Internet-Draft IPFIX Files July 2008 > > > 8.2.7. minExportSeconds . . . . . . . . . . . . . . . . . . . 29 > 8.2.8. minFlowStartSeconds . . . . . . . . . . . . . . . . . 29 > 8.2.9. opaqueOctets . . . . . . . . . . . . . . . . . . . . . 30 > 8.2.10. sessionScope . . . . . . . . . . . . . . . . . . . . . 30 > 9. Recommended Error Resilience Strategies . . . . . . . . . . . 30 > 9.1. Compression Error Resilience . . . . . . . . . . . . . . . 31 > 9.2. Encryption Error Resilience . . . . . . . . . . . . . . . 32 > 10. Recommended File Integration Strategies . . . . . . . . . . . 32 > 10.1. Encapsulation of Non-IPFIX Data in IPFIX Files . . . . . . 32 > 10.2. Encapsulation of IPFIX Files within Other File Formats . . 33 > 11. Security Considerations . . . . . . . . . . . . . . . . . . . 33 > 12. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 34 > 13. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 35 > 14. References . . . . . . . . . . . . . . . . . . . . . . . . . . 35 > 14.1. Normative References . . . . . . . . . . . . . . . . . . . 35 > 14.2. Informative References . . . . . . . . . . . . . . . . . . 35 > Appendix A. Example IPFIX File . . . . . . . . . . . . . . . . . 36 > A.1. Example Options Templates . . . . . . . . . . . . . . . . 38 > A.2. Example Supplemental Options Data . . . . . . . . . . . . 40 > A.3. Example Message Checksum . . . . . . . . . . . . . . . . . 42 > A.4. File Example Data Set . . . . . . . . . . . . . . . . . . 43 > A.5. Complete File Example . . . . . . . . . . . . . . . . . . 43 > Appendix B. Applicability of IPFIX Files to NetFlow V9 flow > storage . . . . . . . . . . . . . . . . . . . . . . . 45 > B.1. Comparing NetFlow V9 to IPFIX . . . . . . . . . . . . . . 45 > B.1.1. Message Header Format . . . . . . . . . . . . . . . . 45 > B.1.2. Set Header Format . . . . . . . . . . . . . . . . . . 46 > B.1.3. Template Format . . . . . . . . . . . . . . . . . . . 47 > B.1.4. Information Model . . . . . . . . . . . . . . . . . . 47 > B.1.5. Template Management . . . . . . . . . . . . . . . . . 47 > B.1.6. Transport . . . . . . . . . . . . . . . . . . . . . . 47 > B.2. A Method for Transforming NetFlow V9 messages to IPFIX . . 48 > B.3. NetFlow V9 Transformation Example . . . . . . . . . . . . 49 > Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 51 > Intellectual Property and Copyright Statements . . . . . . . . . . 53 > > > > > > > > > > > > > > > > > Trammell, et al. Expires January 15, 2009 [Page 3] > > Internet-Draft IPFIX Files July 2008 > > > 1. Introduction > > This document proposes a file format based upon IPFIX, des3.6. Writing IPFIX Files for Device Diagnostics . . . . . . 21 > 8. File Format Metadata Specification . . . . . . . . . . . . . . 21 > > 8.1. Recommended Options Templates for IPFIX Files . . . . . . 21 > 8.1.1. Message Checksum Options Template . . . . . . . . . . 21 > 8.1.2. File Time Window Options Template . . . . . . . . . . 22 > 8.1.3. Export Session Details Options Template . . . . . . . 22 > 8.1.4. Message Details Options Template . . . . . . . . . . . 24 > 8.2. Recommended Information Elements for IPFIX Files . . . . . 26 > 8.2.1. collectionTimeMilliseconds . . . . . . . . . . . . . . 26 > 8.2.2. exportSctpStreamId . . . . . . . . . . . . . . . . . . 27 > 8.2.3. maxExportSeconds . . . . . . . . . . . . . . . . . . . 27 > 8.2.4. maxFlowEndSeconds . . . . . . . . . . . . . . . . . . 28 > 8.2.5. messageMD5Checksum . . . . . . . . . . . . . . . . . . 28 > 8.2.6. messageScope . . . . . . . . . . . . . . . . . . . . . 28 > > > > Trammell, et al. Expires January 15, 2009 [Page 2] > > Internet-Draft IPFIX Files July 2008 > > > 8.2.7. minExportSeconds . . . . . . . . . . . . . . . . . . . 29 > 8.2.8. minFlowStartSeconds . . . . . . . . . . . . . . . . . 29 > 8.2.9. opaqueOctets . . . . . . . . . . . . . . . . . . . . . 30 > 8.2.10. sessionScope . . . . . . . . . . . . . . . . . . . . . 30 > 9. Recommended Error Resilience Strategies . . . . . . . . . . . 30 > 9.1. Compression Error Resilience . . . . . . . . . . . . . . . 31 > 9.2. Encryption Error Resilience . . . . . . . . . . . . . . . 32 > 10. Recommended File Integration Strategies . . . . . . . . . . . 32 > 10.1. Encapsulation of Non-IPFIX Data in IPFIX Files . . . . . . 32 > 10.2. Encapsulation of IPFIX Files within Other File Formats . . 33 > 11. Security Considerations . . . . . . . . . . . . . . . . . . . 33 > 12. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 34 > 13. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 35 > 14. References . . . . . . . . . . . . . . . . . . . . . . . . . . 35 > 14.1. Normative References . . . . . . . . . . . . . . . . . . . 35 > 14.2. Informative References . . . . . . . . . . . . . . . . . . 35 > Appendix A. Example IPFIX File . . . . . . . . . . . . . . . . . 36 > A.1. Example Options Templates . . . . . . . . . . . . . . . . 38 > A.2. Example Supplemental Options Data . . . . . . . . . . . . 40 > A.3. Example Message Checksum . . . . . . . . . . . . . . . . . 42 > A.4. File Example Data Set . . . . . . . . . . . . . . . . . . 43 > A.5. Complete File Example . . . . . . . . . . . . . . . . . . 43 > Appendix B. Applicability of IPFIX Files to NetFlow V9 flow > storage . . . . . . . . . . . . . . . . . . . . . . . 45 > B.1. Comparing NetFlow V9 to IPFIX . . . . . . . . . . . . . . 45 > B.1.1. Message Header Format . . . . . . . . . . . . . . . . 45 > B.1.2. Set Header Format . . . . . . . . . . . . . . . . . . 46 > B.1.3. Template Format . . . . . . . . . . . . . . . . . . . 47 > B.1.4. Information Model . . . . . . . . . . . . . . . . . . 47 > B.1.5. Template Management . . . . . . . . . . . . . . . . . 47 > B.1.6. Transport . . . . . . . . . . . . . . . . . . . . . . 47 > B.2. A Method for Transforming NetFlow V9 messages to IPFIX . . 48 > B.3. NetFlow V9 Transformation Example . . . . . . . . . . . . 49 > Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 51 > Intellectual Property and Copyright Statements . . . . . . . . . . 53 > > > > > > > > > > > > > > > > > Trammell, et al. Expires January 15, 2009 [Page 3] > > Internet-Draft IPFIX Files July 2008 > > > 1. Introduction > > This document proposes a file format based upon IPFIX, digned to > facilitiate interoperability and reusability among a wide variety of > flow storage, processing, and analysis tools. It begins with an > overview of the IPFIX File format, and a quick summary of how IPFIX > "IPFIX-based file format" in the abstract. " IPFIX File format" here. > Files work in Section 3. It then explores the motivation for > proposing a standardized flow file format and using IPFIX as the > basis for this new file format in Section 4. It outlines a set of > requirements for standardized flow storage in Section 5, and explores > the applicability of such a format to various specific application > areas in Section 6. > > The file format specification follows in Section 7, including > And new simply "file format". Maybe I should wait for the definition ;-) > specifications of readers and writers of these files, and additional > specifications that apply in specific use cases. This format makes > use of the IPFIX Options mechanism for additional file metadata, in > order to avoid requiring any protocol or message format extensions, > and to minimize the effort required to adapt IPFIX implementations to > use the file format; a detailed definition of the Options Templates > used for storage metedata appears in Section 8. > > Section 9 and Section 10 provide specific recommendations for error > resilience during long-term storage and integration of IPFIX File > data with other formats. Appendix A contains a detailed example > IPFIX File. > > 1.1. IPFIX Documents Overview > > "Specification of the IPFIX Protocol for the Exchange of IP Traffic > Flow Information" [RFC5101] and its associated documents define the > IPFIX Protocol, which provides network engineers and administrators > with access to IP traffic flow information. > > "Architecture for IP Flow Information Export" [I-D.ietf-ipfix-arch] > defines the architecture for the export of measured IP flow > information out of an IPFIX Exporting Process to an IPFIX Collecting > Process, and the basic terminology used to describe the elements of > this architecture, per the requirements defined in "Requirements for > IP Flow Information Export" [RFC3917]. [RFC5101] then covers the > details of the method for transporting IPFIX Data Records and > Templates via a congestion-aware transport protocol from an IPFIX > Exporting Process to an IPFIX Collecting Process. > > "Information Model for IP Flow Information Export" [RFC5102] > describes the Information Elements used by IPFIX, including details > on Information Element naming, numbering, and data type encoding. > ? > Finally, "IPFIX Applicability" [I-D.ietf-ipfix-as] describes the > Why start with "Finally" while the next section starts with "in addition"? > > > Trammell, et al. Expires January 15, 2009 [Page 4] > > Internet-Draft IPFIX Files July 2008 > > > various applications of the IPFIX protocol and their use of > information exported via IPFIX, and relates the IPFIX architecture to > other measurement architectures and frameworks. > > In addition, "Exporting Type Information for IPFIX Information > Elements" [I-D.ietf-ipfix-exporting-type] specifies a method for > encoding Information Model properties within an IPFIX Message stream. > > This document references [RFC5101] and the architecture document for > terminology, defines IPFIX File Writer and IPFIX File Reader in terms > of the IPFIX Exporting Processes and IPFIX Collecting Process > definitions from [RFC5101], and extends the IPFIX Information Model > defined in [RFC5102] to provide new Information Elements for IPFIX > File metadata. I've seen several instances of "metadata". This is a confusing generic term to me. Maybe it's only me? > It uses the method described in "Exporting Type > Information for IPFIX Information Elements" > [I-D.ietf-ipfix-exporting-type] document to support the self- > descresigned to > facilitiate interoperability and reusability among a wide variety of > flow storage, processing, and analysis tools. It begins with an > overview of the IPFIX File format, and a quick summary of how IPFIX > "IPFIX-based file format" in the abstract. " IPFIX File format" here. > Files work in Section 3. It then explores the motivation for > proposing a standardized flow file format and using IPFIX as the > basis for this new file format in Section 4. It outlines a set of > requirements for standardized flow storage in Section 5, and explores > the applicability of such a format to various specific application > areas in Section 6. > > The file format specification follows in Section 7, including > And new simply "file format". Maybe I should wait for the definition ;-) > specifications of readers and writers of these files, and additional > specifications that apply in specific use cases. This format makes > use of the IPFIX Options mechanism for additional file metadata, in > order to avoid requiring any protocol or message format extensions, > and to minimize the effort required to adapt IPFIX implementations to > use the file format; a detailed definition of the Options Templates > used for storage metedata appears in Section 8. > > Section 9 and Section 10 provide specific recommendations for error > resilience during long-term storage and integration of IPFIX File > data with other formats. Appendix A contains a detailed example > IPFIX File. > > 1.1. IPFIX Documents Overview > > "Specification of the IPFIX Protocol for the Exchange of IP Traffic > Flow Information" [RFC5101] and its associated documents define the > IPFIX Protocol, which provides network engineers and administrators > with access to IP traffic flow information. > > "Architecture for IP Flow Information Export" [I-D.ietf-ipfix-arch] > defines the architecture for the export of measured IP flow > information out of an IPFIX Exporting Process to an IPFIX Collecting > Process, and the basic terminology used to describe the elements of > this architecture, per the requirements defined in "Requirements for > IP Flow Information Export" [RFC3917]. [RFC5101] then covers the > details of the method for transporting IPFIX Data Records and > Templates via a congestion-aware transport protocol from an IPFIX > Exporting Process to an IPFIX Collecting Process. > > "Information Model for IP Flow Information Export" [RFC5102] > describes the Information Elements used by IPFIX, including details > on Information Element naming, numbering, and data type encoding. > ? > Finally, "IPFIX Applicability" [I-D.ietf-ipfix-as] describes the > Why start with "Finally" while the next section starts with "in addition"? > > > Trammell, et al. Expires January 15, 2009 [Page 4] > > Internet-Draft IPFIX Files July 2008 > > > various applications of the IPFIX protocol and their use of > information exported via IPFIX, and relates the IPFIX architecture to > other measurement architectures and frameworks. > > In addition, "Exporting Type Information for IPFIX Information > Elements" [I-D.ietf-ipfix-exporting-type] specifies a method for > encoding Information Model properties within an IPFIX Message stream. > > This document references [RFC5101] and the architecture document for > terminology, defines IPFIX File Writer and IPFIX File Reader in terms > of the IPFIX Exporting Processes and IPFIX Collecting Process > definitions from [RFC5101], and extends the IPFIX Information Model > defined in [RFC5102] to provide new Information Elements for IPFIX > File metadata. I've seen several instances of "metadata". This is a confusing generic term to me. Maybe it's only me? > It uses the method described in "Exporting Type > Information for IPFIX Information Elements" > [I-D.ietf-ipfix-exporting-type] document to support the self- > desiption of IPFIX Files containing enterprise-specific Information > Elements. > Minor point. The last paragraphs contains interesting information. Maybe they're not at the right place in the section "IPFIX Documents Overview" > > 2. Terminology > > Terms used in this document that are defined in the Terminology > section of [RFC5101] are to be interpreted as defined there. > > IPFIX File: An IPFIX File is a serialized stream of IPFIX Messages > stored on a filesystem. Any IPFIX Message stream that would be > considered valid when transported one or more of the specified > IPFIX transports (SCTP, TCP, or UDP) as defined in [RFC5101] is > considered an IPFIX File for purposes of this document; however, > this document extends that definition with recommendations on the > construction of IPFIX Files that meet the requirements identified > herein. > Your first sentence says: IPFIX file = stream of IPFIX Messages _stored on a filesystem_ Your second sentence says: IPFIX file = IPFIX Message stream There is a discrepancy. > IPFIX File Reader: An IPFIX File Reader is a Process which reads > IPFIX Files from a filesystem, and is analogous to an IPFIX > Collecting Process. An IPFIX File Reader MUST behave as an IPFIX > Collecting Process as outlined in [RFC5101], except as modified by > this document. > Not sure if this is adequate to include sentences with MUST with a definition. What about? IPFIX File Reader: An IPFIX File Reader is a Process which reads IPFIX Files from a filesystem. An IPFIX File Reader is analogous to an IPFIX Collecting Process. > IPFIX File Writer: An IPFIX File Writer is a process which writes > IPFIX Files to a filesystem, and is analogous to an IPFIX > Exporting Process. An IPFIX File Writer MUST behave as an IPFIX > Exporting Process as outlined in [RFC5101], except as modified by > this document. > Ditto. > The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", > "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this > > > > Trammell, et al. Expires January 15, 2009 [Page 5] > > Internet-Draft IPFIX Files July 2008 > > > document are to be interpreted as described in [RFC2119]. > > > 3. Design Overview > > An IPFIX File, as defined by this document, is simply a stream > remove "as defined by this document" > containing one or more IPFIX Messages serialized to some filesystem. > Though any set of valid IPFIX Messages can be serialized into an > IPFIX File, the specification proposes guidelines designed to ease > storage and retrieval of flow data using the format. > which format? > IPFIX Files contain only IPFIX Messages; any file metadata such as > checksums or export session details are stored using Options within > the IPFIX Message. I guess that your message is: any file metadata must be encoded in an IPFIX Message. However, existing ones must be modified or new one created? > This design has several advantages, including > complete compatibility with the IPFIX Protocol on the wire and free > manipulability of IPFIX Files through concatenation, appending, and > splitting (on IPFIX Message boundaries). A schematic of a typical > file is shown below: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Trammell, et al. Expires January 15, 2009 [Page 6] > > Internet-Draft IPFIX Files July 2008 > > > +=======================================+ > | IPFIX File | > | +===================================+ | > | | IPFIX Message | | > | | +-------------------------------+ | | > | | | Options Template Set | | | > | | | Options Template Record | | | > | | | . . . cription of IPFIX Files containing enterprise-specific Information > Elements. > Minor point. The last paragraphs contains interesting information. Maybe they're not at the right place in the section "IPFIX Documents Overview" > > 2. Terminology > > Terms used in this document that are defined in the Terminology > section of [RFC5101] are to be interpreted as defined there. > > IPFIX File: An IPFIX File is a serialized stream of IPFIX Messages > stored on a filesystem. Any IPFIX Message stream that would be > considered valid when transported one or more of the specified > IPFIX transports (SCTP, TCP, or UDP) as defined in [RFC5101] is > considered an IPFIX File for purposes of this document; however, > this document extends that definition with recommendations on the > construction of IPFIX Files that meet the requirements identified > herein. > Your first sentence says: IPFIX file = stream of IPFIX Messages _stored on a filesystem_ Your second sentence says: IPFIX file = IPFIX Message stream There is a discrepancy. > IPFIX File Reader: An IPFIX File Reader is a Process which reads > IPFIX Files from a filesystem, and is analogous to an IPFIX > Collecting Process. An IPFIX File Reader MUST behave as an IPFIX > Collecting Process as outlined in [RFC5101], except as modified by > this document. > Not sure if this is adequate to include sentences with MUST with a definition. What about? IPFIX File Reader: An IPFIX File Reader is a Process which reads IPFIX Files from a filesystem. An IPFIX File Reader is analogous to an IPFIX Collecting Process. > IPFIX File Writer: An IPFIX File Writer is a process which writes > IPFIX Files to a filesystem, and is analogous to an IPFIX > Exporting Process. An IPFIX File Writer MUST behave as an IPFIX > Exporting Process as outlined in [RFC5101], except as modified by > this document. > Ditto. > The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", > "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this > > > > Trammell, et al. Expires January 15, 2009 [Page 5] > > Internet-Draft IPFIX Files July 2008 > > > document are to be interpreted as described in [RFC2119]. > > > 3. Design Overview > > An IPFIX File, as defined by this document, is simply a stream > remove "as defined by this document" > containing one or more IPFIX Messages serialized to some filesystem. > Though any set of valid IPFIX Messages can be serialized into an > IPFIX File, the specification proposes guidelines designed to ease > storage and retrieval of flow data using the format. > which format? > IPFIX Files contain only IPFIX Messages; any file metadata such as > checksums or export session details are stored using Options within > the IPFIX Message. I guess that your message is: any file metadata must be encoded in an IPFIX Message. However, existing ones must be modified or new one created? > This design has several advantages, including > complete compatibility with the IPFIX Protocol on the wire and free > manipulability of IPFIX Files through concatenation, appending, and > splitting (on IPFIX Message boundaries). A schematic of a typical > file is shown below: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Trammell, et al. Expires January 15, 2009 [Page 6] > > Internet-Draft IPFIX Files July 2008 > > > +=======================================+ > | IPFIX File | > | +===================================+ | > | | IPFIX Message | | > | | +-------------------------------+ | | > | | | Options Template Set | | | > | | | Options Template Record | | | > | | | . . . | | | > | | +-------------------------------+ | | > | | +-------------------------------+ | | > | | | Template Set | | | > | | | Template Record | | | > | | | . . . | | | > | | +-------------------------------+ | | > | +===================================+ | > | | IPFIX Message | | > | | +-------------------------------+ | | > | | | Data Set | | | > | | | Data Record | | | > | | | . . . | | | > | | +-------------------------------+ | | > | | +-------------------------------+ | | > | | | Data Set | | | > | | | Data Record | | | > | | | . . . | | | > | | +-------------------------------+ | | > | | . . . | | > | +===================================+ | > | . . . | > +=======================================+ > > Figure 1: Typical File Structure > I had to search to see whether or not the IPFIX Message contains the IPFIX Message Header. Maybe one extra sentence would be welcome... From the definition only, it's not obvious. IPFIX Message An IPFIX Message is a message originating at the Exporting Process that carries the IPFIX records of this Exporting Process and whose destination is a Collecting Process. An IPFIX Message is encapsulated at the transport layer. We have to look at figure B of RFC5101. > See Section 7 for details of the implementation of this design, > including specific requirements and guidelines for File Readers and > File Writers, and Information Elements and Options Templates used for > file metadata. > I would remove this paragraph. Already covered. > > 4. Motivation > > There are a wide variety of applications for the file-based storage > There is > of IP flow data, across a continuum of time scales. Tools used in > the analysis of flow data and creation of analysis products often use > files as a convenient unit of work, with an ephemeral lifetime. A > set of flows relevant to a security investigation may be stored in a > file for the duration of that investigation, and further exchanged > among incident handlers via email or within an external incident > > > > Trammell, et al. Expires January 15, 2009 [Page 7] > > Internet-Draft IPFIX Files July 2008 > > > handling workflow application. Sets of flow data relevant to > Sets is confusing as it begins the sentence. Is this the definition from RFC 5101? I guess no. > Internet measurement research may be published as files, much as > libpcap packet trace files are, to provide common data sets for the > Data Sets? > repeatability of research efforts; these files would have lifetimes > measured in months or years. Operational flow measurement systems > also have a need for long-term, archival storage of flow data, either > as a primary flow data repository, or as a backing tier for online > storage in a relational database management system (RDBMS). > > The variety of applications of flow data, and the variety of > presently deployed storage approaches, would seem to indicate the > need for a standard approach to flow storage with applicability > across the continuum of time scales over which flow data is stored. > A storage format based around flat files would best address the > variety of storage requirements. While much work has been done on > structured storage via RDBMS, relational database systems are not a > good basis for format standardization owing to the fact that their > internal data struc | | | > | | +-------------------------------+ | | > | | +-------------------------------+ | | > | | | Template Set | | | > | | | Template Record | | | > | | | . . . | | | > | | +-------------------------------+ | | > | +===================================+ | > | | IPFIX Message | | > | | +-------------------------------+ | | > | | | Data Set | | | > | | | Data Record | | | > | | | . . . | | | > | | +-------------------------------+ | | > | | +-------------------------------+ | | > | | | Data Set | | | > | | | Data Record | | | > | | | . . . | | | > | | +-------------------------------+ | | > | | . . . | | > | +===================================+ | > | . . . | > +=======================================+ > > Figure 1: Typical File Structure > I had to search to see whether or not the IPFIX Message contains the IPFIX Message Header. Maybe one extra sentence would be welcome... From the definition only, it's not obvious. IPFIX Message An IPFIX Message is a message originating at the Exporting Process that carries the IPFIX records of this Exporting Process and whose destination is a Collecting Process. An IPFIX Message is encapsulated at the transport layer. We have to look at figure B of RFC5101. > See Section 7 for details of the implementation of this design, > including specific requirements and guidelines for File Readers and > File Writers, and Information Elements and Options Templates used for > file metadata. > I would remove this paragraph. Already covered. > > 4. Motivation > > There are a wide variety of applications for the file-based storage > There is > of IP flow data, across a continuum of time scales. Tools used in > the analysis of flow data and creation of analysis products often use > files as a convenient unit of work, with an ephemeral lifetime. A > set of flows relevant to a security investigation may be stored in a > file for the duration of that investigation, and further exchanged > among incident handlers via email or within an external incident > > > > Trammell, et al. Expires January 15, 2009 [Page 7] > > Internet-Draft IPFIX Files July 2008 > > > handling workflow application. Sets of flow data relevant to > Sets is confusing as it begins the sentence. Is this the definition from RFC 5101? I guess no. > Internet measurement research may be published as files, much as > libpcap packet trace files are, to provide common data sets for the > Data Sets? > repeatability of research efforts; these files would have lifetimes > measured in months or years. Operational flow measurement systems > also have a need for long-term, archival storage of flow data, either > as a primary flow data repository, or as a backing tier for online > storage in a relational database management system (RDBMS). > > The variety of applications of flow data, and the variety of > presently deployed storage approaches, would seem to indicate the > need for a standard approach to flow storage with applicability > across the continuum of time scales over which flow data is stored. > A storage format based around flat files would best address the > variety of storage requirements. While much work has been done on > structured storage via RDBMS, relational database systems are not a > good basis for format standardization owing to the fact that their > internal data strtures are generally private to a single > implementation and subject to change for internal reasons. Also, > there are a wide variety of operations available on flat files, and > external tools and standards can be leveraged to meet file-based flow > storage requirements. Further, flow data is often not very > semantically complicated, and is managed in very high volume; > therefore, an RDBMS-based flow storage system would not benefit much > from the advantages of relational database technology. > > The simplest way to create a new file format is simply to serialize > some internal data model to disk, with either textual or binary > representation of data elements, and some framing strategy for > delimiting fields and records. "Ad-hoc" file formats such as this > have several important disadvantages. They impose the semantics of > the data model from which they are derived on the file format, and as > such, they are difficult to extend, describe, and standardize. > > Indeed, one de facto standard for the storage of flow data is one of > these ad-hoc formats. A common method of storing data collected via > Cisco NetFlow V5 or V7 is to serialize a stream of raw NetFlow > or V8 > datagrams into files. These NetFlow PDU files consist of a > collection of header-prefixed blocks (corresponding to the datagrams > as received on the wire) containing fixed-length binary flow records. > NetFlow V5 and V7 data may be mixed within a given file, as the > and V8 > header on each datagram defines the NetFlow version of the records > following; there is indeed very little difference between the two > record formats. Why is this important? The important message is that their format is fixed. > While this NetFlow PDU file format has all the > disadvantages of an ad-hoc format, and is not extensible to data > models other than that defined by Cisco NetFlow, it is at least > reasonably well-understood due to its ubiquity. > > > > > Trammell, et al. Expires January 15, 2009 [Page 8] > > Internet-Draft IPFIX Files July 2008 > > > Over the past decade XML markup has emerged as a new "universal" > representation format for structured data. It is intended to be > human-readable; indeed, that is one reason for its rapid adoption. > However XML has limited usefulness for representing network flow > data. Network flow data has a simple, repetitive, non-hierarchical > structure that does not benefit much from XML. An XML representation > of flow data would be an essentially flat list of the attributes and > their values for each flow record. > > The XML approach to data encoding is very heavyweight when compared > to binary flow encoding. XML's use of start- and end-tags, and > plain-text encoding of the actual values, leads to significant > inefficiency in encoding size. Typical network flow datasets can > data sets or Data Sets? What's a _flow _dataset? Throughout the document, my impression is that there is a lack of consistency for some terms: data sets, flow data sets, Data Sets, .... flow, flow record, flow data, etc... file-based, IPFIX file, etc... > contain millions or billions of flows per hour of traffic > represented. Any increase in storage size per record can have > dramatic impact on flow data storage and transfer sizes. While data > compression algorithms can partially remove the redundancy introduced > by XML encoding, they introduce additional overhead of their own. > > A further problem is that XML processing tools require a full XML > parser. XML parsers are fully general and therefore complex, > resource-intensive and relatively slow, introducing significant > processing time overhead for large network-flow datasets. In > contrast, parsers for typical binary flow data encodings are simply > structured, since they only need to parse a very small header and > then have complete knowuctures are generally private to a single > implementation and subject to change for internal reasons. Also, > there are a wide variety of operations available on flat files, and > external tools and standards can be leveraged to meet file-based flow > storage requirements. Further, flow data is often not very > semantically complicated, and is managed in very high volume; > therefore, an RDBMS-based flow storage system would not benefit much > from the advantages of relational database technology. > > The simplest way to create a new file format is simply to serialize > some internal data model to disk, with either textual or binary > representation of data elements, and some framing strategy for > delimiting fields and records. "Ad-hoc" file formats such as this > have several important disadvantages. They impose the semantics of > the data model from which they are derived on the file format, and as > such, they are difficult to extend, describe, and standardize. > > Indeed, one de facto standard for the storage of flow data is one of > these ad-hoc formats. A common method of storing data collected via > Cisco NetFlow V5 or V7 is to serialize a stream of raw NetFlow > or V8 > datagrams into files. These NetFlow PDU files consist of a > collection of header-prefixed blocks (corresponding to the datagrams > as received on the wire) containing fixed-length binary flow records. > NetFlow V5 and V7 data may be mixed within a given file, as the > and V8 > header on each datagram defines the NetFlow version of the records > following; there is indeed very little difference between the two > record formats. Why is this important? The important message is that their format is fixed. > While this NetFlow PDU file format has all the > disadvantages of an ad-hoc format, and is not extensible to data > models other than that defined by Cisco NetFlow, it is at least > reasonably well-understood due to its ubiquity. > > > > > Trammell, et al. Expires January 15, 2009 [Page 8] > > Internet-Draft IPFIX Files July 2008 > > > Over the past decade XML markup has emerged as a new "universal" > representation format for structured data. It is intended to be > human-readable; indeed, that is one reason for its rapid adoption. > However XML has limited usefulness for representing network flow > data. Network flow data has a simple, repetitive, non-hierarchical > structure that does not benefit much from XML. An XML representation > of flow data would be an essentially flat list of the attributes and > their values for each flow record. > > The XML approach to data encoding is very heavyweight when compared > to binary flow encoding. XML's use of start- and end-tags, and > plain-text encoding of the actual values, leads to significant > inefficiency in encoding size. Typical network flow datasets can > data sets or Data Sets? What's a _flow _dataset? Throughout the document, my impression is that there is a lack of consistency for some terms: data sets, flow data sets, Data Sets, .... flow, flow record, flow data, etc... file-based, IPFIX file, etc... > contain millions or billions of flows per hour of traffic > represented. Any increase in storage size per record can have > dramatic impact on flow data storage and transfer sizes. While data > compression algorithms can partially remove the redundancy introduced > by XML encoding, they introduce additional overhead of their own. > > A further problem is that XML processing tools require a full XML > parser. XML parsers are fully general and therefore complex, > resource-intensive and relatively slow, introducing significant > processing time overhead for large network-flow datasets. In > contrast, parsers for typical binary flow data encodings are simply > structured, since they only need to parse a very small header and > then have complete knledge of all following fields for the > particular flow. These can then be read in a very efficient linear > fashion. > > This leads us to propose the IPFIX Message format as the basis for a > new flow data file format. Here is another example: flow data file format > The IPFIX working group, in defining the > IPFIX protocol, has already defined an information model and data > formatting rules for representation of flow data. Especially at > shorter time scales, when a file is a unit of data interchange, the > filesystem may be viewed as simply another IPFIX Message transport > between processes. This format is especially well suited to > representing flow data, as it was designed specifically for flow data > export; it is easily extensible unlike ad-hoc serialization, and > compact unlike XML. In addition, IPFIX is an IETF standard for the > export and collection of flow data; using a common format for storage > and analysis at the collection side allows implementors to use > substantially the same information model and data formatting > implementation for transport as well as storage. > > > 5. Requirements > > In this section, we outline a proposed set of requirements > > > > Trammell, et al. Expires January 15, 2009 [Page 9] > > Internet-Draft IPFIX Files July 2008 > > > [SAINT2007] for any persistent storage format for flow data. First > and foremost, a flow data file format should support storage across > the continuum of time scales important to flow storage applications. > Each of the requirements enumerated in the sections below is broadly > applicable to flow storage applications, though each may be more > important at certain time scales. For each, we first identify the > requirement, then explain how the IPFIX Message format addresses it, > or briefly outline the changes that must be made in order for an > IPFIX-based file format to meet the requirement. > > 5.1. Record Format Flexibility > > Due to the wide variety of flow attributes collected by different > network flow attribute measurement systems, the ideal flow storage > format will not impose a single data model or a specific record type > on the flows it stores. The file format must be flexible and > extensible; that is, it must support the definition of multiple > record types within the file itself, and must be able to support new > field types for data within the records in a graceful way. > A generic comment. So far in the document, you have chosen to use generic terms, not capitalized. I'm not too sure how it helps. I even think that this is confusing. What is a "field types for data" After all, you wrote a specification, which IMHO should reuse the terminology Something such as: Due to the wide variety of Flow attributes collected by different IPFIX Device, the ideal Flow storage format will not impose a single data model or a specific Record type on the flows it stores. The IPFIX File MUST be flexible and extensible; that is, it MUST support the definition of multiple Flow Records within the IPFIX File itself, and MUST be able to support new Information Elements. Now, the use of MUST versus must in the requirement section is debatable. > IPFIX provides record format flexibility through the use of Templates > to describe each Data Record, through the use of an IANA Registry to > define its Information Elements, and through the use of enterprise- > specific Information Elements. > > 5.2. Self Description > > Archived data may be read at a time in the future where any external > reference to the meaning of the data may be lost. The ideal flow > storage format should be self-describing; that is, a process reading > flow data from storage should be able to properly interpret the > stored flows without reference to anything other than standard > sources (e.g., the standards document describing the file format) and > the owledge of all following fields for the > particular flow. These can then be read in a very efficient linear > fashion. > > This leads us to propose the IPFIX Message format as the basis for a > new flow data file format. Here is another example: flow data file format > The IPFIX working group, in defining the > IPFIX protocol, has already defined an information model and data > formatting rules for representation of flow data. Especially at > shorter time scales, when a file is a unit of data interchange, the > filesystem may be viewed as simply another IPFIX Message transport > between processes. This format is especially well suited to > representing flow data, as it was designed specifically for flow data > export; it is easily extensible unlike ad-hoc serialization, and > compact unlike XML. In addition, IPFIX is an IETF standard for the > export and collection of flow data; using a common format for storage > and analysis at the collection side allows implementors to use > substantially the same information model and data formatting > implementation for transport as well as storage. > > > 5. Requirements > > In this section, we outline a proposed set of requirements > > > > Trammell, et al. Expires January 15, 2009 [Page 9] > > Internet-Draft IPFIX Files July 2008 > > > [SAINT2007] for any persistent storage format for flow data. First > and foremost, a flow data file format should support storage across > the continuum of time scales important to flow storage applications. > Each of the requirements enumerated in the sections below is broadly > applicable to flow storage applications, though each may be more > important at certain time scales. For each, we first identify the > requirement, then explain how the IPFIX Message format addresses it, > or briefly outline the changes that must be made in order for an > IPFIX-based file format to meet the requirement. > > 5.1. Record Format Flexibility > > Due to the wide variety of flow attributes collected by different > network flow attribute measurement systems, the ideal flow storage > format will not impose a single data model or a specific record type > on the flows it stores. The file format must be flexible and > extensible; that is, it must support the definition of multiple > record types within the file itself, and must be able to support new > field types for data within the records in a graceful way. > A generic comment. So far in the document, you have chosen to use generic terms, not capitalized. I'm not too sure how it helps. I even think that this is confusing. What is a "field types for data" After all, you wrote a specification, which IMHO should reuse the terminology Something such as: Due to the wide variety of Flow attributes collected by different IPFIX Device, the ideal Flow storage format will not impose a single data model or a specific Record type on the flows it stores. The IPFIX File MUST be flexible and extensible; that is, it MUST support the definition of multiple Flow Records within the IPFIX File itself, and MUST be able to support new Information Elements. Now, the use of MUST versus must in the requirement section is debatable. > IPFIX provides record format flexibility through the use of Templates > to describe each Data Record, through the use of an IANA Registry to > define its Information Elements, and through the use of enterprise- > specific Information Elements. > > 5.2. Self Description > > Archived data may be read at a time in the future where any external > reference to the meaning of the data may be lost. The ideal flow > storage format should be self-describing; that is, a process reading > flow data from storage should be able to properly interpret the > stored flows without reference to anything other than standard > sources (e.g., the standards document describing the file format) and > thstored flow data itself. > > The IPFIX Message format is partially self-describing; that is, IPFIX > Templates containing only IANA-assigned Information Elements can be > completely interpreted according to the IPFIX Information Model > without additional external data. > > However, Templates containing private information elements lack > detailed type and semantic information; a Collecting Process > receiving data described by a template containing private Information > Data Records by a Template. Note: from now on, I won't make this "capitalization" comment again > Elements it does not understand can only treat the data contained > within those Information Elements as octet arrays. To be fully self- > describing, enterprise-specific Information Elements must be > additionally described via IPFIX Options according to the Information > Element Type Options Template defined in "Exporting Type Information > > > > Trammell, et al. Expires January 15, 2009 [Page 10] > > Internet-Draft IPFIX Files July 2008 > > > for IPFIX Information Elements" [I-D.ietf-ipfix-exporting-type]. > > 5.3. Data Compression > > Regardless of the representation format, flow data describing traffic > on real networks tends to be highly compressible. Compression tends > to improve the scalability of flow collection systems, by reducing > the disk storage and I/O bandwidth requirement for a given workload. > The ideal flow storage format should support applications which wish > to leverage this fact by supporting compression of stored data. > > The IPFIX Message format has no support for data compression, as the > IPFIX protocol was designed for speed and simplicity of export. Of > course, any flat file is readily compressible using a wide variety of > external data compression tools, formats, and algorithms; therefore, > this requirement can be met externally. > > However, a couple of simple optimizations can be made by File Writers > to increase the integrity and usability of compressed IPFIX data; > these are outlined in Section 9.1. > > 5.4. Indexing and Searching > > Binary, record stream oriented file formats natively support only one > form of searching, sequential scan in file order. By choosing the > order of records in a file carefully (e.g., by flow start or flow end > time), a file can be indexed by a single key. > > Beyond this, properly addressing indexing is an application-specific > problem, as it inherently involves tradeoffs between storage > complexity In this section you might want to develop on the "storage complexity" What I have in mind is the difficulty for an IPFIX Device to support the "order of records in a file" option. Based on the timeouts described in [IPFIX-ARCH], it's very difficult to do order by flow start without huge buffers or constantly inserting flow records at the right place in the file. Both options are not practical in real-time. Neither in an IPFIX device or a collector. > and retrieval speed, and requirements vary widely based on > time scales and the types of queries used from site to site. > However, a generic standard flow storage format may provide limited > direct support for indexing and searching. > > The ideal flow storage format will support a limited table of > contents facility noting that the records in a file contain data > relating only to certain keys or values of keys, in order to keep > multi-file search implementations from having to scan a file for data > it does not contain. > > The IPFIX Message format has no direct support for indexing. > However, its template mechanism and the technique described in > "Reducing Redundancy in IPFIX and PSAMP Reports" > [I-D.ietf-ipfix-reducing-redundancy] can be used to describe the > contents of a file in a limited way. Additionally, as flow data is > often sorted and divided by time, the start and end time of the flows > in a file me stored flow data itself. > > The IPFIX Message format is partially self-describing; that is, IPFIX > Templates containing only IANA-assigned Information Elements can be > completely interpreted according to the IPFIX Information Model > without additional external data. > > However, Templates containing private information elements lack > detailed type and semantic information; a Collecting Process > receiving data described by a template containing private Information > Data Records by a Template. Note: from now on, I won't make this "capitalization" comment again > Elements it does not understand can only treat the data contained > within those Information Elements as octet arrays. To be fully self- > describing, enterprise-specific Information Elements must be > additionally described via IPFIX Options according to the Information > Element Type Options Template defined in "Exporting Type Information > > > > Trammell, et al. Expires January 15, 2009 [Page 10] > > Internet-Draft IPFIX Files July 2008 > > > for IPFIX Information Elements" [I-D.ietf-ipfix-exporting-type]. > > 5.3. Data Compression > > Regardless of the representation format, flow data describing traffic > on real networks tends to be highly compressible. Compression tends > to improve the scalability of flow collection systems, by reducing > the disk storage and I/O bandwidth requirement for a given workload. > The ideal flow storage format should support applications which wish > to leverage this fact by supporting compression of stored data. > > The IPFIX Message format has no support for data compression, as the > IPFIX protocol was designed for speed and simplicity of export. Of > course, any flat file is readily compressible using a wide variety of > external data compression tools, formats, and algorithms; therefore, > this requirement can be met externally. > > However, a couple of simple optimizations can be made by File Writers > to increase the integrity and usability of compressed IPFIX data; > these are outlined in Section 9.1. > > 5.4. Indexing and Searching > > Binary, record stream oriented file formats natively support only one > form of searching, sequential scan in file order. By choosing the > order of records in a file carefully (e.g., by flow start or flow end > time), a file can be indexed by a single key. > > Beyond this, properly addressing indexing is an application-specific > problem, as it inherently involves tradeoffs between storage > complexity In this section you might want to develop on the "storage complexity" What I have in mind is the difficulty for an IPFIX Device to support the "order of records in a file" option. Based on the timeouts described in [IPFIX-ARCH], it's very difficult to do order by flow start without huge buffers or constantly inserting flow records at the right place in the file. Both options are not practical in real-time. Neither in an IPFIX device or a collector. > and retrieval speed, and requirements vary widely based on > time scales and the types of queries used from site to site. > However, a generic standard flow storage format may provide limited > direct support for indexing and searching. > > The ideal flow storage format will support a limited table of > contents facility noting that the records in a file contain data > relating only to certain keys or values of keys, in order to keep > multi-file search implementations from having to scan a file for data > it does not contain. > > The IPFIX Message format has no direct support for indexing. > However, its template mechanism and the technique described in > "Reducing Redundancy in IPFIX and PSAMP Reports" > [I-D.ietf-ipfix-reducing-redundancy] can be used to describe the > contents of a file in a limited way. Additionally, as flow data is > often sorted and divided by time, the start and end time of the flows > in a fileay be declared using the File Time Window Options Template > > > > Trammell, et al. Expires January 15, 2009 [Page 11] > > Internet-Draft IPFIX Files July 2008 > > > defined in Section 8.1.2. > > 5.5. Data Integrity > > When storing flow data over long time scales, especially for archival > purposes, it is important to ensure that hardware or software faults > do not introduce errors into the data over time. The ideal flow > storage format will support the detection and correction of encoding- > level errors in the data. > > Note that more advanced error correction is almost certainly best > handled at a layer below that addressed by this document. Error > correction is a topic well addressed by the storage industry in > general (e.g. by RAID and other technologies), and by specifying a > flow storage format based upon files, we can leverage these features > to meet this requirement. > > However, the ideal flow storage format will be resilient against > errors, providing an internal facility for the detection of errors > and the ability to isolate errors to as few data records as possible. > > Note that this requirement interacts with the choice of data > compression or encryption algorithm. The use of block compression > algorithms can serve to isolate errors to a single compression block, > unlike stream compressors, which may fail to resynchronize after a > single bit error, invalidating the entire message stream. Similarly, > the use of a stream cipher can serve to isolate errors in the > plaintext without amplifying them as, for example, a cipher in CBC > mode can. See the "Recommended Compression Error Resilience > Strategy" and "Recommended Encryption Error Resilience Strategy" > sections below for more on this interaction. > > The IPFIX Message format does not support data integrity assurance. > the IPFIX specifications [RFC 5101] does not ... > It is assumed that advanced error correction will be provided > externally. For simple error detection support, checksums may be > attached to messages via IPFIX Options according to the Message > Checksum Options Template defined in Section 8.1.1. > > 5.6. Creator Authentication and Confidentiality > > Storage of flow data across long time scales may also require > remove "long time scales" > assurance that no unauthorized entity can read or modify the stored > data. Asymmetric-key cryptography can be applied to this problem, by > signing flow data with the private key of the creator, and encrypting > it with the public keys of those authorized to read it. The ideal > flow storage format will support the encryption and signing of flow > data. > > > > > Trammell, et al. Expires January 15, 2009 [Page 12] > > Internet-Draft IPFIX Files July 2008 > > > As with error correction, this problem has been addressed well at a > layer below that addressed by this document. Instead of specifying a > particular choice of encryption technology, we can leverage the fact > that existing cryptographic technologies work quite well on data > stored in files to meet this requirement. > > Beyond support for the use of TLS for transport over TCP or DTLS for > transport over SCTP or UDP, both of which provide transient > authentication and confidentiality, the IPFIX protocol does not > support this requirement directly. It is assumed that this > requirement will be met externally. > > 5.7. Anonymization and Obfuscation > > To ensure the privacy of individuals and organizations at the > endpoints of communications represented by flow records, it is often > necessary to obfuscate or anonymize stored and exported flow data. > The ideal flow storage format will provide for a notation that a > given information element on a given record type represents > anonymized, rather than real, data. > > The IPFIX M may be declared using the File Time Window Options Template > > > > Trammell, et al. Expires January 15, 2009 [Page 11] > > Internet-Draft IPFIX Files July 2008 > > > defined in Section 8.1.2. > > 5.5. Data Integrity > > When storing flow data over long time scales, especially for archival > purposes, it is important to ensure that hardware or software faults > do not introduce errors into the data over time. The ideal flow > storage format will support the detection and correction of encoding- > level errors in the data. > > Note that more advanced error correction is almost certainly best > handled at a layer below that addressed by this document. Error > correction is a topic well addressed by the storage industry in > general (e.g. by RAID and other technologies), and by specifying a > flow storage format based upon files, we can leverage these features > to meet this requirement. > > However, the ideal flow storage format will be resilient against > errors, providing an internal facility for the detection of errors > and the ability to isolate errors to as few data records as possible. > > Note that this requirement interacts with the choice of data > compression or encryption algorithm. The use of block compression > algorithms can serve to isolate errors to a single compression block, > unlike stream compressors, which may fail to resynchronize after a > single bit error, invalidating the entire message stream. Similarly, > the use of a stream cipher can serve to isolate errors in the > plaintext without amplifying them as, for example, a cipher in CBC > mode can. See the "Recommended Compression Error Resilience > Strategy" and "Recommended Encryption Error Resilience Strategy" > sections below for more on this interaction. > > The IPFIX Message format does not support data integrity assurance. > the IPFIX specifications [RFC 5101] does not ... > It is assumed that advanced error correction will be provided > externally. For simple error detection support, checksums may be > attached to messages via IPFIX Options according to the Message > Checksum Options Template defined in Section 8.1.1. > > 5.6. Creator Authentication and Confidentiality > > Storage of flow data across long time scales may also require > remove "long time scales" > assurance that no unauthorized entity can read or modify the stored > data. Asymmetric-key cryptography can be applied to this problem, by > signing flow data with the private key of the creator, and encrypting > it with the public keys of those authorized to read it. The ideal > flow storage format will support the encryption and signing of flow > data. > > > > > Trammell, et al. Expires January 15, 2009 [Page 12] > > Internet-Draft IPFIX Files July 2008 > > > As with error correction, this problem has been addressed well at a > layer below that addressed by this document. Instead of specifying a > particular choice of encryption technology, we can leverage the fact > that existing cryptographic technologies work quite well on data > stored in files to meet this requirement. > > Beyond support for the use of TLS for transport over TCP or DTLS for > transport over SCTP or UDP, both of which provide transient > authentication and confidentiality, the IPFIX protocol does not > support this requirement directly. It is assumed that this > requirement will be met externally. > > 5.7. Anonymization and Obfuscation > > To ensure the privacy of individuals and organizations at the > endpoints of communications represented by flow records, it is often > necessary to obfuscate or anonymize stored and exported flow data. > The ideal flow storage format will provide for a notation that a > given information element on a given record type represents > anonymized, rather than real, data. > > The IPFIXessage format presently has no support for anonymization > notation. the IPFIX specifications [RFC 5101] does not ... > It should be noted that anonymization is one of the > requirements given for IPFIX in [RFC3917]. The decision to qualify > this requirement with 'MAY' and not 'MUST' in the requirements > document, and its subsequent lack of specification in the current > version of the IPFIX protocol, is due to the fact that anonymization > algorithms are still an open area of research, and that there > currently exist no standardized methods for anonymization. > > No support is presently defined in [RFC5101] or this IPFIX-based File > Format for anonymization, as anonymization notation is an area of > open work for the IPFIX working group. > > 5.8. Session Auditability and Replayability > > Certain use cases for archival flow storage require the storage of > collection infrastructure details alongside the data itself. These > details include information about how and when data was received, and > where it was received from, and are useful for auditing as well as > for the replaying received data for testing purposes. > > The IPFIX Message format contains no direct support for auditability > the IPFIX specifications [RFC 5101] does not ... > and replayability, though the IPFIX Information Model does define > various Information Elements required to represent collection > infrastructure details. These details may be stored in IPFIX Files > using the Export Session Details Options Template defined in > Section 8.1.3 and the Message Details Options Template defined in > > > > Trammell, et al. Expires January 15, 2009 [Page 13] > > Internet-Draft IPFIX Files July 2008 > > > Section 8.1.4. > > 5.9. Performance Characteristics > > The ideal standard flow storage format will not have a significant > negative impact on the performance of the application generating or > processing flow data stored in the format. This is a non-functional > requirement, but it is important to note that a standard that implies > a significant performance penalty is unlikely to be widely > implemented and adopted. > > A static analysis of the IPFIX Message format would seem to suggest > that implementations of it are not particularly prone to slowness; > What is a static analysis? Not prone to slowness... at the condition that there is no re-ordering of flow records, compression, or anonymization before you write into the file. In other words, at the condition that you write in sequence what you receive/want to export. > indeed, a template-based data representation is more easily subject > to optimization for common cases than representations that embed > structural information directly in the data stream (e.g. XML). > However, a full analysis of the impact of using IPFIX Messages as a > basis for flow data storage on read/write performance will require > more implementation experience and performance measurement. > I think the section order was discussed before. I don't remember the conclusion Here I see the applicability after the requirements. > > 6. Applicability > > This section describes the specific applicability of IPFIX Files to > various use cases. IPFIX Files are particularly useful in a flow > collection and processing infrastructure using IPFIX for flow export. > We explore the applicability and provide guidelines for using IPFIX > files for the storage of flow data collected by IPFIX Collecting > Processes and NetFlow V9 collectors, the testing of IPFIX Collecting > Processes, and diagnostics of IPFIX Devices. > > 6.1. Storage of IPFIX-collected Flow Data > > IPFIX Files can naturally be used to store flow data collected by an > IPFIX Collecting Process; indeed, this was one of the primary initial > motivations behind the file format described within this document. > Using IPFIX Files as such provides a single, Message format presently has no support for anonymization > notation. the IPFIX specifications [RFC 5101] does not ... > It should be noted that anonymization is one of the > requirements given for IPFIX in [RFC3917]. The decision to qualify > this requirement with 'MAY' and not 'MUST' in the requirements > document, and its subsequent lack of specification in the current > version of the IPFIX protocol, is due to the fact that anonymization > algorithms are still an open area of research, and that there > currently exist no standardized methods for anonymization. > > No support is presently defined in [RFC5101] or this IPFIX-based File > Format for anonymization, as anonymization notation is an area of > open work for the IPFIX working group. > > 5.8. Session Auditability and Replayability > > Certain use cases for archival flow storage require the storage of > collection infrastructure details alongside the data itself. These > details include information about how and when data was received, and > where it was received from, and are useful for auditing as well as > for the replaying received data for testing purposes. > > The IPFIX Message format contains no direct support for auditability > the IPFIX specifications [RFC 5101] does not ... > and replayability, though the IPFIX Information Model does define > various Information Elements required to represent collection > infrastructure details. These details may be stored in IPFIX Files > using the Export Session Details Options Template defined in > Section 8.1.3 and the Message Details Options Template defined in > > > > Trammell, et al. Expires January 15, 2009 [Page 13] > > Internet-Draft IPFIX Files July 2008 > > > Section 8.1.4. > > 5.9. Performance Characteristics > > The ideal standard flow storage format will not have a significant > negative impact on the performance of the application generating or > processing flow data stored in the format. This is a non-functional > requirement, but it is important to note that a standard that implies > a significant performance penalty is unlikely to be widely > implemented and adopted. > > A static analysis of the IPFIX Message format would seem to suggest > that implementations of it are not particularly prone to slowness; > What is a static analysis? Not prone to slowness... at the condition that there is no re-ordering of flow records, compression, or anonymization before you write into the file. In other words, at the condition that you write in sequence what you receive/want to export. > indeed, a template-based data representation is more easily subject > to optimization for common cases than representations that embed > structural information directly in the data stream (e.g. XML). > However, a full analysis of the impact of using IPFIX Messages as a > basis for flow data storage on read/write performance will require > more implementation experience and performance measurement. > I think the section order was discussed before. I don't remember the conclusion Here I see the applicability after the requirements. > > 6. Applicability > > This section describes the specific applicability of IPFIX Files to > various use cases. IPFIX Files are particularly useful in a flow > collection and processing infrastructure using IPFIX for flow export. > We explore the applicability and provide guidelines for using IPFIX > files for the storage of flow data collected by IPFIX Collecting > Processes and NetFlow V9 collectors, the testing of IPFIX Collecting > Processes, and diagnostics of IPFIX Devices. > > 6.1. Storage of IPFIX-collected Flow Data > > IPFIX Files can naturally be used to store flow data collected by an > IPFIX Collecting Process; indeed, this was one of the primary initial > motivations behind the file format described within this document. > Using IPFIX Files as such provides a singlestandard, well- > understood encoding to be used for flow data on disk and on the wire, > and allows IPFIX implementations to leverage substantially the same > code for flow export and flow storage. In addition, the storage of > single Transport Sessions in IPFIX Files is particularly important > for network measurement research, allowing repeatability of > experiments by providing a format for the storage and exchange of > IPFIX flow trace data much as the libpcap format is used for > experiments on packet trace data. > > > > > > > Trammell, et al. Expires January 15, 2009 [Page 14] > > Internet-Draft IPFIX Files July 2008 > > > 6.2. Storage of NetFlow V9-collected Flow Data > > Although the IPFIX protocol is based on the Cisco Netflow Services, > Version 9 (NetFlow V9) protocol [RFC3954], the two have diverged > since work began on IPFIX. However, since the NetFlow V9 information > model is a compatible subset of the IPFIX information model, it is > possible to use IPFIX files to store collected NetFlow V9 flow data. > This approach may be particularly useful in multi-vendor, multi- > protocol collection infrastructures using both NetFlow V9 and IPFIX > to export flow data. > > The applicability of IPFIX Files to this use case is outlined in > Appendix B. > > 6.3. Testing IPFIX Collecting Processes > > IPFIX Files can be used to store IPFIX Messages for the testing of > IPFIX Collecting Processes. A variety of test cases may be stored in > IPFIX Files. First, IPFIX data sets collected in real network > Data Sets > environments and stored in an IPFIX File can be used as input to > check the behavior of new or extended implementations of IPFIX > Collectors. Furthermore, IPFIX Files can be used to validate the > operation of a given IPFIX Collecting Process in a new environment, > i.e., to test with recorded IPFIX data from the target network before > installing the Collecting Process in the network. > > The IPFIX File format can also be used to store artificial, non- > compliant reference messages for specific Collecting Process test > cases. Examples for such test cases are sets of IPFIX records with > undefined Information Elements, Data Records described by missing > Templates, or incorrectly framed messages or data sets. > Representative error handling test cases are defined in "IPFIX > Testing" [I-D.ietf-ipfix-testing]. > > Furthermore, fast replay of IPFIX records stored in a file can be > IPFIX records -> IPFIX Messages > used for stress/load tests (e.g., high rate of incoming Data Records, > large Templates with high Information Element counts), as described > in "IPFIX Testing" [I-D.ietf-ipfix-testing]. The provisioning and > use of a set of reference files for testing simplifies the > performance of tests and increases the comparability of test results. > > 6.4. IPFIX Device Diagnostics > > As an IPFIX File can be used store any collection of flows, the > format may also be used for dumping and storing various types of flow > data for IPFIX Device diagnostics (e.g., the open flow cache of a > Metering Process or the flow backlog of an Exporting or Collecting > Process at the time of a process reset or crash). File-based storage > > > > Trammell, et al. Expires January 15, 2009 [Page 15] > > Internet-Draft IPFIX Files July 2008 > > > is preferable to remote transmission in such error-recovery > situations. > > > 7. Detailed Description > > An IPFIX File, as introduced in Section 3 and elaborated below, is at > its core simply an IPFIX Message stream serialized to some > filesystem. Why re-explain the definition here? > Any valid serialized IPFIX Message stream MUST be > accepted by a File Reader as a valid IPFIX file. In this way, the > filesystem is simply treated as another IPFIX transport alongside > SCTP, TCP, and U, standard, well- > understood encoding to be used for flow data on disk and on the wire, > and allows IPFIX implementations to leverage substantially the same > code for flow export and flow storage. In addition, the storage of > single Transport Sessions in IPFIX Files is particularly important > for network measurement research, allowing repeatability of > experiments by providing a format for the storage and exchange of > IPFIX flow trace data much as the libpcap format is used for > experiments on packet trace data. > > > > > > > Trammell, et al. Expires January 15, 2009 [Page 14] > > Internet-Draft IPFIX Files July 2008 > > > 6.2. Storage of NetFlow V9-collected Flow Data > > Although the IPFIX protocol is based on the Cisco Netflow Services, > Version 9 (NetFlow V9) protocol [RFC3954], the two have diverged > since work began on IPFIX. However, since the NetFlow V9 information > model is a compatible subset of the IPFIX information model, it is > possible to use IPFIX files to store collected NetFlow V9 flow data. > This approach may be particularly useful in multi-vendor, multi- > protocol collection infrastructures using both NetFlow V9 and IPFIX > to export flow data. > > The applicability of IPFIX Files to this use case is outlined in > Appendix B. > > 6.3. Testing IPFIX Collecting Processes > > IPFIX Files can be used to store IPFIX Messages for the testing of > IPFIX Collecting Processes. A variety of test cases may be stored in > IPFIX Files. First, IPFIX data sets collected in real network > Data Sets > environments and stored in an IPFIX File can be used as input to > check the behavior of new or extended implementations of IPFIX > Collectors. Furthermore, IPFIX Files can be used to validate the > operation of a given IPFIX Collecting Process in a new environment, > i.e., to test with recorded IPFIX data from the target network before > installing the Collecting Process in the network. > > The IPFIX File format can also be used to store artificial, non- > compliant reference messages for specific Collecting Process test > cases. Examples for such test cases are sets of IPFIX records with > undefined Information Elements, Data Records described by missing > Templates, or incorrectly framed messages or data sets. > Representative error handling test cases are defined in "IPFIX > Testing" [I-D.ietf-ipfix-testing]. > > Furthermore, fast replay of IPFIX records stored in a file can be > IPFIX records -> IPFIX Messages > used for stress/load tests (e.g., high rate of incoming Data Records, > large Templates with high Information Element counts), as described > in "IPFIX Testing" [I-D.ietf-ipfix-testing]. The provisioning and > use of a set of reference files for testing simplifies the > performance of tests and increases the comparability of test results. > > 6.4. IPFIX Device Diagnostics > > As an IPFIX File can be used store any collection of flows, the > format may also be used for dumping and storing various types of flow > data for IPFIX Device diagnostics (e.g., the open flow cache of a > Metering Process or the flow backlog of an Exporting or Collecting > Process at the time of a process reset or crash). File-based storage > > > > Trammell, et al. Expires January 15, 2009 [Page 15] > > Internet-Draft IPFIX Files July 2008 > > > is preferable to remote transmission in such error-recovery > situations. > > > 7. Detailed Description > > An IPFIX File, as introduced in Section 3 and elaborated below, is at > its core simply an IPFIX Message stream serialized to some > filesystem. Why re-explain the definition here? > Any valid serialized IPFIX Message stream MUST be > accepted by a File Reader as a valid IPFIX file. In this way, the > filesystem is simply treated as another IPFIX transport alongside > SCTP, TCP, andDP. In contrast to normal IPFIX operation, what does it mean "normal IPFIX operation"? > the time > between a File Writer writing an IPFIX Message stream to a File and a > File Reader reading it can be extremely variable. In other words, > this notional file transport has unusually high latency, as the File > Reader and File Writer do not necessarily run at the same time. > Is this important, since the IPFIX Message contains the correct time anyway? Or do you want to say: don't rely on the write time in the file? > This section specifies the detailed actions of File Readers and File > Writers in handling IPFIX Files, and further specifies actions of > File Writers in specific use cases. Unless otherwise specified > herein, where appropriate IPFIX File Writers MUST behave as IPFIX > Exporting Processes, and IPFIX File Readers MUST behave as IPFIX > Collecting Processes. > > 7.1. File Reader Specification > > An IPFIX File Reader MUST accept as valid any serialized IPFIX > Message stream that would be considered valid by one or more of the > other defined IPFIX transport layers. You explained that already a few paragraphs before. From a spec. point of view, can we say that an IPFIX File Reader must comply to the Collecting Process specifications from RFC5101? > Practically, this means that > the union of template management features supported by SCTP, TCP, and > UDP MUST be supported in IPFIX Files. The following requirements > I have a problem with "requirement" since the requirements were treated in the previous section. I would remove this sentence. > apply to IPFIX File Readers: > > o File Readers MUST accept IPFIX Messages containing Template Sets, > Options Template Sets, and Data Sets within the same message, as > with IPFIX over TCP or UDP. > > o File Readers MUST accept Template Sets that define templates > Template > already defined within the file, as may occur with template > IPFIX File Template > retransmission when using IPFIX over UDP as described in section > 10.3.6 of [RFC5101]. In the event of a conflict between a resent > definition and a previous definition, the File Reader MUST assume > that the new template replaces the old, as consistent with UDP > template expiration and ID reuse. > What does the File Reader must do in that case? Replace or simply write in sequence? Btw, it depends on the transport. For SCTP, that's impossible: Template IDs are unique per SCTP association and per Observation Domain. If the Collecting Process receives a Template that has already been received but that has not previously been withdrawn (i.e., a Template Record from the same Exporter Observation Domain with the same Template ID received on the SCTP association), then the Collecting Process MUST shut down the association. What if the IPFIX Message is malformed? Write or not? Useful for diagnostic but ... So basically the answer is: it depends. If the Collecting Process receives a malformed IPFIX Message, it MUST reset the SCTP association, discard the IPFIX Message, and SHOULD log the error. Note that non-zero Set padding does not constitute a malformed IPFIX Message. > o File Readers MUST accept Template Withdrawals as described in > section 8 of [RFC5101], provided that the Template to be withdrawn > is defined, as is the case with IPFIX over TCP and SCTP. > So the IPFIX Writer MUST write the TWM into the IPFIX File. > > > Trammell, et al. Expires January 15, 2009 [Page 16] > > Internet-Draft IPFIX Files July 2008 > > > Note that some applications, particularly those storing large > collections of data over long periods of time, may benefit from the > ability to treat a collection of IPFIX Files as a single Transport > Session. A File Reader MAY be configurable to treat a collection of > Files (e.g., all the files in a directory) as a single Transport > UDP. In contrast to normal IPFIX operation, what does it mean "normal IPFIX operation"? > the time > between a File Writer writing an IPFIX Message stream to a File and a > File Reader reading it can be extremely variable. In other words, > this notional file transport has unusually high latency, as the File > Reader and File Writer do not necessarily run at the same time. > Is this important, since the IPFIX Message contains the correct time anyway? Or do you want to say: don't rely on the write time in the file? > This section specifies the detailed actions of File Readers and File > Writers in handling IPFIX Files, and further specifies actions of > File Writers in specific use cases. Unless otherwise specified > herein, where appropriate IPFIX File Writers MUST behave as IPFIX > Exporting Processes, and IPFIX File Readers MUST behave as IPFIX > Collecting Processes. > > 7.1. File Reader Specification > > An IPFIX File Reader MUST accept as valid any serialized IPFIX > Message stream that would be considered valid by one or more of the > other defined IPFIX transport layers. You explained that already a few paragraphs before. From a spec. point of view, can we say that an IPFIX File Reader must comply to the Collecting Process specifications from RFC5101? > Practically, this means that > the union of template management features supported by SCTP, TCP, and > UDP MUST be supported in IPFIX Files. The following requirements > I have a problem with "requirement" since the requirements were treated in the previous section. I would remove this sentence. > apply to IPFIX File Readers: > > o File Readers MUST accept IPFIX Messages containing Template Sets, > Options Template Sets, and Data Sets within the same message, as > with IPFIX over TCP or UDP. > > o File Readers MUST accept Template Sets that define templates > Template > already defined within the file, as may occur with template > IPFIX File Template > retransmission when using IPFIX over UDP as described in section > 10.3.6 of [RFC5101]. In the event of a conflict between a resent > definition and a previous definition, the File Reader MUST assume > that the new template replaces the old, as consistent with UDP > template expiration and ID reuse. > What does the File Reader must do in that case? Replace or simply write in sequence? Btw, it depends on the transport. For SCTP, that's impossible: Template IDs are unique per SCTP association and per Observation Domain. If the Collecting Process receives a Template that has already been received but that has not previously been withdrawn (i.e., a Template Record from the same Exporter Observation Domain with the same Template ID received on the SCTP association), then the Collecting Process MUST shut down the association. What if the IPFIX Message is malformed? Write or not? Useful for diagnostic but ... So basically the answer is: it depends. If the Collecting Process receives a malformed IPFIX Message, it MUST reset the SCTP association, discard the IPFIX Message, and SHOULD log the error. Note that non-zero Set padding does not constitute a malformed IPFIX Message. > o File Readers MUST accept Template Withdrawals as described in > section 8 of [RFC5101], provided that the Template to be withdrawn > is defined, as is the case with IPFIX over TCP and SCTP. > So the IPFIX Writer MUST write the TWM into the IPFIX File. > > > Trammell, et al. Expires January 15, 2009 [Page 16] > > Internet-Draft IPFIX Files July 2008 > > > Note that some applications, particularly those storing large > collections of data over long periods of time, may benefit from the > ability to treat a collection of IPFIX Files as a single Transport > Session. A File Reader MAY be configurable to treat a collection of > Files (e.g., all the files in a directory) as a single Transport Session. However, a File Reader MUST NOT treat a single IPFIX File > as containing multiple Transport Sessions. > I disagree. This was discussed already on the list. > If an IPFIX File uses the technique described in "Reducing Redundancy > in IPFIX and PSAMP Reports" [I-D.ietf-ipfix-reducing-redundancy] AND > all of the non-Options Templates in the File contain the > commonPropertiesId Information Element, a File Reader MAY assume the > set of commonPropertiesId definitions provides a complete table of > contents for the File for searching purposes. > > 7.2. File Writer Specification > > While any valid serialized IPFIX Message stream is a valid IPFIX > File, the following recommendations will improve representation > simplicity and read performance in the general case, where possible. > > File Writers SHOULD emit each Template Set or Options Template Set to > appear in the file before any Data Set described by the Templates > within that Set, to ensure the File Reader can decode every Data Set > without waiting to process subsequent Templates or Options Templates. > If the File Write is on a Collector, - Not possible with UDP. - Not possible with PR-SCTP without the [IPFIX-PER-STREAM] draft, which you should mention. Obviously, removing the restriction "one file = one stream" > File Writers SHOULD emit Data Records described by Options Templates > to appear in the file before any Data Records which depend on the > scopes defined by those options. > If the File Write is on a Collector, - Not possible with UDP. - Not possible with PR-SCTP without the [IPFIX-PER-STREAM] draft, which you should mention Obviously, removing the restriction "one file = one stream" > File Writers SHOULD use Template Withdrawals to withdraw Templates if > template IDs need to be reused. In this case, the new Templates > reusing those IDs SHOULD appear directly in the file after the > Template Withdrawals making the IDs available for reuse. If the File Write is on a Collector, - Not possible with UDP. - Not possible with PR-SCTP without the [IPFIX-PER-STREAM] draft, which you should mention Obviously, removing the restriction "one file = one stream" > Template > Withdrawals SHOULD NOT be used unless necessary to reuse template > IDs. > I don't understand what the File Writer should do with that statement? ... because I was assuming that the File Writer would simply write what it receives... > Each IPFIX File is generally synonymous with a single Transport > Session. File Writers SHOULD store the Templates and Options > required to decode the data within the File in the File itself, and > File Readers SHOULD NOT use Templates or Options defined in one file > to decode or interpret Data Sets in another. > So the IPFIX File should keep in memory the (Options) Template for that active Transport Session in case where it lasts a long time, where several IPFIX Files are required in order to insert the (Options) Template in the subsequent IPFIX Files? A simple example: [IPFIX-RED-RED] used across IPFIX Files > File Writers SHOULD write IPFIX Messages within an IPFIX File in > ascending Export Time order. > > File Writers MAY write Data Records to an IPFIX File in any order. > However, File Writers that write flow records to an IPFIX File in > > > > Trammell, et al. Expires January 15, 2009 [Page 17] > > Internet-Draft IPFIX Files July 2008 > > > flowStartTime or flowEndTime order SHOULD be consistent in this > ordering within each File. > > 7.3. Specific File Writer Use Cases > Why not put this section as 7.2.1? Some of my remarks above might be treated below... > The specifications in this section apply to specific situations. > Each section below extends or modifies the base File Writer > specification in Section 7.2. Considerations for collocation of a > File Writer with IPFIX Collecting Processes and > Session. However, a File Reader MUST NOT treat a single IPFIX File > as containing multiple Transport Sessions. > I disagree. This was discussed already on the list. > If an IPFIX File uses the technique described in "Reducing Redundancy > in IPFIX and PSAMP Reports" [I-D.ietf-ipfix-reducing-redundancy] AND > all of the non-Options Templates in the File contain the > commonPropertiesId Information Element, a File Reader MAY assume the > set of commonPropertiesId definitions provides a complete table of > contents for the File for searching purposes. > > 7.2. File Writer Specification > > While any valid serialized IPFIX Message stream is a valid IPFIX > File, the following recommendations will improve representation > simplicity and read performance in the general case, where possible. > > File Writers SHOULD emit each Template Set or Options Template Set to > appear in the file before any Data Set described by the Templates > within that Set, to ensure the File Reader can decode every Data Set > without waiting to process subsequent Templates or Options Templates. > If the File Write is on a Collector, - Not possible with UDP. - Not possible with PR-SCTP without the [IPFIX-PER-STREAM] draft, which you should mention. Obviously, removing the restriction "one file = one stream" > File Writers SHOULD emit Data Records described by Options Templates > to appear in the file before any Data Records which depend on the > scopes defined by those options. > If the File Write is on a Collector, - Not possible with UDP. - Not possible with PR-SCTP without the [IPFIX-PER-STREAM] draft, which you should mention Obviously, removing the restriction "one file = one stream" > File Writers SHOULD use Template Withdrawals to withdraw Templates if > template IDs need to be reused. In this case, the new Templates > reusing those IDs SHOULD appear directly in the file after the > Template Withdrawals making the IDs available for reuse. If the File Write is on a Collector, - Not possible with UDP. - Not possible with PR-SCTP without the [IPFIX-PER-STREAM] draft, which you should mention Obviously, removing the restriction "one file = one stream" > Template > Withdrawals SHOULD NOT be used unless necessary to reuse template > IDs. > I don't understand what the File Writer should do with that statement? ... because I was assuming that the File Writer would simply write what it receives... > Each IPFIX File is generally synonymous with a single Transport > Session. File Writers SHOULD store the Templates and Options > required to decode the data within the File in the File itself, and > File Readers SHOULD NOT use Templates or Options defined in one file > to decode or interpret Data Sets in another. > So the IPFIX File should keep in memory the (Options) Template for that active Transport Session in case where it lasts a long time, where several IPFIX Files are required in order to insert the (Options) Template in the subsequent IPFIX Files? A simple example: [IPFIX-RED-RED] used across IPFIX Files > File Writers SHOULD write IPFIX Messages within an IPFIX File in > ascending Export Time order. > > File Writers MAY write Data Records to an IPFIX File in any order. > However, File Writers that write flow records to an IPFIX File in > > > > Trammell, et al. Expires January 15, 2009 [Page 17] > > Internet-Draft IPFIX Files July 2008 > > > flowStartTime or flowEndTime order SHOULD be consistent in this > ordering within each File. > > 7.3. Specific File Writer Use Cases > Why not put this section as 7.2.1? Some of my remarks above might be treated below... > The specifications in this section apply to specific situations. > Each section below extends or modifies the base File Writer > specification in Section 7.2. Considerations for collocation of a > File Writer with IPFIX Collecting Processes anMetering Processes > are given, as are specific guidelines for using IPFIX Files for > archival storage, or as documents. Also covered are the use of IPFIX > Files in the testing and diagnostics of IPFIX Devices. > > 7.3.1. Collocating a File Writer with a Collecting Process > > When collocating a File Writer with an IPFIX Collecting Process for > archival storage of collected data in IPFIX Files as described in > Section 6.1, the following recommendations may improve the usefulness > of the stored data. > > The simplest way for a to store the data collected in a single > Transport Session is to simply write the incoming IPFIX Messages to > an IPFIX File as they are collected. However, the resulting files > IPFIX File > will lack information about the IPFIX Transport Session used to > export them, such as the network addresses of the Exporting and > Collecting Processes and the protocols used to transport them. In > this case, if information about the Transport Session is required, > the File Writer SHOULD store a single IPFIX Transport Session in an > IPFIX File and SHOULD record information about the Transport Session > using the Export Session Details Options Template described in > Section 8.1.3. > > Additional per-Message information MAY be recorded by the File Writer > using the Message Details Options Template described in > Section 8.1.4. Per-message information includes the time at which > per-message or per-Message (2 lines above) > each IPFIX Message was received at the Collecting Process, and can be > used to resend IPFIX Messages while keeping the original measurement > plane traffic profile. > > When collocating a File Writer with a Collecting Process, the Export > Time of each Message SHOULD be the Export Time of the Message > received by the Collecting Process containing the first Data Record > in the Message. Note that File Writers storing IPFIX data collected > from an IPFIX Collecting Process using SCTP as the transport protocol > SHOULD interleave messages from multiple streams in order to preserve > Export Time order, and SHOULD reorder the written messages as > necessary to ensure that each Template Set or Options Template Set > appears in the file before any Data Set described by the Templates > That's very difficult without the [IPFIX-PER-STREAM] with PR-SCTP > > > Trammell, et al. Expires January 15, 2009 [Page 18] > > Internet-Draft IPFIX Files July 2008 > > > within that Set. > > 7.3.2. Collocating a File Writer with a Metering Process > > Note that File Writers may also be collocated directly with IPFIX > Metering Processes, for writing measured information directly to disk > without intermediate IPFIX Exporting or Collecting Processes. This > arrangement may be particularly useful when providing data to an > analysis environment with an IPFIX File based workflow, or when > testing Metering Processes during development. > > When collocating a File Writer with a Metering Process, note that > Information Elements associated with Exporting or Collecting > Processes are meaningless, and SHOULD NOT appear in the Export > Session Details Options Template described in Section 8.1.3 or the > Message Details Options Template described in Section 8.1.4. > > When collocating a File Writer with an Exporting Process, the Export > Time of each Message SHOULD be the time at which the first Data > Record in the Message was received from the Exporting Process. > From the EP or MP. I would say EP, but your first sentence in this section says "without intermediate IPFIX Exporting " > 7.3.3. Using IPFIX Files for Archival Storage > > While in the general case File Writers should store one Transport > Session per IPFIX File, some applications storing large collections > of data over long periods of time may benefit from the ability to > treat a collection of IPFIX Files as a singd Metering Processes > are given, as are specific guidelines for using IPFIX Files for > archival storage, or as documents. Also covered are the use of IPFIX > Files in the testing and diagnostics of IPFIX Devices. > > 7.3.1. Collocating a File Writer with a Collecting Process > > When collocating a File Writer with an IPFIX Collecting Process for > archival storage of collected data in IPFIX Files as described in > Section 6.1, the following recommendations may improve the usefulness > of the stored data. > > The simplest way for a to store the data collected in a single > Transport Session is to simply write the incoming IPFIX Messages to > an IPFIX File as they are collected. However, the resulting files > IPFIX File > will lack information about the IPFIX Transport Session used to > export them, such as the network addresses of the Exporting and > Collecting Processes and the protocols used to transport them. In > this case, if information about the Transport Session is required, > the File Writer SHOULD store a single IPFIX Transport Session in an > IPFIX File and SHOULD record information about the Transport Session > using the Export Session Details Options Template described in > Section 8.1.3. > > Additional per-Message information MAY be recorded by the File Writer > using the Message Details Options Template described in > Section 8.1.4. Per-message information includes the time at which > per-message or per-Message (2 lines above) > each IPFIX Message was received at the Collecting Process, and can be > used to resend IPFIX Messages while keeping the original measurement > plane traffic profile. > > When collocating a File Writer with a Collecting Process, the Export > Time of each Message SHOULD be the Export Time of the Message > received by the Collecting Process containing the first Data Record > in the Message. Note that File Writers storing IPFIX data collected > from an IPFIX Collecting Process using SCTP as the transport protocol > SHOULD interleave messages from multiple streams in order to preserve > Export Time order, and SHOULD reorder the written messages as > necessary to ensure that each Template Set or Options Template Set > appears in the file before any Data Set described by the Templates > That's very difficult without the [IPFIX-PER-STREAM] with PR-SCTP > > > Trammell, et al. Expires January 15, 2009 [Page 18] > > Internet-Draft IPFIX Files July 2008 > > > within that Set. > > 7.3.2. Collocating a File Writer with a Metering Process > > Note that File Writers may also be collocated directly with IPFIX > Metering Processes, for writing measured information directly to disk > without intermediate IPFIX Exporting or Collecting Processes. This > arrangement may be particularly useful when providing data to an > analysis environment with an IPFIX File based workflow, or when > testing Metering Processes during development. > > When collocating a File Writer with a Metering Process, note that > Information Elements associated with Exporting or Collecting > Processes are meaningless, and SHOULD NOT appear in the Export > Session Details Options Template described in Section 8.1.3 or the > Message Details Options Template described in Section 8.1.4. > > When collocating a File Writer with an Exporting Process, the Export > Time of each Message SHOULD be the time at which the first Data > Record in the Message was received from the Exporting Process. > From the EP or MP. I would say EP, but your first sentence in this section says "without intermediate IPFIX Exporting " > 7.3.3. Using IPFIX Files for Archival Storage > > While in the general case File Writers should store one Transport > Session per IPFIX File, some applications storing large collections > of data over long periods of time may benefit from the ability to > treat a collection of IPFIX Files as a sile Transport Session. A > File Writer MAY be configurable to write data from a single Transport > Session into multiple IPFIX Files; however, File Writers supporting > such a configuration option MUST provide a configuration option to > support one-file-per-session behavior for interoperability purposes. > > File Writers compressing or encrypting archival data and File Readers > reading compressed or encrypted archival data SHOULD follow the > recommendations in Section 9. > > 7.3.4. Using IPFIX Files as Documents > > When IPFIX Files are used as documents, to store a set of flows > relevant to query, investigation, or other common context, or for the > publication of flow data sets relevant to network research, each File > MUST be readable as a single Transport Session, self-contained and > making no reference to metadata stored in separate Files, in order to > ensure interoperability. > > When writing Files to be used as documents, File Writers may emit the > special Data Records described by Options Templates before any other > Data Records in the File, in the following order, to ease the > > > > Trammell, et al. Expires January 15, 2009 [Page 19] > > Internet-Draft IPFIX Files July 2008 > > > inspection and use of documents by File Readers: > I would use two sentences, otherwise it's confusing. The may concerns "may emit" or "may emit in order". Btw isn't it MAY? > o Time Window records described by the File Time Window Options > Template as defined in Section 8.1.2 below; followed by > > o commonPropertiesId definitions as described in "Reducing > Redundancy in IPFIX and PSAMP Reports" > [I-D.ietf-ipfix-reducing-redundancy]; followed by > > o Information Element Type Records as described in "Exporting Type > Information for IPFIX Information Elements" > [I-D.ietf-ipfix-exporting-type]; followed by > > o Export Session details records described by the Export Session > Details Options Template as defined in Section 8.1.3 below. > > The Export Time of each Message within a File used as a document > SHOULD be the time at which the Message was written by the File > Writer. > > If an IPFIX File used as a document uses the technique described in > "Reducing Redundancy in IPFIX and PSAMP Reports" > [I-D.ietf-ipfix-reducing-redundancy] AND all of the non-Options > Templates in the File contain the commonPropertiesId Information > Element, a File Reader MAY assume the set of commonPropertiesId > definitions provides a complete table of contents for the File for > searching purposes. > > 7.3.5. Using IPFIX Files for Testing > > IPFIX Files can be used for testing IPFIX Collecting Processes in two > ways. First, IPFIX Files can be used to store specific flow data for > regression and stress testing of Collectors; there are no special > considerations for IPFIX Files used in this way. > > Second, IPFIX Files are useful for storing reference messages which > do not comply to the IPFIX Protocol in order to test the error > handling and recovery behavior of Collectors. Of course, IPFIX Files > intended to be used in this application necessarily MAY violate any > of of the specifications in this document or in [RFC5101], and such > of > files MUST NOT be transmitted to Collecting Processes or given as > input File Readers not under test. > > Note that an extremely simple IPFIX Exporting Process may be crafted > for testing purposes by simply reading an IPFIX File and transmitting > it directly to a Collecting Process. Similarly, an extremely simple > Collecting Process may be crafted for testing purposes by simply > accepting connections and/or IPFIX Messages from Exporting Processes > > > > Trammell, et al. Expires January 15, 2009 [Page 20] > > Internet-Draft IPFIX Files July 2008 > > > and writing the session's message stngle Transport Session. A > File Writer MAY be configurable to write data from a single Transport > Session into multiple IPFIX Files; however, File Writers supporting > such a configuration option MUST provide a configuration option to > support one-file-per-session behavior for interoperability purposes. > > File Writers compressing or encrypting archival data and File Readers > reading compressed or encrypted archival data SHOULD follow the > recommendations in Section 9. > > 7.3.4. Using IPFIX Files as Documents > > When IPFIX Files are used as documents, to store a set of flows > relevant to query, investigation, or other common context, or for the > publication of flow data sets relevant to network research, each File > MUST be readable as a single Transport Session, self-contained and > making no reference to metadata stored in separate Files, in order to > ensure interoperability. > > When writing Files to be used as documents, File Writers may emit the > special Data Records described by Options Templates before any other > Data Records in the File, in the following order, to ease the > > > > Trammell, et al. Expires January 15, 2009 [Page 19] > > Internet-Draft IPFIX Files July 2008 > > > inspection and use of documents by File Readers: > I would use two sentences, otherwise it's confusing. The may concerns "may emit" or "may emit in order". Btw isn't it MAY? > o Time Window records described by the File Time Window Options > Template as defined in Section 8.1.2 below; followed by > > o commonPropertiesId definitions as described in "Reducing > Redundancy in IPFIX and PSAMP Reports" > [I-D.ietf-ipfix-reducing-redundancy]; followed by > > o Information Element Type Records as described in "Exporting Type > Information for IPFIX Information Elements" > [I-D.ietf-ipfix-exporting-type]; followed by > > o Export Session details records described by the Export Session > Details Options Template as defined in Section 8.1.3 below. > > The Export Time of each Message within a File used as a document > SHOULD be the time at which the Message was written by the File > Writer. > > If an IPFIX File used as a document uses the technique described in > "Reducing Redundancy in IPFIX and PSAMP Reports" > [I-D.ietf-ipfix-reducing-redundancy] AND all of the non-Options > Templates in the File contain the commonPropertiesId Information > Element, a File Reader MAY assume the set of commonPropertiesId > definitions provides a complete table of contents for the File for > searching purposes. > > 7.3.5. Using IPFIX Files for Testing > > IPFIX Files can be used for testing IPFIX Collecting Processes in two > ways. First, IPFIX Files can be used to store specific flow data for > regression and stress testing of Collectors; there are no special > considerations for IPFIX Files used in this way. > > Second, IPFIX Files are useful for storing reference messages which > do not comply to the IPFIX Protocol in order to test the error > handling and recovery behavior of Collectors. Of course, IPFIX Files > intended to be used in this application necessarily MAY violate any > of of the specifications in this document or in [RFC5101], and such > of > files MUST NOT be transmitted to Collecting Processes or given as > input File Readers not under test. > > Note that an extremely simple IPFIX Exporting Process may be crafted > for testing purposes by simply reading an IPFIX File and transmitting > it directly to a Collecting Process. Similarly, an extremely simple > Collecting Process may be crafted for testing purposes by simply > accepting connections and/or IPFIX Messages from Exporting Processes > > > > Trammell, et al. Expires January 15, 2009 [Page 20] > > Internet-Draft IPFIX Files July 2008 > > > and writing the session's message ream to an IPFIX File. > > 7.3.6. Writing IPFIX Files for Device Diagnostics > > IPFIX Files can be used in the debugging of devices which use flow > data as internal state, as a common format for the representation of > flow tables. In such situations, the opaueOctets information element > can be used to store additional non-IPFIX encoded, non-flow > information (e.g., stack backtraces, process state, etc.) within the > IPFIX File as in Section 10.1; the IPFIX flow table information could > also be embedded in a larger proprietary diagnostic format using > delimiters as in Section 10.2 > > > 8. File Format Metadata Specification > > This section defines the Options Templates used for IPFIX File > metadata, and the Information Elements they require. > > 8.1. Recommended Options Templates for IPFIX Files > > The following Options Templates allow IPFIX Message streams to meet > the requirements outlined above without extension to the message > format or protocol. They are defined in terms of existing > Information Elements defined in [RFC5102], the Information Elements > defined in "Exporting Type Information for IPFIX Information > Elements" [I-D.ietf-ipfix-exporting-type], as well as Information > Elements defined in Section 8.2. IPFIX File Readers and Writers > SHOULD support these options templates as defined below. > There is a SHOULD here. See my comments about may versus MAY above. > In addition, IPFIX File Readers and Writers SHOULD support the > Options Templates defined in "Exporting Type Information for IPFIX > Information Elements" [I-D.ietf-ipfix-exporting-type] in order to > support self-description of enterprise-specific Information Elements. > > 8.1.1. Message Checksum Options Template > > The Message Checksum Options Template specifies the structure of a > Data Record for attaching an MD5 message checksum to an IPFIX > Message. An MD5 message checksum as described MAY be used if long- > term data integrity is important to the application. The described > Data Record MUST appear only once per IPFIX Message, but MAY appear > anywhere within the Message. > We observed an IPFIX Message, with its own length. Now, we have to add the Message Checksum Options Template. - Is this a new IPFIX Message? Then we have to say if this comes after or before the IPFIX Message it refers to. - Or we have to modify the observed IPFIX Message: changing the length of the IPFIX Message header. If this is the case (and I guess it is, but it's not clearly mentioned), then there is a problem. I export an IPFIX Message with the maximum length, the IPFIX Writer adds this Options Template before writing it into the file. Now, I transfer this file. Will the other IPFIX Reader reject it because it's not IPFIX compliant, i.e. too big? In other words, should the File Writer strictly follow RFC5101 or not? > The template SHOULD contain the following Information Elements: > > > > > > > Trammell, et al. Expires January 15, 2009 [Page 21] > > Internet-Draft IPFIX Files July 2008 > > > +--------------------+----------------------------------------------+ > | IE | Description | > +--------------------+----------------------------------------------+ > | messageScope | A marker denoting this Option applies to the | > | [scope] | whole IPFIX Message; content is ignored. | > | | This Information Element MUST be defined as | > | | a Scope Field. | > | messageMD5Checksum | The MD5 checksum of the containing IPFIX | > | | Message. | > +--------------------+----------------------------------------------+ > > 8.1.2. File Time Window Options Template > > The File Time Window Options Template specifies the structure of a > Data Record for attaching a time wistream to an IPFIX File. > > 7.3.6. Writing IPFIX Files for Device Diagnostics > > IPFIX Files can be used in the debugging of devices which use flow > data as internal state, as a common format for the representation of > flow tables. In such situations, the opaueOctets information element > can be used to store additional non-IPFIX encoded, non-flow > information (e.g., stack backtraces, process state, etc.) within the > IPFIX File as in Section 10.1; the IPFIX flow table information could > also be embedded in a larger proprietary diagnostic format using > delimiters as in Section 10.2 > > > 8. File Format Metadata Specification > > This section defines the Options Templates used for IPFIX File > metadata, and the Information Elements they require. > > 8.1. Recommended Options Templates for IPFIX Files > > The following Options Templates allow IPFIX Message streams to meet > the requirements outlined above without extension to the message > format or protocol. They are defined in terms of existing > Information Elements defined in [RFC5102], the Information Elements > defined in "Exporting Type Information for IPFIX Information > Elements" [I-D.ietf-ipfix-exporting-type], as well as Information > Elements defined in Section 8.2. IPFIX File Readers and Writers > SHOULD support these options templates as defined below. > There is a SHOULD here. See my comments about may versus MAY above. > In addition, IPFIX File Readers and Writers SHOULD support the > Options Templates defined in "Exporting Type Information for IPFIX > Information Elements" [I-D.ietf-ipfix-exporting-type] in order to > support self-description of enterprise-specific Information Elements. > > 8.1.1. Message Checksum Options Template > > The Message Checksum Options Template specifies the structure of a > Data Record for attaching an MD5 message checksum to an IPFIX > Message. An MD5 message checksum as described MAY be used if long- > term data integrity is important to the application. The described > Data Record MUST appear only once per IPFIX Message, but MAY appear > anywhere within the Message. > We observed an IPFIX Message, with its own length. Now, we have to add the Message Checksum Options Template. - Is this a new IPFIX Message? Then we have to say if this comes after or before the IPFIX Message it refers to. - Or we have to modify the observed IPFIX Message: changing the length of the IPFIX Message header. If this is the case (and I guess it is, but it's not clearly mentioned), then there is a problem. I export an IPFIX Message with the maximum length, the IPFIX Writer adds this Options Template before writing it into the file. Now, I transfer this file. Will the other IPFIX Reader reject it because it's not IPFIX compliant, i.e. too big? In other words, should the File Writer strictly follow RFC5101 or not? > The template SHOULD contain the following Information Elements: > > > > > > > Trammell, et al. Expires January 15, 2009 [Page 21] > > Internet-Draft IPFIX Files July 2008 > > > +--------------------+----------------------------------------------+ > | IE | Description | > +--------------------+----------------------------------------------+ > | messageScope | A marker denoting this Option applies to the | > | [scope] | whole IPFIX Message; content is ignored. | > | | This Information Element MUST be defined as | > | | a Scope Field. | > | messageMD5Checksum | The MD5 checksum of the containing IPFIX | > | | Message. | > +--------------------+----------------------------------------------+ > > 8.1.2. File Time Window Options Template > > The File Time Window Options Template specifies the structure of a > Data Record for attaching a time ndow to an IPFIX File; this Data > Record is referred to as a time window record. A time window record > defines the earliest flow start time and the latest flow end time of > the flow records within a File. One and only one time window record > MAY appear within an IPFIX File if the time window information is > available; a File Writer MUST NOT write more than one time window > record to an IPFIX File. A File Writer that writes a time window > record to a File MUST NOT write any Flow with a start time before the > beginning of the window or an end time after the end of the window to > that File. > Where should we add this one? - in a new IPFIX Message? - in an existing one? If yes, which one? > The template SHOULD contain the following Information Elements: > > +---------------------+---------------------------------------------+ > | IE | Description | > +---------------------+---------------------------------------------+ > | sessionScope | A marker denoting this Option applies to | > | [scope] | the whole IPFIX Transport Session (i.e., | > | | IPFIX File); content is ignored. This | > Why do you say: Transport Session and not IPFIX File? > | | Information Element MUST be defined as a | > | | Scope Field. | > | minFlowStartSeconds | The start time of the earliest flow in the | > | | Transport Session (i.e., File) in epoch | > | | seconds. | > | maxFlowEndSeconds | The end time of the latest flow in the | > | | Transport Session (i.e., File) in epoch | > | | seconds. | > +---------------------+---------------------------------------------+ > From our email exchange: Benoit: - 7.1.2 and 7.1.3. Why minFlowStartSeconds, maxFlowStartSeconds, minExportSeconds, maxExportSeconds? Why not any of the time related I.E.? Not to duplicate the I.Es? Brian: Basically, yes; the idea is that when you're using the first two IEs to attach time windows to files, second resolution is sufficient, and there's no need to get more precise than that. This may be an overly restrictive assumption. The export time window information, however, is explicitly in seconds, because it describes the IPFIX Export Time, which is also in seconds. My feedback is: - it's not specified that you meant the IPFIX Export Time. - "there's no need to get more precise than that". I disagree, the answer is: it depends. > 8.1.3. Export Session Details Options Template > > The Export Session Details Options Template specifies the structure > of a Data Record for recording the details of an IPFIX Transport > Session in an IPFIX File. It is intended for use in storing a single > > > > Trammell, et al. Expires January 15, 2009 [Page 22] > > Internet-Draft IPFIX Files July 2008 > > > complete IPFIX Transport Session in a single IPFIX File. The > described Data Record SHOULD appear only once in a given IPFIX File. > Same remark: where? > The template SHOULD contain the following Information Elements, > subject to applicability as noted on each Information Element: > MAY contain more. For example, new I.E. about SCTP streams number, or stream ID, or whatever in the future. > +----------------------------+--------------------------------------+ > | IE | Description | > +----------------------------+--------------------------------------+ > | sessionScope [scope] | A marker denoting this Option | > | | applies to the whole IPFIX Transport | > | | Session (i.e., IPFIX File); content | > | | is igwindow to an IPFIX File; this Data > Record is referred to as a time window record. A time window record > defines the earliest flow start time and the latest flow end time of > the flow records within a File. One and only one time window record > MAY appear within an IPFIX File if the time window information is > available; a File Writer MUST NOT write more than one time window > record to an IPFIX File. A File Writer that writes a time window > record to a File MUST NOT write any Flow with a start time before the > beginning of the window or an end time after the end of the window to > that File. > Where should we add this one? - in a new IPFIX Message? - in an existing one? If yes, which one? > The template SHOULD contain the following Information Elements: > > +---------------------+---------------------------------------------+ > | IE | Description | > +---------------------+---------------------------------------------+ > | sessionScope | A marker denoting this Option applies to | > | [scope] | the whole IPFIX Transport Session (i.e., | > | | IPFIX File); content is ignored. This | > Why do you say: Transport Session and not IPFIX File? > | | Information Element MUST be defined as a | > | | Scope Field. | > | minFlowStartSeconds | The start time of the earliest flow in the | > | | Transport Session (i.e., File) in epoch | > | | seconds. | > | maxFlowEndSeconds | The end time of the latest flow in the | > | | Transport Session (i.e., File) in epoch | > | | seconds. | > +---------------------+---------------------------------------------+ > From our email exchange: Benoit: - 7.1.2 and 7.1.3. Why minFlowStartSeconds, maxFlowStartSeconds, minExportSeconds, maxExportSeconds? Why not any of the time related I.E.? Not to duplicate the I.Es? Brian: Basically, yes; the idea is that when you're using the first two IEs to attach time windows to files, second resolution is sufficient, and there's no need to get more precise than that. This may be an overly restrictive assumption. The export time window information, however, is explicitly in seconds, because it describes the IPFIX Export Time, which is also in seconds. My feedback is: - it's not specified that you meant the IPFIX Export Time. - "there's no need to get more precise than that". I disagree, the answer is: it depends. > 8.1.3. Export Session Details Options Template > > The Export Session Details Options Template specifies the structure > of a Data Record for recording the details of an IPFIX Transport > Session in an IPFIX File. It is intended for use in storing a single > > > > Trammell, et al. Expires January 15, 2009 [Page 22] > > Internet-Draft IPFIX Files July 2008 > > > complete IPFIX Transport Session in a single IPFIX File. The > described Data Record SHOULD appear only once in a given IPFIX File. > Same remark: where? > The template SHOULD contain the following Information Elements, > subject to applicability as noted on each Information Element: > MAY contain more. For example, new I.E. about SCTP streams number, or stream ID, or whatever in the future. > +----------------------------+--------------------------------------+ > | IE | Description | > +----------------------------+--------------------------------------+ > | sessionScope [scope] | A marker denoting this Option | > | | applies to the whole IPFIX Transport | > | | Session (i.e., IPFIX File); content | > | | is nored. This Information | > | | Element MUST be defined as a Scope | > | | Field. | > | exporterIPv4Address | IPv4 address of the IPFIX Exporting | > | | Process from which the Messages in | > | | this Transport Session were | > | | received. Present only for | > | | Exporting Processes with an IPv4 | > | | interface. For multi-homed SCTP | > | | associations, this SHOULD be the | > | | primary path endpoint address of the | > | | Exporting Process. | > | exporterIPv6Address | IPv6 address of the IPFIX Exporting | > | | Process from which the Messages in | > | | this Transport Session were | > | | received. Present only for | > | | Exporting Processes with an IPv6 | > | | interface. For multi-homed SCTP | > | | associations, this SHOULD be the | > | | primary path endpoint address of the | > | | Exporting Process. | > | exporterTransportPort | The source port from which the | > | | Messages in this Transport Session | > | | were received. | > | collectorIPv4Address | IPv4 address of the IPFIX Collecting | > | | Process which received the Messages | > | | in this Transport Session. Present | > | | only for Collecting Processes with | > | | an IPv4 interface. For multi-homed | > | | SCTP associations, this SHOULD be | > | | the primary path endpoint address of | > | | the Collecting Process. | > > > > > > > > Trammell, et al. Expires January 15, 2009 [Page 23] > > Internet-Draft IPFIX Files July 2008 > > > | collectorIPv6Address | IPv6 address of the IPFIX Collecting | > | | Process which received the Messages | > | | in this Transport Session. Present | > | | only for Collecting Processes with | > | | an IPv6 interface. For multi-homed | > | | SCTP associations, this SHOULD be | > | | the primary path endpoint address of | > | | the Collecting Process. | > | collectorTransportPort | The destination port on which the | > | | Messages in this Transport Session | > | | were received. | > | collectorTransportProtocol | The IP Protocol Identifier of the | > | | transport protocol used to transport | > | | Messages within this Transport | > | | Session. | > | collectorProtocolVersion | The version of the IPFIX Protocol | > | | used to transport Messages within | > | | this Transport Session. | > | minExportSeconds | The Export Time of the first Message | > | | in the Transport Session. | > | maxExportSeconds | The Export Time of the last Mignored. This Information | > | | Element MUST be defined as a Scope | > | | Field. | > | exporterIPv4Address | IPv4 address of the IPFIX Exporting | > | | Process from which the Messages in | > | | this Transport Session were | > | | received. Present only for | > | | Exporting Processes with an IPv4 | > | | interface. For multi-homed SCTP | > | | associations, this SHOULD be the | > | | primary path endpoint address of the | > | | Exporting Process. | > | exporterIPv6Address | IPv6 address of the IPFIX Exporting | > | | Process from which the Messages in | > | | this Transport Session were | > | | received. Present only for | > | | Exporting Processes with an IPv6 | > | | interface. For multi-homed SCTP | > | | associations, this SHOULD be the | > | | primary path endpoint address of the | > | | Exporting Process. | > | exporterTransportPort | The source port from which the | > | | Messages in this Transport Session | > | | were received. | > | collectorIPv4Address | IPv4 address of the IPFIX Collecting | > | | Process which received the Messages | > | | in this Transport Session. Present | > | | only for Collecting Processes with | > | | an IPv4 interface. For multi-homed | > | | SCTP associations, this SHOULD be | > | | the primary path endpoint address of | > | | the Collecting Process. | > > > > > > > > Trammell, et al. Expires January 15, 2009 [Page 23] > > Internet-Draft IPFIX Files July 2008 > > > | collectorIPv6Address | IPv6 address of the IPFIX Collecting | > | | Process which received the Messages | > | | in this Transport Session. Present | > | | only for Collecting Processes with | > | | an IPv6 interface. For multi-homed | > | | SCTP associations, this SHOULD be | > | | the primary path endpoint address of | > | | the Collecting Process. | > | collectorTransportPort | The destination port on which the | > | | Messages in this Transport Session | > | | were received. | > | collectorTransportProtocol | The IP Protocol Identifier of the | > | | transport protocol used to transport | > | | Messages within this Transport | > | | Session. | > | collectorProtocolVersion | The version of the IPFIX Protocol | > | | used to transport Messages within | > | | this Transport Session. | > | minExportSeconds | The Export Time of the first Message | > | | in the Transport Session. | > | maxExportSeconds | The Export Time of the lastessage | > | | in the Transport Session. | > +----------------------------+--------------------------------------+ > > 8.1.4. Message Details Options Template > > The Message Details Options Template specifies the structure of a > Data Record for attaching additional export details to an IPFIX > Message. These details include the time at which a message was > IPFIX Message? > received and information about the export and collection > infrastructure used to transport the Message. This Options Template > also allows the storage of the export session metadata provided the > Export Session Details Options Template, for storing information from > multiple Transport Sessions in the same IPFIX File. > > The template SHOULD contain the following Information Elements, > subject to applicability as noted for each Information Element. Note > that when used in conjunction with the Export Session Details Options > Template, when storing a single complete IPFIX Transport Session in > an IPFIX File, this template SHOULD contain only the messageScope and > collectionTimeMilliseconds Information Elements, and the > exportSctpStreamId Information Element for Messages transported via > SCTP. > > > > > > > > > Trammell, et al. Expires January 15, 2009 [Page 24] > > Internet-Draft IPFIX Files July 2008 > > > +----------------------------+--------------------------------------+ > | IE | Description | > +----------------------------+--------------------------------------+ > | messageScope [scope] | A marker denoting this Option | > | | applies to the whole IPFIX message; | > | | content is ignored. This | > | | Information Element MUST be defined | > | | as a Scope Field. | > | collectionTimeMilliseconds | The absolute time at which this | > | | Message was received by the IPFIX | > | | Collecting Process. | > same remark about the timing. Why not be flexible. > | exporterIPv4Address | IPv4 address of the IPFIX Exporting | > | | Process from which this Message was | > | | received. Present only for | > | | Exporting Processes with an IPv4 | > | | interface, and if this information | > | | is not available via the Export | > | | Session Details Options Template. | > | | For multi-homed SCTP associations, | > | | this SHOULD be the primary path | > | | endpoint address of the Exporting | > | | Process. | > | exporterIPv6Address | IPv6 address of the IPFIX Exporting | > | | Process this Message was received. | > | | Present only for Exporting Processes | > | | with an IPv6 interface, and if this | > | | information is not available via the | > | | Export Session Details Options | > | | Template. For multi-homed SCTP | > | | associations, this SHOULD be the | > | | primary path endpoint address of the | > | | Exporting Process. | > | exporterTransportPort | The source port from which this | > | | Message received. Present only if | > | Message | > | | in the Transport Session. | > +----------------------------+--------------------------------------+ > > 8.1.4. Message Details Options Template > > The Message Details Options Template specifies the structure of a > Data Record for attaching additional export details to an IPFIX > Message. These details include the time at which a message was > IPFIX Message? > received and information about the export and collection > infrastructure used to transport the Message. This Options Template > also allows the storage of the export session metadata provided the > Export Session Details Options Template, for storing information from > multiple Transport Sessions in the same IPFIX File. > > The template SHOULD contain the following Information Elements, > subject to applicability as noted for each Information Element. Note > that when used in conjunction with the Export Session Details Options > Template, when storing a single complete IPFIX Transport Session in > an IPFIX File, this template SHOULD contain only the messageScope and > collectionTimeMilliseconds Information Elements, and the > exportSctpStreamId Information Element for Messages transported via > SCTP. > > > > > > > > > Trammell, et al. Expires January 15, 2009 [Page 24] > > Internet-Draft IPFIX Files July 2008 > > > +----------------------------+--------------------------------------+ > | IE | Description | > +----------------------------+--------------------------------------+ > | messageScope [scope] | A marker denoting this Option | > | | applies to the whole IPFIX message; | > | | content is ignored. This | > | | Information Element MUST be defined | > | | as a Scope Field. | > | collectionTimeMilliseconds | The absolute time at which this | > | | Message was received by the IPFIX | > | | Collecting Process. | > same remark about the timing. Why not be flexible. > | exporterIPv4Address | IPv4 address of the IPFIX Exporting | > | | Process from which this Message was | > | | received. Present only for | > | | Exporting Processes with an IPv4 | > | | interface, and if this information | > | | is not available via the Export | > | | Session Details Options Template. | > | | For multi-homed SCTP associations, | > | | this SHOULD be the primary path | > | | endpoint address of the Exporting | > | | Process. | > | exporterIPv6Address | IPv6 address of the IPFIX Exporting | > | | Process this Message was received. | > | | Present only for Exporting Processes | > | | with an IPv6 interface, and if this | > | | information is not available via the | > | | Export Session Details Options | > | | Template. For multi-homed SCTP | > | | associations, this SHOULD be the | > | | primary path endpoint address of the | > | | Exporting Process. | > | exporterTransportPort | The source port from which this | > | | Message received. Present only if | > | | this information is not available | > | | via the Export Session Details | > | | Options Template. | > | collectorIPv4Address | IPv4 address of the IPFIX Collecting | > | | Process which received this Message. | > | | Present only for Collecting | > | | Processes with an IPv4 interface, | > | | and if this information is not | > | | available via the Export Session | > | | Details Options Template. For | > | | multi-homed SCTP associations, this | > | | SHOULD be the primary path endpoint | > | | address of the Collecting Process. | > > > > > Trammell, et al. Expires January 15, 2009 [Page 25] > > Internet-Draft IPFIX Files July 2008 > > > | collectorIPv6Address | IPv6 address of the IPFIX Collecting | > | | Process which received this Message. | > | | Present only for Collecting | > | | Processes with an IPv6 interface, | > | | and if this information is not | > | | available via the Export Session | > | | Details Options Template. For | > | | multi-homed SCTP associations, this | > | | SHOULD be the primary path endpoint | > | | address of the Collecting Process. | > | collectorTransportPort | The destination port on which this | > | | Message was received. Present only | > | | if this information is not available | > | | via the Export Session Details | > | | Options Template. | > | collectorTransportProtocol | The IP Protocol Identifier of the | > | | transport protocol used to transport | > | | this Message. Present only if this | > | | information is not available via the | > | | Export Session Details Options | > | | Template. | > | collectorProtocolVersion | The version of the IPFIX Protocol | > | | used to transport this Message. | > | | Present only if this information is | > | | not available via the Export Session | > | | Details Options Template. | > | exportSctpStreamId | The SCTP stream used to transport | > | | this Message. Present only if the | > | | Message was transported via SCTP. | > +----------------------------+--------------------------------------+ > > 8.2. Recommended Information Elements for IPFIX Files > > The following Information Elements are used by the options templates > in Section 8.1 to allow IPFIX Message streams to meet the > requirements outlined above without extension of the message format > or protocol. IPFIX File Readers and Writers SHOULD support these > Information Elements as defined below. > > In addition, IPFIX File Readers and Writers SHOULD support the > Information Elements defined in "Exporting Type Information for IPFIX > Information Elements" [I-D.ietf-ipfix-exporting-type] in order to > support full self-description of Information Elements. > > 8.2.1. collectionTimeMilliseconds > > > > > > > Trammell, et a | this information is not available | > | | via the Export Session Details | > | | Options Template. | > | collectorIPv4Address | IPv4 address of the IPFIX Collecting | > | | Process which received this Message. | > | | Present only for Collecting | > | | Processes with an IPv4 interface, | > | | and if this information is not | > | | available via the Export Session | > | | Details Options Template. For | > | | multi-homed SCTP associations, this | > | | SHOULD be the primary path endpoint | > | | address of the Collecting Process. | > > > > > Trammell, et al. Expires January 15, 2009 [Page 25] > > Internet-Draft IPFIX Files July 2008 > > > | collectorIPv6Address | IPv6 address of the IPFIX Collecting | > | | Process which received this Message. | > | | Present only for Collecting | > | | Processes with an IPv6 interface, | > | | and if this information is not | > | | available via the Export Session | > | | Details Options Template. For | > | | multi-homed SCTP associations, this | > | | SHOULD be the primary path endpoint | > | | address of the Collecting Process. | > | collectorTransportPort | The destination port on which this | > | | Message was received. Present only | > | | if this information is not available | > | | via the Export Session Details | > | | Options Template. | > | collectorTransportProtocol | The IP Protocol Identifier of the | > | | transport protocol used to transport | > | | this Message. Present only if this | > | | information is not available via the | > | | Export Session Details Options | > | | Template. | > | collectorProtocolVersion | The version of the IPFIX Protocol | > | | used to transport this Message. | > | | Present only if this information is | > | | not available via the Export Session | > | | Details Options Template. | > | exportSctpStreamId | The SCTP stream used to transport | > | | this Message. Present only if the | > | | Message was transported via SCTP. | > +----------------------------+--------------------------------------+ > > 8.2. Recommended Information Elements for IPFIX Files > > The following Information Elements are used by the options templates > in Section 8.1 to allow IPFIX Message streams to meet the > requirements outlined above without extension of the message format > or protocol. IPFIX File Readers and Writers SHOULD support these > Information Elements as defined below. > > In addition, IPFIX File Readers and Writers SHOULD support the > Information Elements defined in "Exporting Type Information for IPFIX > Information Elements" [I-D.ietf-ipfix-exporting-type] in order to > support full self-description of Information Elements. > > 8.2.1. collectionTimeMilliseconds > > > > > > > Trammell, etl. Expires January 15, 2009 [Page 26] > > Internet-Draft IPFIX Files July 2008 > > > Description: The absolute timestamp at which the data within the > scope containing this Information Element was received by a > Collecting Process. This Information Element SHOULD be bound to > its containing IPFIX Message via IPFIX Options and the > messageScope Information Element, as defined below. > > Abstract Data Type: dateTimeMilliseconds > > ElementId: TBD1 > > Status: current > > 8.2.2. exportSctpStreamId > > Description: The value of the SCTP Stream Identifier used by the > Exporting Process for exporting IPFIX Message data. This is > carried in the Stream Identifier field of the header of the SCTP > DATA chunk containing the IPFIX Message(s). > > Abstract Data Type: unsigned16 > > Data Type Semantics: identifier > > ElementId: TBD2 > > Status: current > > 8.2.3. maxExportSeconds > > Description: The absolute Export Time of the latest IPFIX Message > within the scope containing this Information Element. This > Information Element SHOULD be bound to its containing IPFIX > Transport Session (i.e., File) via an options record and the > sessionScope Information Element, as defined below, and SHOULD > appear only once in a given IPFIX File. > > Abstract Data Type: dateTimeSeconds > > ElementId: TBD3 > > Status: current > > Units: seconds > > > > > > > > > Trammell, et al. Expires January 15, 2009 [Page 27] > > Internet-Draft IPFIX Files July 2008 > > > 8.2.4. maxFlowEndSeconds > > Description: The latest absolute timestamp of the last packet > within any Flow within the scope containing this Information > Element, rounded up to the second. This Information Element > SHOULD be bound to its containing IPFIX Transport Session (i.e., > File) via an IPFIX Options and the sessionScope Information > Element, as defined below, and SHOULD appear only once in a given > IPFIX File. > > Abstract Data Type: dateTimeSeconds > > ElementId: TBD4 > > Status: current > > Units: seconds > > 8.2.5. messageMD5Checksum > > Description: The MD5 checksum of the IPFIX Message containing this > record. This Information Element SHOULD be bound to its > containing IPFIX Message via an options record and the > messageScope Information Element, as defined below, and SHOULD > appear only once in a given IPFIX Message. To calculate the value > of this Information Element, first buffer the containing IPFIX > Message, setting the value of this Information Element to all > zeroes. Then caluclate the MD5 checksum of the resulting buffer > as defined in [RFC1321], place the resulting value in this > Information Element, and export the buffered message. > > Abstract Data Type: octetArray (16 bytes) > > ElementId: TBD5 > > Status: current > > Reference: RFC 1321, The MD5 Message-Digest Algorithm [RFC1321] > > 8.2.6. messageScope > > Description: The presence of this Information Element as scope in > an Options Template signifies that the options described by the > Template apply to the IPFIX Message that contains them. It is > defined for general purpose message scoping of options, and > proposed specifically to allow the attachment a checksum to a > message via IPFIX Options. The value of this Information Element > MUST be written as 0 by the File Writer or Exporting Process. The > > > > Trammell, et al. Expires January 15, 2009 [Page 28] > > Internet-Draft IPFIX Files July 2008 > > > value of this Information Element MUST be ignored by the File > Reader or the Collecting Process. > > Abstract Data Type: octet > > ElementId: TBD6 > > Statu al. Expires January 15, 2009 [Page 26] > > Internet-Draft IPFIX Files July 2008 > > > Description: The absolute timestamp at which the data within the > scope containing this Information Element was received by a > Collecting Process. This Information Element SHOULD be bound to > its containing IPFIX Message via IPFIX Options and the > messageScope Information Element, as defined below. > > Abstract Data Type: dateTimeMilliseconds > > ElementId: TBD1 > > Status: current > > 8.2.2. exportSctpStreamId > > Description: The value of the SCTP Stream Identifier used by the > Exporting Process for exporting IPFIX Message data. This is > carried in the Stream Identifier field of the header of the SCTP > DATA chunk containing the IPFIX Message(s). > > Abstract Data Type: unsigned16 > > Data Type Semantics: identifier > > ElementId: TBD2 > > Status: current > > 8.2.3. maxExportSeconds > > Description: The absolute Export Time of the latest IPFIX Message > within the scope containing this Information Element. This > Information Element SHOULD be bound to its containing IPFIX > Transport Session (i.e., File) via an options record and the > sessionScope Information Element, as defined below, and SHOULD > appear only once in a given IPFIX File. > > Abstract Data Type: dateTimeSeconds > > ElementId: TBD3 > > Status: current > > Units: seconds > > > > > > > > > Trammell, et al. Expires January 15, 2009 [Page 27] > > Internet-Draft IPFIX Files July 2008 > > > 8.2.4. maxFlowEndSeconds > > Description: The latest absolute timestamp of the last packet > within any Flow within the scope containing this Information > Element, rounded up to the second. This Information Element > SHOULD be bound to its containing IPFIX Transport Session (i.e., > File) via an IPFIX Options and the sessionScope Information > Element, as defined below, and SHOULD appear only once in a given > IPFIX File. > > Abstract Data Type: dateTimeSeconds > > ElementId: TBD4 > > Status: current > > Units: seconds > > 8.2.5. messageMD5Checksum > > Description: The MD5 checksum of the IPFIX Message containing this > record. This Information Element SHOULD be bound to its > containing IPFIX Message via an options record and the > messageScope Information Element, as defined below, and SHOULD > appear only once in a given IPFIX Message. To calculate the value > of this Information Element, first buffer the containing IPFIX > Message, setting the value of this Information Element to all > zeroes. Then caluclate the MD5 checksum of the resulting buffer > as defined in [RFC1321], place the resulting value in this > Information Element, and export the buffered message. > > Abstract Data Type: octetArray (16 bytes) > > ElementId: TBD5 > > Status: current > > Reference: RFC 1321, The MD5 Message-Digest Algorithm [RFC1321] > > 8.2.6. messageScope > > Description: The presence of this Information Element as scope in > an Options Template signifies that the options described by the > Template apply to the IPFIX Message that contains them. It is > defined for general purpose message scoping of options, and > proposed specifically to allow the attachment a checksum to a > message via IPFIX Options. The value of this Information Element > MUST be written as 0 by the File Writer or Exporting Process. The > > > > Trammell, et al. Expires January 15, 2009 [Page 28] > > Internet-Draft IPFIX Files July 2008 > > > value of this Information Element MUST be ignored by the File > Reader or the Collecting Process. > > Abstract Data Type: octet > > ElementId: TBD6 > > Stas: current > > 8.2.7. minExportSeconds > > Description: The absolute Export Time of the earliest IPFIX Message > within the scope containing this Information Element. This > Information Element SHOULD be bound to its containing IPFIX > Transport Session (i.e., File) via an options record and the > sessionScope Information Element, as defined below, and SHOULD > appear only once in a given IPFIX File. > > Abstract Data Type: dateTimeSeconds > > ElementId: TBD7 > > Status: current > > Units: seconds > > 8.2.8. minFlowStartSeconds > > Description: The earliest absolute timestamp of the first packet > within any Flow within the scope containing this Information > Element, rounded down to the second. This Information Element > SHOULD be bound to its containing IPFIX Transport Session (i.e., > File) via an options record and the sessionScope Information > Element, as defined below, and SHOULD appear only once in a given > IPFIX File. > > Abstract Data Type: dateTimeSeconds > > ElementId: TBD8 > > Status: current > > Units: seconds > > > > > > > > > Trammell, et al. Expires January 15, 2009 [Page 29] > > Internet-Draft IPFIX Files July 2008 > > > 8.2.9. opaqueOctets > > Description: This Information Element is used to encapsulate non- > IPFIX data into an IPFIX Message stream, for the purpose of > allowing a non-IPFIX data processor to store a data stream inline > within an IPFIX file. A Collecting Process or File Writer MUST > NOT try to interpret this binary data. This Information Element > differs from paddingOctets as its contents are meaningful in some > non-IPFIX context, while the contents of paddingOctets MUST be > 0x00 and are intended only for Information Element alignment. > > Abstract Data Type: octet > > ElementId: TBD9 > > Status: current > > 8.2.10. sessionScope > > Description: The presence of this Information Element as scope in > an Options Template signifies that the options described by the > Template apply to the IPFIX Transport Session that contains them. > Note that as all options are implicitly scoped to Transport > Session and Observation Domain, this Information Element is > equivalent to a "null" scope. It is defined for general purpose > session scoping of options, and proposed specifically to allow the > attachment of time window to a file via IPFIX Options. The value > of this Information Element MUST be written as 0 by the File > Writer or Exporting Process. The value of this Information > Element MUST be ignored by the File Reader or the Collecting > Process. > > Abstract Data Type: octet > > ElementId: TBD10 > > Status: current > > > 9. Recommended Error Resilience Strategies > > This section describes recommended methods for making IPFIX Files > resilient to errors during storage. It is intended primarily for > applications using IPFIX Files for long-term archival storage of flow > data. > > > > > > > Trammell, et al. Expires January 15, 2009 [Page 30] > > Internet-Draft IPFIX Files July 2008 > > > 9.1. Compression Error Resilience > > Note that, since any file may be compressed and decompressed with a > variety of widely available tools implementing a variety of > compression standards (both specified and de facto), compression of > IPFIX File data can be accomplished externally. However, compression > at the file level is not particularly resilient to errors; in the > worst case, a single bit error in a stream-compressed file may result > in the loss of the entire file. > > To limit the impact of errors on the recoverability of compressed > data, we recommend the use of block compression where possible. > Ideally, the block compression algorithm should supporttus: current > > 8.2.7. minExportSeconds > > Description: The absolute Export Time of the earliest IPFIX Message > within the scope containing this Information Element. This > Information Element SHOULD be bound to its containing IPFIX > Transport Session (i.e., File) via an options record and the > sessionScope Information Element, as defined below, and SHOULD > appear only once in a given IPFIX File. > > Abstract Data Type: dateTimeSeconds > > ElementId: TBD7 > > Status: current > > Units: seconds > > 8.2.8. minFlowStartSeconds > > Description: The earliest absolute timestamp of the first packet > within any Flow within the scope containing this Information > Element, rounded down to the second. This Information Element > SHOULD be bound to its containing IPFIX Transport Session (i.e., > File) via an options record and the sessionScope Information > Element, as defined below, and SHOULD appear only once in a given > IPFIX File. > > Abstract Data Type: dateTimeSeconds > > ElementId: TBD8 > > Status: current > > Units: seconds > > > > > > > > > Trammell, et al. Expires January 15, 2009 [Page 29] > > Internet-Draft IPFIX Files July 2008 > > > 8.2.9. opaqueOctets > > Description: This Information Element is used to encapsulate non- > IPFIX data into an IPFIX Message stream, for the purpose of > allowing a non-IPFIX data processor to store a data stream inline > within an IPFIX file. A Collecting Process or File Writer MUST > NOT try to interpret this binary data. This Information Element > differs from paddingOctets as its contents are meaningful in some > non-IPFIX context, while the contents of paddingOctets MUST be > 0x00 and are intended only for Information Element alignment. > > Abstract Data Type: octet > > ElementId: TBD9 > > Status: current > > 8.2.10. sessionScope > > Description: The presence of this Information Element as scope in > an Options Template signifies that the options described by the > Template apply to the IPFIX Transport Session that contains them. > Note that as all options are implicitly scoped to Transport > Session and Observation Domain, this Information Element is > equivalent to a "null" scope. It is defined for general purpose > session scoping of options, and proposed specifically to allow the > attachment of time window to a file via IPFIX Options. The value > of this Information Element MUST be written as 0 by the File > Writer or Exporting Process. The value of this Information > Element MUST be ignored by the File Reader or the Collecting > Process. > > Abstract Data Type: octet > > ElementId: TBD10 > > Status: current > > > 9. Recommended Error Resilience Strategies > > This section describes recommended methods for making IPFIX Files > resilient to errors during storage. It is intended primarily for > applications using IPFIX Files for long-term archival storage of flow > data. > > > > > > > Trammell, et al. Expires January 15, 2009 [Page 30] > > Internet-Draft IPFIX Files July 2008 > > > 9.1. Compression Error Resilience > > Note that, since any file may be compressed and decompressed with a > variety of widely available tools implementing a variety of > compression standards (both specified and de facto), compression of > IPFIX File data can be accomplished externally. However, compression > at the file level is not particularly resilient to errors; in the > worst case, a single bit error in a stream-compressed file may result > in the loss of the entire file. > > To limit the impact of errors on the recoverability of compressed > data, we recommend the use of block compression where possible. > Ideally, the block compression algorithm should suppo the > identification and isolation of blocks containing errors; bzip2 is an > example of such a block compressor. > > Since the block boundary of a block-compressed IPFIX File may fall in > the middle of an IPFIX Message, resynchronization of an IPFIX Message > stream by a File Reader after a compression error requires some care. > The beginning of an IPFIX Message may be identified by its header > signature (the Version field of the Message Header, 0x00 0x0A, > followed by a 16-bit Message Length), but simply searching for the > first occurance of the Version field is insufficient, since these two > bytes may occur in valid IPFIX Template or Data Sets. > > Therefore, we propose the following algorithm for File Readers to > resynchronize an IPFIX Message Stream after skipping a compressed > block containing errors: > > 1. Search after the error for the first occurrence of the octet > string 0x00, 0x0A (the IPFIX Message Header Version field.) > > 2. Treat this field as the beginning of a candidate IPFIX Message. > Read the two bytes following the Version field as a Message > Length, and seek to that offset from the beginning of the > candidate IPFIX Message. > > 3. If the first two octets after the candidate IPFIX Message are > 0x00, 0x0A (i.e., the IPFIX Message Header Version field of the > next message in the stream), or if the end of the file is reached > precisely at the end of the candidate IPFIX Message, presume that > the candidate IPFIX Message is valid, and begin reading the IPFIX > File from the start of the candidate IPFIX Message. > > 4. If not, or if the seek reaches end-of-file or another block > containing errors before finding the end of the candidate > message, go back to step 1, starting the search two bytes from > the start of the candidate IPFIX Message. > > > > Trammell, et al. Expires January 15, 2009 [Page 31] > > Internet-Draft IPFIX Files July 2008 > > > The algorithm above will improperly identify a non-message as a > message approximately 1 in 2^32 times, assuming random IPFIX data. > It may be expanded to consider multiple candidate IPFIX Messages in > order to increase reliability. > > In applications (e.g. archival storage) in which error resilience is > very important, File Writers SHOULD use block compression algorithms, > and MAY attempt to align IPFIX Messages within compression blocks to > ease resynchronization after errors, if such is supported by the > chosen block compressor. File Readers SHOULD use the > resynchronization algorithm above to minimize data loss due to > compression errors. > > 9.2. Encryption Error Resilience > > File-level encryption has error resilience issues similar to file- > level compression. Single bit errors in the encrypted data stream > can result in unreadability of the entire remaining file, dependent > on the encryption method used. The use of CBC (Cipher Block > Chaining) mode, which suffers from this low error resilience, is > relatively common. > > In applications (e.g. archival storage) in which error resilience is > very important, File Writers SHOULD use a stream cipher, for example > a block cipher in OFB (Output Feedback) mode (often referred to as > stream mode) instead of modes like CBC when encrypting, since errors > are not amplified by stream ciphers: A single-bit error in the > ciphertext results in a single bit error in the plaintext. > Alternatively File Writers SHOULD use any other cipher which can > resynchronize after bit errors. An example is a block cipher in CBC > mode that is reinitialized after a specific amount of data has been > encrypted. The maximum data loss per bit-error is then up to the > next reinitialization point. In this case, File Writers SHOULD also > use the Message Checksum Options Template to attach a checksum to > each Irt the > identification and isolation of blocks containing errors; bzip2 is an > example of such a block compressor. > > Since the block boundary of a block-compressed IPFIX File may fall in > the middle of an IPFIX Message, resynchronization of an IPFIX Message > stream by a File Reader after a compression error requires some care. > The beginning of an IPFIX Message may be identified by its header > signature (the Version field of the Message Header, 0x00 0x0A, > followed by a 16-bit Message Length), but simply searching for the > first occurance of the Version field is insufficient, since these two > bytes may occur in valid IPFIX Template or Data Sets. > > Therefore, we propose the following algorithm for File Readers to > resynchronize an IPFIX Message Stream after skipping a compressed > block containing errors: > > 1. Search after the error for the first occurrence of the octet > string 0x00, 0x0A (the IPFIX Message Header Version field.) > > 2. Treat this field as the beginning of a candidate IPFIX Message. > Read the two bytes following the Version field as a Message > Length, and seek to that offset from the beginning of the > candidate IPFIX Message. > > 3. If the first two octets after the candidate IPFIX Message are > 0x00, 0x0A (i.e., the IPFIX Message Header Version field of the > next message in the stream), or if the end of the file is reached > precisely at the end of the candidate IPFIX Message, presume that > the candidate IPFIX Message is valid, and begin reading the IPFIX > File from the start of the candidate IPFIX Message. > > 4. If not, or if the seek reaches end-of-file or another block > containing errors before finding the end of the candidate > message, go back to step 1, starting the search two bytes from > the start of the candidate IPFIX Message. > > > > Trammell, et al. Expires January 15, 2009 [Page 31] > > Internet-Draft IPFIX Files July 2008 > > > The algorithm above will improperly identify a non-message as a > message approximately 1 in 2^32 times, assuming random IPFIX data. > It may be expanded to consider multiple candidate IPFIX Messages in > order to increase reliability. > > In applications (e.g. archival storage) in which error resilience is > very important, File Writers SHOULD use block compression algorithms, > and MAY attempt to align IPFIX Messages within compression blocks to > ease resynchronization after errors, if such is supported by the > chosen block compressor. File Readers SHOULD use the > resynchronization algorithm above to minimize data loss due to > compression errors. > > 9.2. Encryption Error Resilience > > File-level encryption has error resilience issues similar to file- > level compression. Single bit errors in the encrypted data stream > can result in unreadability of the entire remaining file, dependent > on the encryption method used. The use of CBC (Cipher Block > Chaining) mode, which suffers from this low error resilience, is > relatively common. > > In applications (e.g. archival storage) in which error resilience is > very important, File Writers SHOULD use a stream cipher, for example > a block cipher in OFB (Output Feedback) mode (often referred to as > stream mode) instead of modes like CBC when encrypting, since errors > are not amplified by stream ciphers: A single-bit error in the > ciphertext results in a single bit error in the plaintext. > Alternatively File Writers SHOULD use any other cipher which can > resynchronize after bit errors. An example is a block cipher in CBC > mode that is reinitialized after a specific amount of data has been > encrypted. The maximum data loss per bit-error is then up to the > next reinitialization point. In this case, File Writers SHOULD also > use the Message Checksum Options Template to attach a checksum to > eachPFIX Message in the IPFIX File, in order to support the > recognition of errors in the decrypted data. > > > 10. Recommended File Integration Strategies > > This section describes methods for integrating IPFIX File data with > other file formats. > > 10.1. Encapsulation of Non-IPFIX Data in IPFIX Files > > At times it may be useful to export or store non-IPFIX data inline in > an IPFIX File or Message stream. To do this cleanly, this data must > be encapsulated into IPFIX Messages so that an IPFIX File Reader or > > > > Trammell, et al. Expires January 15, 2009 [Page 32] > > Internet-Draft IPFIX Files July 2008 > > > Collecting Process can handle it without any need to interpret it. > At the same time, this data must not be changed during transmission > or storage. The opaqueOctets Information Element as defined in > Section 8.2.9 is provided for this encapsulation. > A new IPFIX Message is required or not? Each time a new IPFIX Message is required (maybe the case for your metadata cases), what do we do with the Sequence Number? Is the File Writer consider as a mediation function then? > Processing the encapsulated non-IPFIX data is left to a separate > processing mechanisms that can identify encapsulated non-IPFIX data > in an IPFIX message stream, but need not have any other IPFIX > handling capability, except the ability to skip over all IPFIX > messages that do not encapsulate non-IPFIX data. > > The Message Checksum Options Template, described in Section 8.1.1 may > be used as a uniform mechanism to identify errors within encapsulated > data. > > Note that this mechanism can only encapsulate data objects up to > 65,515 octets in length. If the space available in one IPFIX Message > is not enough for the amount of data to be encapsulated, then the > data must be broken into smaller segments that are encapsulated into > consecutive IPFIX Messages. Is this a generic statement (see my remark above in the metadata section) or only applicable here? > Any additional structuring or semantics > of the raw data is outside the scope of IPFIX and must be implemented > within the encapsulated binary data itself. Furthermore, the raw > encapsulated data cannot be assumed by an IPFIX File Reader to have > any specific format. > > 10.2. Encapsulation of IPFIX Files within Other File Formats > > Consequently, it may also be useful to reverse the encapsulation, > that is, to export or store IPFIX data inline within a non-IPFIX file > or data stream. This makes sense when the other file format is not > compatible with the encapsulation described above in Section 10.1. > Generally speaking, the encapsulation here will be specific to the > format of the containing file. For example, IPFIX files may be > embedded in XML elements using hex or Base64 encoding, or in raw > binary files using start and end delimiters or some form of run- > length encoding. As there are as many potential encapsulations here > as there are potential file formats, the specifics of each are out of > scope for this specification. > > > 11. Security Considerations > > The IPFIX-based file format itself does not directly introduce > security issues. Rather it is used to store information which may > for privacy or business reasons be considered sensitive. The file > format must therefore provide appropriate procedures to guarantee the > integrity and confidentiality of the stored information. > > > > > Trammell, et al. Expires January 15, 2009 [Page 33] > > Internet-Draft IPFIX Files July 2008 > > > The underlying protocol used to exchange the information that will be > stored using the format proposed in this document must as well apply > appropriate procedures to guarantee the integrity and confidentiality > of the exported information. Such issues are addressed in [RFC5101]. > > Imple IPFIX Message in the IPFIX File, in order to support the > recognition of errors in the decrypted data. > > > 10. Recommended File Integration Strategies > > This section describes methods for integrating IPFIX File data with > other file formats. > > 10.1. Encapsulation of Non-IPFIX Data in IPFIX Files > > At times it may be useful to export or store non-IPFIX data inline in > an IPFIX File or Message stream. To do this cleanly, this data must > be encapsulated into IPFIX Messages so that an IPFIX File Reader or > > > > Trammell, et al. Expires January 15, 2009 [Page 32] > > Internet-Draft IPFIX Files July 2008 > > > Collecting Process can handle it without any need to interpret it. > At the same time, this data must not be changed during transmission > or storage. The opaqueOctets Information Element as defined in > Section 8.2.9 is provided for this encapsulation. > A new IPFIX Message is required or not? Each time a new IPFIX Message is required (maybe the case for your metadata cases), what do we do with the Sequence Number? Is the File Writer consider as a mediation function then? > Processing the encapsulated non-IPFIX data is left to a separate > processing mechanisms that can identify encapsulated non-IPFIX data > in an IPFIX message stream, but need not have any other IPFIX > handling capability, except the ability to skip over all IPFIX > messages that do not encapsulate non-IPFIX data. > > The Message Checksum Options Template, described in Section 8.1.1 may > be used as a uniform mechanism to identify errors within encapsulated > data. > > Note that this mechanism can only encapsulate data objects up to > 65,515 octets in length. If the space available in one IPFIX Message > is not enough for the amount of data to be encapsulated, then the > data must be broken into smaller segments that are encapsulated into > consecutive IPFIX Messages. Is this a generic statement (see my remark above in the metadata section) or only applicable here? > Any additional structuring or semantics > of the raw data is outside the scope of IPFIX and must be implemented > within the encapsulated binary data itself. Furthermore, the raw > encapsulated data cannot be assumed by an IPFIX File Reader to have > any specific format. > > 10.2. Encapsulation of IPFIX Files within Other File Formats > > Consequently, it may also be useful to reverse the encapsulation, > that is, to export or store IPFIX data inline within a non-IPFIX file > or data stream. This makes sense when the other file format is not > compatible with the encapsulation described above in Section 10.1. > Generally speaking, the encapsulation here will be specific to the > format of the containing file. For example, IPFIX files may be > embedded in XML elements using hex or Base64 encoding, or in raw > binary files using start and end delimiters or some form of run- > length encoding. As there are as many potential encapsulations here > as there are potential file formats, the specifics of each are out of > scope for this specification. > > > 11. Security Considerations > > The IPFIX-based file format itself does not directly introduce > security issues. Rather it is used to store information which may > for privacy or business reasons be considered sensitive. The file > format must therefore provide appropriate procedures to guarantee the > integrity and confidentiality of the stored information. > > > > > Trammell, et al. Expires January 15, 2009 [Page 33] > > Internet-Draft IPFIX Files July 2008 > > > The underlying protocol used to exchange the information that will be > stored using the format proposed in this document must as well apply > appropriate procedures to guarantee the integrity and confidentiality > of the exported information. Such issues are addressed in [RFC5101]. > > Impmentors of IPFIX File Writers which store data taken from an > IPFIX Collecting Process using TLS or DTLS for transport security > should note that IPFIX Files may present a potential breach of > confidentiality if IPFIX data collected using TLS or DTLS is stored > in unencrypted files, and should consider providing an external file > encryption option to mitigate this risk. > > > 12. IANA Considerations > > This document specifies the creation of several new IPFIX Information > Elements in the IPFIX Information Element registry located at > http://www.iana.org/assignments/ipfix, as defined in Section 8.2 > above. IANA has assigned the following Information Element numbers > for their respective Information Elements as specified below: > > o Information Element number TBD1 for the collectionTimeMilliseconds > Information Element. > > o Information Element number TBD2 for the exportSctpStreamId > Information Element. > > o Information Element number TBD3 for the maxExportSeconds > Information Element. > > o Information Element number TBD4 for the maxFlowEndSeconds > Information Element. > > o Information Element number TBD5 for the messageMD5Checksum > Information Element. > > o Information Element number TBD6 for the messageScope Information > Element. > > o Information Element number TBD7 for the minExportSeconds > Information Element. > > o Information Element number TBD8 for the minFlowStartSeconds > Information Element. > > o Information Element number TBD9 for the opaqueOctets Information > Element. > > > > > Trammell, et al. Expires January 15, 2009 [Page 34] > > Internet-Draft IPFIX Files July 2008 > > > o Information Element number TBD10 for the sessionScope Information > Element. > > [NOTE for IANA: The text TBDn should be replaced with the respective > assigned Information Element numbers where they appear in this > document.] > > > 13. Acknowledgements > > Thanks to Maurizio Molina, Tom Kosnar, and Andreas Kind for technical > assistance with the requirements for a standard flow storage format. > Thanks to Benoit Claise, Paul Aitken, and Andrew Johnson for their > reviews and feedback. > > > 14. References > > 14.1. Normative References > > [RFC5101] Claise, B., "Specification of the IP Flow Information > Export (IPFIX) Protocol for the Exchange of IP Traffic > Flow Information", RFC 5101, January 2008. > > [RFC5102] Quittek, J., Bryant, S., Claise, B., Aitken, P., and J. > Meyer, "Information Model for IP Flow Information Export", > RFC 5102, January 2008. > > [I-D.ietf-ipfix-reducing-redundancy] > Boschi, E., "Reducing Redundancy in IP Flow Information > Export (IPFIX) and Packet Sampling (PSAMP) Reports", > draft-ietf-ipfix-reducing-redundancy-04 (work in > progress), May 2007. > > [I-D.ietf-ipfix-exporting-type] > Boschi, E., Trammell, B., Mark, L., and T. Zseby, > "Exporting Type Information for IPFIX Information > Elements", draft-ietf-ipfix-exporting-type-01 (work in > progress), February 2008. > > [RFC1321] Rivest, R., "The MD5 Message-Digest Algorithm", RFC 1321, > April 1992. > > 14.2. Informative References > > [I-D.ietf-ipfix-arch] > Sadasivan, G. and N. Brownlee, "Architecture Model for IP > Flow Information Export", draft-ietf-ipfix-arch-02 (work > > > > Trammell, et al. Expires January 15, 2009 [Page 35] > > Internet-Draft IPFIX Files July 2008 > > > in progress), October 2003. > > [I-D.ietf-ipfix-as] > Zseby, T., "IPFIX Applicability", draft-ietf-ipfix-as-12 > (work in progress), July 2007. > > [RFC5103] Trammell, B. and E. Boschi, "Bidirectional Fllementors of IPFIX File Writers which store data taken from an > IPFIX Collecting Process using TLS or DTLS for transport security > should note that IPFIX Files may present a potential breach of > confidentiality if IPFIX data collected using TLS or DTLS is stored > in unencrypted files, and should consider providing an external file > encryption option to mitigate this risk. > > > 12. IANA Considerations > > This document specifies the creation of several new IPFIX Information > Elements in the IPFIX Information Element registry located at > http://www.iana.org/assignments/ipfix, as defined in Section 8.2 > above. IANA has assigned the following Information Element numbers > for their respective Information Elements as specified below: > > o Information Element number TBD1 for the collectionTimeMilliseconds > Information Element. > > o Information Element number TBD2 for the exportSctpStreamId > Information Element. > > o Information Element number TBD3 for the maxExportSeconds > Information Element. > > o Information Element number TBD4 for the maxFlowEndSeconds > Information Element. > > o Information Element number TBD5 for the messageMD5Checksum > Information Element. > > o Information Element number TBD6 for the messageScope Information > Element. > > o Information Element number TBD7 for the minExportSeconds > Information Element. > > o Information Element number TBD8 for the minFlowStartSeconds > Information Element. > > o Information Element number TBD9 for the opaqueOctets Information > Element. > > > > > Trammell, et al. Expires January 15, 2009 [Page 34] > > Internet-Draft IPFIX Files July 2008 > > > o Information Element number TBD10 for the sessionScope Information > Element. > > [NOTE for IANA: The text TBDn should be replaced with the respective > assigned Information Element numbers where they appear in this > document.] > > > 13. Acknowledgements > > Thanks to Maurizio Molina, Tom Kosnar, and Andreas Kind for technical > assistance with the requirements for a standard flow storage format. > Thanks to Benoit Claise, Paul Aitken, and Andrew Johnson for their > reviews and feedback. > > > 14. References > > 14.1. Normative References > > [RFC5101] Claise, B., "Specification of the IP Flow Information > Export (IPFIX) Protocol for the Exchange of IP Traffic > Flow Information", RFC 5101, January 2008. > > [RFC5102] Quittek, J., Bryant, S., Claise, B., Aitken, P., and J. > Meyer, "Information Model for IP Flow Information Export", > RFC 5102, January 2008. > > [I-D.ietf-ipfix-reducing-redundancy] > Boschi, E., "Reducing Redundancy in IP Flow Information > Export (IPFIX) and Packet Sampling (PSAMP) Reports", > draft-ietf-ipfix-reducing-redundancy-04 (work in > progress), May 2007. > > [I-D.ietf-ipfix-exporting-type] > Boschi, E., Trammell, B., Mark, L., and T. Zseby, > "Exporting Type Information for IPFIX Information > Elements", draft-ietf-ipfix-exporting-type-01 (work in > progress), February 2008. > > [RFC1321] Rivest, R., "The MD5 Message-Digest Algorithm", RFC 1321, > April 1992. > > 14.2. Informative References > > [I-D.ietf-ipfix-arch] > Sadasivan, G. and N. Brownlee, "Architecture Model for IP > Flow Information Export", draft-ietf-ipfix-arch-02 (work > > > > Trammell, et al. Expires January 15, 2009 [Page 35] > > Internet-Draft IPFIX Files July 2008 > > > in progress), October 2003. > > [I-D.ietf-ipfix-as] > Zseby, T., "IPFIX Applicability", draft-ietf-ipfix-as-12 > (work in progress), July 2007. > > [RFC5103] Trammell, B. and E. Boschi, "Bidirectional ow Export > Using IP Flow Information Export (IPFIX)", RFC 5103, > January 2008. > > [I-D.ietf-ipfix-testing] > Schmoll, C., Aitken, P., and B. Claise, "Guidelines for IP > Flow Information eXport (IPFIX) Testing", > draft-ietf-ipfix-testing-05 (work in progress), > April 2008. > > [RFC3954] Claise, B., "Cisco Systems NetFlow Services Export Version > 9", RFC 3954, October 2004. > > [RFC3917] Quittek, J., Zseby, T., Claise, B., and S. Zander, > "Requirements for IP Flow Information Export (IPFIX)", > RFC 3917, October 2004. > > [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate > Requirement Levels", BCP 14, RFC 2119, March 1997. > > [SAINT2007] > Trammell, B., Boschi, E., Mark, L., and T. Zseby, > "Requirements for a standardized flow storage solution", > in Proceedings of the SAINT 2007 workshop on Internet > Measurement Technology, Hiroshima, Japan, January 2007. > > > Appendix A. Example IPFIX File > > In this section we will explore an example IPFIX File which > demonstrates the various features of the IPFIX File format. This > file contains flow records described by a single Template. This file > also contains a File Time Window record to note the start and end > time of the data, and an Export Session Details record to record > collection infrastructure information. Each Message within this File > also contains a Message Checksum record, as this file may be > externally encrypted and/or stored as an archive. The structure of > this file is shown in Figure 2. > > > > > > > > Trammell, et al. Expires January 15, 2009 [Page 36] > > Internet-Draft IPFIX Files July 2008 > > > +=================================================+ > | IPFIX Message seq. 0 | > | +---------------------------------------------+ | > | | Template Set (id 2) 1 rec | | > | | Data Tmpl. id 256 | | > | +---------------------------------------------+ | > | | Options Template Set (id 3) 3 recs | | > | | File Time Window Opt. Tmpl. id 257 | | > | | Message Checksum Opt. Tmpl. id 259 | | > | | Export Session Details Opt. Tmpl. id 258 | | > | +---------------------------------------------+ | > | | Data Set (id 259) [Message Checksum] 1 rec | | > | +---------------------------------------------+ | > +=================================================+ > | IPFIX Message seq. 1 | > | +---------------------------------------------+ | > | | Data Set (id 257) [File Time Window] 1 rec | | > | +---------------------------------------------+ | > | | Data Set (id 258) [Export Session] 1 rec | | > | +---------------------------------------------+ | > | | Data Set (id 259) [Message Checksum] 1 rec | | > | +---------------------------------------------+ | > +=================================================+ > | IPFIX Message seq. 4 | > | +---------------------------------------------+ | > | | Data Set (id 256) 50 recs | | > | | contains flow data | | > | +---------------------------------------------+ | > | | Data Set (id 259) [Message Checksum] 1 rec | | > | +---------------------------------------------+ | > +=================================================+ > | IPFIX Message seq. 55 | > | Flow Export > Using IP Flow Information Export (IPFIX)", RFC 5103, > January 2008. > > [I-D.ietf-ipfix-testing] > Schmoll, C., Aitken, P., and B. Claise, "Guidelines for IP > Flow Information eXport (IPFIX) Testing", > draft-ietf-ipfix-testing-05 (work in progress), > April 2008. > > [RFC3954] Claise, B., "Cisco Systems NetFlow Services Export Version > 9", RFC 3954, October 2004. > > [RFC3917] Quittek, J., Zseby, T., Claise, B., and S. Zander, > "Requirements for IP Flow Information Export (IPFIX)", > RFC 3917, October 2004. > > [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate > Requirement Levels", BCP 14, RFC 2119, March 1997. > > [SAINT2007] > Trammell, B., Boschi, E., Mark, L., and T. Zseby, > "Requirements for a standardized flow storage solution", > in Proceedings of the SAINT 2007 workshop on Internet > Measurement Technology, Hiroshima, Japan, January 2007. > > > Appendix A. Example IPFIX File > > In this section we will explore an example IPFIX File which > demonstrates the various features of the IPFIX File format. This > file contains flow records described by a single Template. This file > also contains a File Time Window record to note the start and end > time of the data, and an Export Session Details record to record > collection infrastructure information. Each Message within this File > also contains a Message Checksum record, as this file may be > externally encrypted and/or stored as an archive. The structure of > this file is shown in Figure 2. > > > > > > > > Trammell, et al. Expires January 15, 2009 [Page 36] > > Internet-Draft IPFIX Files July 2008 > > > +=================================================+ > | IPFIX Message seq. 0 | > | +---------------------------------------------+ | > | | Template Set (id 2) 1 rec | | > | | Data Tmpl. id 256 | | > | +---------------------------------------------+ | > | | Options Template Set (id 3) 3 recs | | > | | File Time Window Opt. Tmpl. id 257 | | > | | Message Checksum Opt. Tmpl. id 259 | | > | | Export Session Details Opt. Tmpl. id 258 | | > | +---------------------------------------------+ | > | | Data Set (id 259) [Message Checksum] 1 rec | | > | +---------------------------------------------+ | > +=================================================+ > | IPFIX Message seq. 1 | > | +---------------------------------------------+ | > | | Data Set (id 257) [File Time Window] 1 rec | | > | +---------------------------------------------+ | > | | Data Set (id 258) [Export Session] 1 rec | | > | +---------------------------------------------+ | > | | Data Set (id 259) [Message Checksum] 1 rec | | > | +---------------------------------------------+ | > +=================================================+ > | IPFIX Message seq. 4 | > | +---------------------------------------------+ | > | | Data Set (id 256) 50 recs | | > | | contains flow data | | > | +---------------------------------------------+ | > | | Data Set (id 259) [Message Checksum] 1 rec | | > | +---------------------------------------------+ | > +=================================================+ > | IPFIX Message seq. 55 | > | . . . | > > Figure 2: File Example Structure > > The template describing the data records contains a flow start > timestamp, an IPv4 5-tuple, and packet and octet total counts. The > Template Set defining this is as shown in Figure 3 below: > > > > > > > > > > > > > Trammell, et al. Expires January 15, 2009 [Page 37] > > Internet-Draft IPFIX Files July 2008 > > > 1 2 3 > 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | Set ID = 2 | Length = 40 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | Template ID = 256 | Field Count = 8 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > |0| flowStartSeconds = 150 | Field Length = 4 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > |0| sourceIPv4Address = 8 | Field Length = 4 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > |0| dest.IPv4Address = 12 | Field Length = 4 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > |0| sourceTransportPort = 7 | Field Length = 2 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > |0| dest.TransportPort = 11 | Field Length = 2 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > |0| protocolIdentifier = 4 | Field Length = 1 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > |0| octetTotalCount = 85 | Field Length = 4 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > |0| packetTotalCount = 86 | Field Length = 4 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > > Figure 3: File Example Data Template > > A.1. Example Options Templates > > This is followed by an Options Template Set containing the options > templates required to read the File: the File Time Window Options > Template defined in Section 8.1.2 above, the Export Session Details > Options Template defined in Section 8.1.3 above, and the Message > Checksum Options Template defined in Section 8.1.1 above. This > Options Template Set is shown in Figure 4 and Figure 5 below: > > > > > > > > > > > > > > > > > > Trammell, et al. Expires January 15, 2009 [Page 38] > > Internet-Draft IPFIX Files July 2008 > > > 1 2 3 > 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | Set ID = 3 | Length = 80 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | Template ID = 257 | Field Count = 3 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | Scope Field Count = 1 |0| sessionScope = TBD10 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | Field Length = 1 |0| minFlowStartSeconds = TBD8 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | Field Length = 4 |0| maxFlowEndSeconds = TBD4 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | Field Length = 4 | Template ID = 259 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | Field Count = 2 | Scope Field Count = 1 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > |0| messageScope = TBD6 | Field Length = 1 | . . . | > > Figure 2: File Example Structure > > The template describing the data records contains a flow start > timestamp, an IPv4 5-tuple, and packet and octet total counts. The > Template Set defining this is as shown in Figure 3 below: > > > > > > > > > > > > > Trammell, et al. Expires January 15, 2009 [Page 37] > > Internet-Draft IPFIX Files July 2008 > > > 1 2 3 > 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | Set ID = 2 | Length = 40 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | Template ID = 256 | Field Count = 8 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > |0| flowStartSeconds = 150 | Field Length = 4 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > |0| sourceIPv4Address = 8 | Field Length = 4 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > |0| dest.IPv4Address = 12 | Field Length = 4 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > |0| sourceTransportPort = 7 | Field Length = 2 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > |0| dest.TransportPort = 11 | Field Length = 2 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > |0| protocolIdentifier = 4 | Field Length = 1 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > |0| octetTotalCount = 85 | Field Length = 4 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > |0| packetTotalCount = 86 | Field Length = 4 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > > Figure 3: File Example Data Template > > A.1. Example Options Templates > > This is followed by an Options Template Set containing the options > templates required to read the File: the File Time Window Options > Template defined in Section 8.1.2 above, the Export Session Details > Options Template defined in Section 8.1.3 above, and the Message > Checksum Options Template defined in Section 8.1.1 above. This > Options Template Set is shown in Figure 4 and Figure 5 below: > > > > > > > > > > > > > > > > > > Trammell, et al. Expires January 15, 2009 [Page 38] > > Internet-Draft IPFIX Files July 2008 > > > 1 2 3 > 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | Set ID = 3 | Length = 80 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | Template ID = 257 | Field Count = 3 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | Scope Field Count = 1 |0| sessionScope = TBD10 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | Field Length = 1 |0| minFlowStartSeconds = TBD8 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | Field Length = 4 |0| maxFlowEndSeconds = TBD4 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | Field Length = 4 | Template ID = 259 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | Field Count = 2 | Scope Field Count = 1 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > |0| messageScope = TBD6 | Field Length = 1 > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > |0| messageMD5Checksum = TBD5 | Field Length = 16 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > > Figure 4: File Example Options Templates (Time Window and Checksum) > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Trammell, et al. Expires January 15, 2009 [Page 39] > > Internet-Draft IPFIX Files July 2008 > > > 1 2 3 > 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | Template ID = 258 | Field Count = 9 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | Scope Field Count = 1 |0| sessionScope = TBD10 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | Field Length = 1 |0| exporterIPv4Address = 130 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | Field Length = 4 |0| collectorIPv4Address = 211 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | Field Length = 4 |0| exporterTransportPort = 217 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | Field Length = 2 |0| col.TransportPort = 216 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | Field Length = 2 |0| col.TransportProtocol = 215 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | Field Length = 1 |0| col.ProtocolVersion = 214 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | Field Length = 1 |0| minExportSeconds = TBD7 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | Field Length = 4 |0| maxExportSeconds = TBD3 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | Field Length = 4 | set padding (2 octets) | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > > Figure 5: File Example Options Templates, Continued (Session Details) > > A.2. Example Supplemental Options Data > > Following the templates required to decode the file is the > supplemental options information used to describe the file's contents > and type information. First comes the File Time Window record; it > notes that the file contains data from 9 October 2007 between > 00:01:13 and 23:56:27 UTC, and appears as in Figure 6: > > > > > > > > > > > > > > > > > Trammell, et al. Expires January 15, 2009 [Page 40] > > Internet-Draft IPFIX Files July 2008 > > > 1 2 3 > 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | Set ID = 257 | Length = 13 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | sessionScope | minFlowStartSeconds > | 0 | 2007-10-09 00:01:13 UTC . . . > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | maxFlowEndSeconds > . . . | 2007-10-09 23:56:27 UTC . . . > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | > . . . | > +-+-+-+-+-+-+-+-+ > > Figure 6: File Example Time Window > > This is followed by information about how the data in the file was > collected, in the Export Session Details record. This record notes > that the session stored in this file was sent via SCTP from an > exporter at 192.0.2.30 port 32769 to an collector at 192.0.2.40 port > 4739, and| > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > |0| messageMD5Checksum = TBD5 | Field Length = 16 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > > Figure 4: File Example Options Templates (Time Window and Checksum) > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Trammell, et al. Expires January 15, 2009 [Page 39] > > Internet-Draft IPFIX Files July 2008 > > > 1 2 3 > 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | Template ID = 258 | Field Count = 9 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | Scope Field Count = 1 |0| sessionScope = TBD10 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | Field Length = 1 |0| exporterIPv4Address = 130 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | Field Length = 4 |0| collectorIPv4Address = 211 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | Field Length = 4 |0| exporterTransportPort = 217 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | Field Length = 2 |0| col.TransportPort = 216 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | Field Length = 2 |0| col.TransportProtocol = 215 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | Field Length = 1 |0| col.ProtocolVersion = 214 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | Field Length = 1 |0| minExportSeconds = TBD7 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | Field Length = 4 |0| maxExportSeconds = TBD3 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | Field Length = 4 | set padding (2 octets) | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > > Figure 5: File Example Options Templates, Continued (Session Details) > > A.2. Example Supplemental Options Data > > Following the templates required to decode the file is the > supplemental options information used to describe the file's contents > and type information. First comes the File Time Window record; it > notes that the file contains data from 9 October 2007 between > 00:01:13 and 23:56:27 UTC, and appears as in Figure 6: > > > > > > > > > > > > > > > > > Trammell, et al. Expires January 15, 2009 [Page 40] > > Internet-Draft IPFIX Files July 2008 > > > 1 2 3 > 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | Set ID = 257 | Length = 13 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | sessionScope | minFlowStartSeconds > | 0 | 2007-10-09 00:01:13 UTC . . . > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | maxFlowEndSeconds > . . . | 2007-10-09 23:56:27 UTC . . . > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | > . . . | > +-+-+-+-+-+-+-+-+ > > Figure 6: File Example Time Window > > This is followed by information about how the data in the file was > collected, in the Export Session Details record. This record notes > that the session stored in this file was sent via SCTP from an > exporter at 192.0.2.30 port 32769 to an collector at 192.0.2.40 port > 4739, a contains messages exported between 00:01:57 and 23:57:12 > UTC on 9 October 2007; it is represented in its Data Set as in > Figure 7: > > > > > > > > > > > > > > > > > > > > > > > > > > > > Trammell, et al. Expires January 15, 2009 [Page 41] > > Internet-Draft IPFIX Files July 2008 > > > 1 2 3 > 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | Set ID = 258 | Length = 27 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | sessionScope | exporterIPv4Address > | 0 | 192.0.2.30 . . . > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | collectorIPv4Address > . . . | 192.0.2.31 . . . > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | exporterTransportPort | cTPort > . . . | 32769 | 4739 . . . > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | cTProtocol | cPVersion | > . . . | 132 | 10 | . . . > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > minExportSeconds | > . . . 2007-10-09 00:01:57 UTC | . . . > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > maxExportSeconds | > . . . 2007-10-09 23:57:12 UTC | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > > Figure 7: File Example Export Session Details > > A.3. Example Message Checksum > > Each IPFIX Message within the file is completed with a Message > Checksum record; the structure of this record within its Data Set is > as in Figure 8: > > 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | Set ID = 259 | Length = 24 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | messageScope | | > | 0 | | > +-+-+-+-+-+-+-+-+ | > | messageMD5Checksum | > | (16 byte MD5 checksum of options message) | > | | > | | > | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | | set padding (3 octets) | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > > Figure 8: File Example Message Checksum > > > > Trammell, et al. Expires January 15, 2009 [Page 42] > > Internet-Draft IPFIX Files July 2008 > > > A.4. File Example Data Set > > After the templates and supplemental options information comes the > data itself. The first record of an example Data Set is shown with > its message and set headers in Figure 9: > > 1 2 3 > 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | Version = 10 | Length = 1296 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | Export Time = 2007-10-09 00:01:57 UTC | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | Sequend contains messages exported between 00:01:57 and 23:57:12 > UTC on 9 October 2007; it is represented in its Data Set as in > Figure 7: > > > > > > > > > > > > > > > > > > > > > > > > > > > > Trammell, et al. Expires January 15, 2009 [Page 41] > > Internet-Draft IPFIX Files July 2008 > > > 1 2 3 > 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | Set ID = 258 | Length = 27 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | sessionScope | exporterIPv4Address > | 0 | 192.0.2.30 . . . > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | collectorIPv4Address > . . . | 192.0.2.31 . . . > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | exporterTransportPort | cTPort > . . . | 32769 | 4739 . . . > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | cTProtocol | cPVersion | > . . . | 132 | 10 | . . . > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > minExportSeconds | > . . . 2007-10-09 00:01:57 UTC | . . . > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > maxExportSeconds | > . . . 2007-10-09 23:57:12 UTC | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > > Figure 7: File Example Export Session Details > > A.3. Example Message Checksum > > Each IPFIX Message within the file is completed with a Message > Checksum record; the structure of this record within its Data Set is > as in Figure 8: > > 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | Set ID = 259 | Length = 24 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | messageScope | | > | 0 | | > +-+-+-+-+-+-+-+-+ | > | messageMD5Checksum | > | (16 byte MD5 checksum of options message) | > | | > | | > | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | | set padding (3 octets) | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > > Figure 8: File Example Message Checksum > > > > Trammell, et al. Expires January 15, 2009 [Page 42] > > Internet-Draft IPFIX Files July 2008 > > > A.4. File Example Data Set > > After the templates and supplemental options information comes the > data itself. The first record of an example Data Set is shown with > its message and set headers in Figure 9: > > 1 2 3 > 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | Version = 10 | Length = 1296 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | Export Time = 2007-10-09 00:01:57 UTC | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | Seqnce Number = 4 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | Observation Domain ID = 1 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | Set ID = 256 | Length = 1254 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | flowStartSeconds | > | 2007-10-09 00:01:13 UTC | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | sourceIPv4Address | > | 192.0.2.2 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | destinationIPv4Address | > | 192.0.2.3 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | sourceTransportPort | destinationTransportPort | > | 32770 | 80 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | protocolId | totalOctetCount > | 6 | 18000 . . . > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | totalPacketCount > . . . | 65 . . . > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | (49 more records) > . . . | > +-+-+-+-+-+-+-+-+ > > Figure 9: File Example Data Set > > A.5. Complete File Example > > Bringing together the examples above and adding message headers as > appropriate, a hex dump of the first 317 bytes of the example file > constructed above would appear as in the annotated Figure 10 below. > > > > Trammell, et al. Expires January 15, 2009 [Page 43] > > Internet-Draft IPFIX Files July 2008 > > > [EDITOR'S NOTE: In this figure, xx refers to unassigned IANA IE > numbers as in the IANA Considerations section above; cs refers to > message checksum bytes that depend on the rest of the message > contents. These will have to be replaced if we keep this example > once the IE numbers are assigned.] > > 0:|00 0A 00 A0 47 0A B6 E5 00 00 00 00 00 00 00 01 > [^ first message header (length 160 bytes) --> > 16:|00 02 00 28 01 00 00 08 00 96 00 04 00 08 00 04 > [^ data template set --> > 32: 00 0C 00 04 00 07 00 02 00 0B 00 02 00 04 00 01 > > 48: 00 55 00 04 00 56 00 04|00 03 00 50 01 01 00 03 > [^ opt template set --> > 64: 00 01 xx xx 00 01 xx xx 00 04 xx xx 00 04 01 03 > > 80: 00 02 00 01 xx xx 00 01 xx xx 00 10 01 02 00 09 > > 96: 00 01 xx xx 00 01 00 82 00 04 00 D3 00 04 00 D9 > > 112: 00 02 00 D8 00 02 00 D7 00 01 00 D0 00 01 xx xx > > 128: 00 04 xx xx 00 04 00 00|01 03 00 18 00 cs cs cs > [^ checksum record --> > 144: cs cs cs cs cs cs cs cs cs cs cs cs cs 00 00 00 > > 176:|00 0A 00 50 47 0A B6 E5 00 00 00 01 00 00 00 01 > [^ second message header (length 80 bytes) --> > 192:|01 01 00 0E 00 47 0A B6 B9 47 0C 07 1B 00|01 02 > [^ time window rec -> [ session detail rec ^ --> > 208: 00 1C 00 C0 00 02 1E 0C 00 02 1F 80 01 12 83 84 > > 224: 0A 47 0A B6 E5 47 0C 07 48 00|01 03 00 18 00 cs > [ message checksum rec ^ --> > 240: cs cs cs cs cs cs cs cs cs cs cs cs cs cs cs 00 > > 256:|00 0A 05 10 47 0A B6 E5 00 00 00 06 00 00 00 01 > [^ third message header (length 1296 bytes) --> > 272:|01 00 04 E6|47 0A B6 B9 C0 00 02 02 C0 00 02 03 > [^ set hdr ][^ first data rec --> > 288: 80 02 00 50 06 00 00 46 50 00 00 00 41 > > Figure 10: File Example Hex Dump > > > > >uence Number = 4 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | Observation Domain ID = 1 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | Set ID = 256 | Length = 1254 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | flowStartSeconds | > | 2007-10-09 00:01:13 UTC | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | sourceIPv4Address | > | 192.0.2.2 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | destinationIPv4Address | > | 192.0.2.3 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | sourceTransportPort | destinationTransportPort | > | 32770 | 80 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | protocolId | totalOctetCount > | 6 | 18000 . . . > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | totalPacketCount > . . . | 65 . . . > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | (49 more records) > . . . | > +-+-+-+-+-+-+-+-+ > > Figure 9: File Example Data Set > > A.5. Complete File Example > > Bringing together the examples above and adding message headers as > appropriate, a hex dump of the first 317 bytes of the example file > constructed above would appear as in the annotated Figure 10 below. > > > > Trammell, et al. Expires January 15, 2009 [Page 43] > > Internet-Draft IPFIX Files July 2008 > > > [EDITOR'S NOTE: In this figure, xx refers to unassigned IANA IE > numbers as in the IANA Considerations section above; cs refers to > message checksum bytes that depend on the rest of the message > contents. These will have to be replaced if we keep this example > once the IE numbers are assigned.] > > 0:|00 0A 00 A0 47 0A B6 E5 00 00 00 00 00 00 00 01 > [^ first message header (length 160 bytes) --> > 16:|00 02 00 28 01 00 00 08 00 96 00 04 00 08 00 04 > [^ data template set --> > 32: 00 0C 00 04 00 07 00 02 00 0B 00 02 00 04 00 01 > > 48: 00 55 00 04 00 56 00 04|00 03 00 50 01 01 00 03 > [^ opt template set --> > 64: 00 01 xx xx 00 01 xx xx 00 04 xx xx 00 04 01 03 > > 80: 00 02 00 01 xx xx 00 01 xx xx 00 10 01 02 00 09 > > 96: 00 01 xx xx 00 01 00 82 00 04 00 D3 00 04 00 D9 > > 112: 00 02 00 D8 00 02 00 D7 00 01 00 D0 00 01 xx xx > > 128: 00 04 xx xx 00 04 00 00|01 03 00 18 00 cs cs cs > [^ checksum record --> > 144: cs cs cs cs cs cs cs cs cs cs cs cs cs 00 00 00 > > 176:|00 0A 00 50 47 0A B6 E5 00 00 00 01 00 00 00 01 > [^ second message header (length 80 bytes) --> > 192:|01 01 00 0E 00 47 0A B6 B9 47 0C 07 1B 00|01 02 > [^ time window rec -> [ session detail rec ^ --> > 208: 00 1C 00 C0 00 02 1E 0C 00 02 1F 80 01 12 83 84 > > 224: 0A 47 0A B6 E5 47 0C 07 48 00|01 03 00 18 00 cs > [ message checksum rec ^ --> > 240: cs cs cs cs cs cs cs cs cs cs cs cs cs cs cs 00 > > 256:|00 0A 05 10 47 0A B6 E5 00 00 00 06 00 00 00 01 > [^ third message header (length 1296 bytes) --> > 272:|01 00 04 E6|47 0A B6 B9 C0 00 02 02 C0 00 02 03 > [^ set hdr ][^ first data rec --> > 288: 80 02 00 50 06 00 00 46 50 00 00 00 41 > > Figure 10: File Example Hex Dump > > > > > > > > Trammell, et al. Expires January 15, 2009 [Page 44] > > Internet-Draft IPFIX Files July 2008 > > > Appendix B. Applicability of IPFIX Files to NetFlow V9 flow storage > > As the IPFIX Message format is nearly a superset of the NetFlow V9 > packet format, IPFIX Files can be used for store NetFlow V9 data > relatively easily. This section describes a method for doing so. > The differences between the two protocols are outlined in > Appendix B.1 below. A simple, lightweight, message-for-message > translation method for transforming V9 Packets into IPFIX Messages > for storage within IPFIX Files is described in Appendix B.2. An > example of this translation method is given in Appendix B.3. > > B.1. Comparing NetFlow V9 to IPFIX > > With a few caveats, the IPFIX Protocol is a superset of the NetFlow > V9 protocol, having evolved from it largely through a process of > feature addition to bring it into compliance with the IPFIX > Requirements and the needs of stakeholders within the IPFIX Working > Group. This appendix outlines the differences between the two > protocols. It is informative only, and presented as an exploration > of the two protocols to motivate the usage of IPFIX Files to store > V9-collected flow data. > > B.1.1. Message Header Format > > Both NetFlow V9 and IPFIX use streams of messages prefixed by a > message header, though the message header differs significantly > between the two. Note that in NetFlow V9 terminology, these messages > are called packets, and messages must be delimited by datagram > boundaries. IPFIX does not have this constraint. The header formats > are detailed below: > > 0 1 2 3 > 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | Version Number | Count | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | sysUpTime | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | UNIX Secs | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | Sequence Number | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | Source ID | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > > Figure 11: NetFlow V9 Packet Header Format > > > > > > Trammell, et al. Expires January 15, 2009 [Page 45] > > Internet-Draft IPFIX Files July 2008 > > > 0 1 2 3 > 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | Version Number | Length | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | Export Time | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | Sequence Number | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | Observation Domain ID | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > > Figure 12: IPFIX Message Header Format > > Version Number: The IPFIX Version Number MUST be 10, while the > NetFlow V9 Version Number MUST be 9. > > Length vs. Count: The Count field in the NetFlow V9 packet header > counts records in the message (including data and template > records), while the Length field in the IPFIX Message Header > > > > > > Trammell, et al. Expires January 15, 2009 [Page 44] > > Internet-Draft IPFIX Files July 2008 > > > Appendix B. Applicability of IPFIX Files to NetFlow V9 flow storage > > As the IPFIX Message format is nearly a superset of the NetFlow V9 > packet format, IPFIX Files can be used for store NetFlow V9 data > relatively easily. This section describes a method for doing so. > The differences between the two protocols are outlined in > Appendix B.1 below. A simple, lightweight, message-for-message > translation method for transforming V9 Packets into IPFIX Messages > for storage within IPFIX Files is described in Appendix B.2. An > example of this translation method is given in Appendix B.3. > > B.1. Comparing NetFlow V9 to IPFIX > > With a few caveats, the IPFIX Protocol is a superset of the NetFlow > V9 protocol, having evolved from it largely through a process of > feature addition to bring it into compliance with the IPFIX > Requirements and the needs of stakeholders within the IPFIX Working > Group. This appendix outlines the differences between the two > protocols. It is informative only, and presented as an exploration > of the two protocols to motivate the usage of IPFIX Files to store > V9-collected flow data. > > B.1.1. Message Header Format > > Both NetFlow V9 and IPFIX use streams of messages prefixed by a > message header, though the message header differs significantly > between the two. Note that in NetFlow V9 terminology, these messages > are called packets, and messages must be delimited by datagram > boundaries. IPFIX does not have this constraint. The header formats > are detailed below: > > 0 1 2 3 > 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | Version Number | Count | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | sysUpTime | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | UNIX Secs | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | Sequence Number | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | Source ID | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > > Figure 11: NetFlow V9 Packet Header Format > > > > > > Trammell, et al. Expires January 15, 2009 [Page 45] > > Internet-Draft IPFIX Files July 2008 > > > 0 1 2 3 > 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | Version Number | Length | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | Export Time | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | Sequence Number | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | Observation Domain ID | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > > Figure 12: IPFIX Message Header Format > > Version Number: The IPFIX Version Number MUST be 10, while the > NetFlow V9 Version Number MUST be 9. > > Length vs. Count: The Count field in the NetFlow V9 packet header > counts records in the message (including data and template > records), while the Length field in the IPFIX Message Header > counts octets in the message. Note that this implies that NetFlow > V9 collectors must rely on datagram boundaries or some other > external delimeter; or otherwise must completely consume a message > before finding its end. > > System Uptime: System uptime in milliseconds is exported in the > NetFlow V9 packet header. This field is not present in the IPFIX > Message Header, and must be exported using an IPFIX Option if > required. > > Export Time: Aside from being called UNIX Secs in the NetFlow V9 > packet header specification, the export time in seconds since 1 > January 1970 at 0000 UTC appears in both NetFlow V9 and IPFIX > message headers. > > Sequence Number: The NetFlow V9 Sequence Number counts packets, > while the IPFIX Sequence Number counts records in Data Sets. Both > are scoped to Observation Domain. > > Observation Domain ID: Similarly, the NetFlow V9 sourceID has > become the IPFIX Observation Domain ID. > > B.1.2. Set Header Format > > Set headers are identical between NetFlow V9 and IPFIX; that is, each > Set (FlowSet in NetFlow V9 terminology) is prefixed by a 4-byte set > header containing the Set ID and the length of the set in octets. > > > > > Trammell, et al. Expires January 15, 2009 [Page 46] > > Internet-Draft IPFIX Files July 2008 > > > Note that the special Set IDs are different between IPFIX and NetFlow > V9. IPFIX Template Sets are identified by Set ID 2, while NetFlow V9 > Template FlowSets are identified by Set ID 0. Similarly, IPFIX > Options Template Sets are identified by Set ID 3, while NetFlow V9 > Options Template FlowSets are identified by Set ID 1. > > Both protocols reserve Set IDs 0-255, and use Set IDs 256-65535 for > Data Sets (or FlowSets, in NetFlow V9 terminology). > > B.1.3. Template Format > > Template FlowSets in NetFlow V9 support a subset of functionality of > those in IPFIX. Specifically, NetFlow V9 does not have any support > for vendor-specific Information Elements as IPFIX does, so there is > no enterprise bit or facility for associating a private enterprise > number with an information element. > > Options Template FlowSets in NetFlow V9 are similar to Options > Template Sets in IPFIX in the same way. > > B.1.4. Information Model > > The NetFlow V9 field type definitions are a compatible subset of, and > have evolved in concert with, the IPFIX Information Model. IPFIX > Information Element numbers in the range 1-127 are defined by the > IPFIX Information Model [RFC5102] to be compatible with the > corresponding NetFlow V9 field types. > > B.1.5. Template Management > > NetFlow V9 has no concept of a Transport Session as in IPFIX, as > NetFlow V9 was designed with a connectionless transport in mind. > Template IDs are therefore scoped to an Exporting Process lifetime > (i.e., an Exporting Process instance between restarts). There is no > facility in NetFlow V9 as in IPFIX for Template withdrawal or > Template ID reuse. Template retransmission at the Exporter works as > in UDP-based IPFIX Exporting Processes. > > B.1.6. Transport > > In practice, though NetFlow V9 is designed to be transport- > independent, it is transported only over UDP. There is no facility > as in IPFIX for full connection-oriented transport without datagram > boundaries, due to the use of a record count field as opposed to a > message length field in the packet header. There is no support in > NetFlow V9 for transport layer security via TLS or DTLS. > > > > > > Trammell, et al. Expires January 15, 2009 [Page 47] > > Internet-Draft IPFIX Files July 2008 > > > B.2. A Method for Transforming NetFlow V9 messages to IPFIX > > This appendix describes a method for transforming NetFlow V9 Packets > into IPFIX Messages, which can be used to store NetFlow V9 data in > counts octets in the message. Note that this implies that NetFlow > V9 collectors must rely on datagram boundaries or some other > external delimeter; or otherwise must completely consume a message > before finding its end. > > System Uptime: System uptime in milliseconds is exported in the > NetFlow V9 packet header. This field is not present in the IPFIX > Message Header, and must be exported using an IPFIX Option if > required. > > Export Time: Aside from being called UNIX Secs in the NetFlow V9 > packet header specification, the export time in seconds since 1 > January 1970 at 0000 UTC appears in both NetFlow V9 and IPFIX > message headers. > > Sequence Number: The NetFlow V9 Sequence Number counts packets, > while the IPFIX Sequence Number counts records in Data Sets. Both > are scoped to Observation Domain. > > Observation Domain ID: Similarly, the NetFlow V9 sourceID has > become the IPFIX Observation Domain ID. > > B.1.2. Set Header Format > > Set headers are identical between NetFlow V9 and IPFIX; that is, each > Set (FlowSet in NetFlow V9 terminology) is prefixed by a 4-byte set > header containing the Set ID and the length of the set in octets. > > > > > Trammell, et al. Expires January 15, 2009 [Page 46] > > Internet-Draft IPFIX Files July 2008 > > > Note that the special Set IDs are different between IPFIX and NetFlow > V9. IPFIX Template Sets are identified by Set ID 2, while NetFlow V9 > Template FlowSets are identified by Set ID 0. Similarly, IPFIX > Options Template Sets are identified by Set ID 3, while NetFlow V9 > Options Template FlowSets are identified by Set ID 1. > > Both protocols reserve Set IDs 0-255, and use Set IDs 256-65535 for > Data Sets (or FlowSets, in NetFlow V9 terminology). > > B.1.3. Template Format > > Template FlowSets in NetFlow V9 support a subset of functionality of > those in IPFIX. Specifically, NetFlow V9 does not have any support > for vendor-specific Information Elements as IPFIX does, so there is > no enterprise bit or facility for associating a private enterprise > number with an information element. > > Options Template FlowSets in NetFlow V9 are similar to Options > Template Sets in IPFIX in the same way. > > B.1.4. Information Model > > The NetFlow V9 field type definitions are a compatible subset of, and > have evolved in concert with, the IPFIX Information Model. IPFIX > Information Element numbers in the range 1-127 are defined by the > IPFIX Information Model [RFC5102] to be compatible with the > corresponding NetFlow V9 field types. > > B.1.5. Template Management > > NetFlow V9 has no concept of a Transport Session as in IPFIX, as > NetFlow V9 was designed with a connectionless transport in mind. > Template IDs are therefore scoped to an Exporting Process lifetime > (i.e., an Exporting Process instance between restarts). There is no > facility in NetFlow V9 as in IPFIX for Template withdrawal or > Template ID reuse. Template retransmission at the Exporter works as > in UDP-based IPFIX Exporting Processes. > > B.1.6. Transport > > In practice, though NetFlow V9 is designed to be transport- > independent, it is transported only over UDP. There is no facility > as in IPFIX for full connection-oriented transport without datagram > boundaries, due to the use of a record count field as opposed to a > message length field in the packet header. There is no support in > NetFlow V9 for transport layer security via TLS or DTLS. > > > > > > Trammell, et al. Expires January 15, 2009 [Page 47] > > Internet-Draft IPFIX Files July 2008 > > > B.2. A Method for Transforming NetFlow V9 messages to IPFIX > > This appendix describes a method for transforming NetFlow V9 Packets > into IPFIX Messages, which can be used to store NetFlow V9 data in > IPFIX Files. A process transforming NetFlow V9 Packets into IPFIX > Messages must handle the fact that NetFlow V9 Packets and IPFIX > Messages are framed differently, that sequence numbering works > differently, and that the NetFlow V9 field type definitions are only > compatible with the IPFIX Information Model field and/or information > element numbers below Information Element number 128. > > For each incoming NetFlow V9 packet, the transformation process must: > > 1. Verify that the Version field of the packet header is 9. > > 2. Verify that the Sequence Number field of the packet header is > valid. > > 3. Scan the packet to: > > 1. verify that it contains no Templates with field numbers > outside the range 1-127; > > 2. verify that it contains no FlowSets with Set IDs between 2 > and 255 inclusive; > > 3. verify that it contains the number of records in FlowSets, > Template FlowSets, and Options Template FlowSets declared in > the Count field of the packet header; and > > 4. count the number of records in FlowSets for calculating the > IPFIX Sequence number. > > 4. Calculate a Sequence Number for each IPFIX Observation Domain by > storing the last Sequence Number sent for each Observation Domain > plus the count of records in FlowSets in the previous step to be > sent as the Sequence Number for the IPFIX Message within that > Observation Domain following this one. > > 5. Generate a new IPFIX Message Header with: > > 1. a Version field of 10; > > 2. a Length field with the number of octets in the IPFIX > Message, generally available by subtracting 4 from the length > of the NetFlow V9 packet as returned from the transport layer > (accounting for the difference in message header lengths); > > > > > Trammell, et al. Expires January 15, 2009 [Page 48] > > Internet-Draft IPFIX Files July 2008 > > > 3. the Sequence Number calculated for this message by the > Sequence Number calculation step; and > > 4. Export Time and Observation Domain ID taken from the UNIX > secs and Source ID fields of the NetFlow V9 packet header, > respectively. > > 6. Copy each FlowSet from the Netflow V9 packet to the IPFIX Message > after the header. Replace Set ID 0 with Set ID 2 for Template > Sets, and Set ID 1 with Set ID 3 for Options Template Sets. > > Note that this process loses system uptime information; if such > information is required, the transformation process will have to > export that information using IPFIX Options. This may require a more > sophisticated transformation process structure. > > B.3. NetFlow V9 Transformation Example > > The following two figures show a single NetFlow V9 packet with > templates and the corresponding IPFIX Message, exporting a single > flow record representing 60,303 octets sent from 192.0.2.2 to > 192.0.2.3. This would be the 3rd packet exported in Observation > Domain 33 from the NetFlow V9 exporter, containing records starting > with the 12th record (packet and record sequence numbers count from > 0). > > The ** symbol in the IPFIX example shows those fields that required > modification from the NetFlow V9 packet by the transformation > process. > > > > > > > > > > > > > > > > > > > > > > > Trammell, et al. Expires January 15, 2009 [Page 49] > > Internet-Draft IPFIX Files July 2008 > > > 1 2 3 > 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | Version = 9 | Count = 2 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | Uptime = 3750 IPFIX Files. A process transforming NetFlow V9 Packets into IPFIX > Messages must handle the fact that NetFlow V9 Packets and IPFIX > Messages are framed differently, that sequence numbering works > differently, and that the NetFlow V9 field type definitions are only > compatible with the IPFIX Information Model field and/or information > element numbers below Information Element number 128. > > For each incoming NetFlow V9 packet, the transformation process must: > > 1. Verify that the Version field of the packet header is 9. > > 2. Verify that the Sequence Number field of the packet header is > valid. > > 3. Scan the packet to: > > 1. verify that it contains no Templates with field numbers > outside the range 1-127; > > 2. verify that it contains no FlowSets with Set IDs between 2 > and 255 inclusive; > > 3. verify that it contains the number of records in FlowSets, > Template FlowSets, and Options Template FlowSets declared in > the Count field of the packet header; and > > 4. count the number of records in FlowSets for calculating the > IPFIX Sequence number. > > 4. Calculate a Sequence Number for each IPFIX Observation Domain by > storing the last Sequence Number sent for each Observation Domain > plus the count of records in FlowSets in the previous step to be > sent as the Sequence Number for the IPFIX Message within that > Observation Domain following this one. > > 5. Generate a new IPFIX Message Header with: > > 1. a Version field of 10; > > 2. a Length field with the number of octets in the IPFIX > Message, generally available by subtracting 4 from the length > of the NetFlow V9 packet as returned from the transport layer > (accounting for the difference in message header lengths); > > > > > Trammell, et al. Expires January 15, 2009 [Page 48] > > Internet-Draft IPFIX Files July 2008 > > > 3. the Sequence Number calculated for this message by the > Sequence Number calculation step; and > > 4. Export Time and Observation Domain ID taken from the UNIX > secs and Source ID fields of the NetFlow V9 packet header, > respectively. > > 6. Copy each FlowSet from the Netflow V9 packet to the IPFIX Message > after the header. Replace Set ID 0 with Set ID 2 for Template > Sets, and Set ID 1 with Set ID 3 for Options Template Sets. > > Note that this process loses system uptime information; if such > information is required, the transformation process will have to > export that information using IPFIX Options. This may require a more > sophisticated transformation process structure. > > B.3. NetFlow V9 Transformation Example > > The following two figures show a single NetFlow V9 packet with > templates and the corresponding IPFIX Message, exporting a single > flow record representing 60,303 octets sent from 192.0.2.2 to > 192.0.2.3. This would be the 3rd packet exported in Observation > Domain 33 from the NetFlow V9 exporter, containing records starting > with the 12th record (packet and record sequence numbers count from > 0). > > The ** symbol in the IPFIX example shows those fields that required > modification from the NetFlow V9 packet by the transformation > process. > > > > > > > > > > > > > > > > > > > > > > > Trammell, et al. Expires January 15, 2009 [Page 49] > > Internet-Draft IPFIX Files July 2008 > > > 1 2 3 > 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | Version = 9 | Count = 2 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | Uptime = 37405 ms (1:02:30.405) | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | Export Time = 1171557627 epoch sec (2007-02-15 16:40:27) | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | Sequence Number = 2 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | Observation Domain ID = 33 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | Set ID = 0 | Set Length = 20 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | Template ID = 256 | Field Count = 3 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | IPV4_SRC_ADDR = 8 | Field Length = 4 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | IPV4_DST_ADDR = 12 | Field Length = 4 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | IN_BYTES = 1 | Field Length = 4 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | Set ID = 256 | Set Length = 16 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | IPV4_SRC_ADDR | > | 192.0.2.2 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | IPV4_DST_ADDR | > | 192.0.2.3 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | IN_BYTES | > | 60303 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > > Figure 13: Example NetFlow V9 Packet > > > > > > > > > > > > > > > > Trammell, et al. Expires January 15, 2009 [Page 50] > > Internet-Draft IPFIX Files July 2008 > > > 1 2 3 > 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | ** Version = 10 | ** Length = 52 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | Export Time = 1171557627 epoch sec (2007-02-15 16:40:27) | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | ** Sequence Number = 11 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | Observation Domain ID = 33 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | ** Set ID = 2 | Set Length = 20 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | Template ID = 256 | Field Count = 3 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > |0| sourceIPv4Address = 8 | Field Length = 4 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > |0| destinationIPv4Address = 12 | Field Length = 4 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > |0| octetDeltaCount = 1 | Field Length = 4 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | Set ID = 256 | Set Length = 16 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | sourceIPv4Address | > | 192.0.2.2 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-50405 ms (1:02:30.405) | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | Export Time = 1171557627 epoch sec (2007-02-15 16:40:27) | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | Sequence Number = 2 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | Observation Domain ID = 33 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | Set ID = 0 | Set Length = 20 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | Template ID = 256 | Field Count = 3 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | IPV4_SRC_ADDR = 8 | Field Length = 4 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | IPV4_DST_ADDR = 12 | Field Length = 4 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | IN_BYTES = 1 | Field Length = 4 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | Set ID = 256 | Set Length = 16 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | IPV4_SRC_ADDR | > | 192.0.2.2 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | IPV4_DST_ADDR | > | 192.0.2.3 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | IN_BYTES | > | 60303 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > > Figure 13: Example NetFlow V9 Packet > > > > > > > > > > > > > > > > Trammell, et al. Expires January 15, 2009 [Page 50] > > Internet-Draft IPFIX Files July 2008 > > > 1 2 3 > 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | ** Version = 10 | ** Length = 52 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | Export Time = 1171557627 epoch sec (2007-02-15 16:40:27) | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | ** Sequence Number = 11 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | Observation Domain ID = 33 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | ** Set ID = 2 | Set Length = 20 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | Template ID = 256 | Field Count = 3 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > |0| sourceIPv4Address = 8 | Field Length = 4 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > |0| destinationIPv4Address = 12 | Field Length = 4 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > |0| octetDeltaCount = 1 | Field Length = 4 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | Set ID = 256 | Set Length = 16 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | sourceIPv4Address | > | 192.0.2.2 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | destinationIPv4Address | > | 192.0.2.3 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | octetDeltaCount | > | 60303 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > > Figure 14: Corresponding Example IPFIX Message > > > Authors' Addresses > > Brian Trammell > Hitachi Europe > c/o ETH Zurich > Gloriastrasse 35 > 8092 Zurich > Switzerland > > Phone: +41 44 632 70 13 > Email: brian.trammell@hitachi-eu.com > > > > > Trammell, et al. Expires January 15, 2009 [Page 51] > > Internet-Draft IPFIX Files July 2008 > > > Elisa Boschi > Hitachi Europe > c/o ETH Zurich > Gloriastrasse 35 > 8092 Zurich > Switzerland > > Phone: +41 44 632 70 57 > Email: elisa.boschi@hitachi-eu.com > > > Lutz Mark > Fraunhofer IFAM > Weiner Str. 12 > 38259 Bremen > Germany > > Phone: +49 421 2246206 > Email: lutz.mark@ifam.fraunhofer.de > > > Tanja Zseby > Fraunhofer Institute for Open Communication Systems > Kaiserin-Augusta-Allee 31 > 10589 Berlin > Germany > > Phone: +49 30 3463 7153 > Email: tanja.zseby@fokus.fraunhofer.de > > > Arno Wagner > Swiss Federal Institute of Technology Zurich > Gloriastrasse 35 > 8092 Zurich > Switzerland > > Phone: +41 44 632 70 04 > Email: arno@wagner.name > > > > > > > > > > > > > Trammell, et al. Expires January 15, 2009 [Page 52] > > Internet-Draft IPFIX Files July 2008 > > > Full Copyright Statement > > Copyright (C) The IETF Trust (2008). > > This document is subject to the rights, licenses and restrictions > contained in BCP 78, and except as set forth therein, the authors > retain all their rights. > > This document and the information contained herein are provided on an > "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS > OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY, THE IETF TRUST AND > THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS > OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF > THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED > WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. > > > Intellectual Property > > The IETF takes no position regarding the validity or scope of any > Intellectual Property Rights or other rights that might be claimed to > pertain to the implementation or use of the technology described in > this document or the extent to which any license under such rights > might or might not be available; nor does it represent that it has > made any independent effort to identify any such rights. Information > on the procedures with respect to rights in RFC documents can be > found in BCP 78 and BCP 79. > > Copies of IPR disclosures made to the IETF Secretariat and any > assurances of licenses to be made available, or the result of an > attempt made to obtain a general license or permission for the use of > such proprietary rights by implementers or users of this > specification can be obtained from the IETF on-line IPR repository at > http://www.ietf.org/ipr. > > The IETF invites any interested party to bring to its attention any > copyrights, patents or patent applications, or other proprietary > rights that may cover technology that may be required to implement > this standard. Please address the information to the IETF at > ietf-ipr@ietf.org. > > > > > > > > > > > > Trammell, et al. Expires January 15, 2009 [Page 53] > > > --------------050305070204080904090002 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit | destinationIPv4Address | > | 192.0.2.3 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | octetDeltaCount | > | 60303 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > > Figure 14: Corresponding Example IPFIX Message > > > Authors' Addresses > > Brian Trammell > Hitachi Europe > c/o ETH Zurich > Gloriastrasse 35 > 8092 Zurich > Switzerland > > Phone: +41 44 632 70 13 > Email: brian.trammell@hitachi-eu.com > > > > > Trammell, et al. Expires January 15, 2009 [Page 51] > > Internet-Draft IPFIX Files July 2008 > > > Elisa Boschi > Hitachi Europe > c/o ETH Zurich > Gloriastrasse 35 > 8092 Zurich > Switzerland > > Phone: +41 44 632 70 57 > Email: elisa.boschi@hitachi-eu.com > > > Lutz Mark > Fraunhofer IFAM > Weiner Str. 12 > 38259 Bremen > Germany > > Phone: +49 421 2246206 > Email: lutz.mark@ifam.fraunhofer.de > > > Tanja Zseby > Fraunhofer Institute for Open Communication Systems > Kaiserin-Augusta-Allee 31 > 10589 Berlin > Germany > > Phone: +49 30 3463 7153 > Email: tanja.zseby@fokus.fraunhofer.de > > > Arno Wagner > Swiss Federal Institute of Technology Zurich > Gloriastrasse 35 > 8092 Zurich > Switzerland > > Phone: +41 44 632 70 04 > Email: arno@wagner.name > > > > > > > > > > > > > Trammell, et al. Expires January 15, 2009 [Page 52] > > Internet-Draft IPFIX Files July 2008 > > > Full Copyright Statement > > Copyright (C) The IETF Trust (2008). > > This document is subject to the rights, licenses and restrictions > contained in BCP 78, and except as set forth therein, the authors > retain all their rights. > > This document and the information contained herein are provided on an > "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS > OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY, THE IETF TRUST AND > THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS > OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF > THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED > WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. > > > Intellectual Property > > The IETF takes no position regarding the validity or scope of any > Intellectual Property Rights or other rights that might be claimed to > pertain to the implementation or use of the technology described in > this document or the extent to which any license under such rights > might or might not be available; nor does it represent that it has > made any independent effort to identify any such rights. Information > on the procedures with respect to rights in RFC documents can be > found in BCP 78 and BCP 79. > > Copies of IPR disclosures made to the IETF Secretariat and any > assurances of licenses to be made available, or the result of an > attempt made to obtain a general license or permission for the use of > such proprietary rights by implementers or users of this > specification can be obtained from the IETF on-line IPR repository at > http://www.ietf.org/ipr. > > The IETF invites any interested party to bring to its attention any > copyrights, patents or patent applications, or other proprietary > rights that may cover technology that may be required to implement > this standard. Please address the information to the IETF at > ietf-ipr@ietf.org. > > > > > > > > > > > > Trammell, et al. Expires January 15, 2009 [Page 53] > > > --------------050305070204080904090002 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Dear all,

Here is my feedback.
In preparation for this review, I read first the email from Paul and Gerhard. I will try not to repeat their feedback.

Regards, Benoit.

 




IPFIX Working Group                                          B. Trammell
Internet-Draft                                                 E. Boschi
Intended status: Standards Track                          Hitachi Europe
Expires: January 15, 2009                                        L. Mark
                                                         Fraunhofer IFAM
                                                                T. Zseby
                                                        Fraunhofer FOKUS
                                                               A. Wagner
                                                              ETH Zurich
                                                           July 14, 2008


                       An IPFIX-Based File Format
Add "Specifications" ?
                      draft-ietf-ipfix-file-02.txt

Status of this Memo

   By submitting this Internet-Draft, each author represents that any
   applicable patent or other IPR claims of which he or she is aware
   have been or will be disclosed, and any of which he or she becomes
   aware will be disclosed, in accordance with Section 6 of BCP 79.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF), its areas, and its working groups.  Note that
   other groups may also distribute working documents as Internet-
   Drafts.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   The list of current Internet-Drafts can be accessed at
   http://www.ietf.org/ietf/1id-abstracts.txt.

   The list of Internet-Draft Shadow Directories can be accessed at
   http://www.ietf.org/shadow.html.

   This Internet-Draft will expire on January 15, 2009.

Abstract

   This document describes a file format for the storage of flow data
   based upon the IPFIX Message format.  It proposes a set of
   requirements for flat-file, binary flow data file formats, then
   applies the IPFIX message format to these requirements to build a new
Message
   file format.  This IPFIX-based file format is designed to facilitate
   interoperability and reusability among a wide variety of flow



Trammell, et al.        Expires January 15, 2009                [Page 1]

Internet-Draft                 IPFIX Files                     July 2008


   storage, processing, and analysis tools.


Table of Contents

   1.  Introduction . . . . . . . . . . . . . . . . . . . . . . . . .  4
     1.1.  IPFIX Documents Overview . . . . . . . . . . . . . . . . .  4
   2.  Terminology  . . . . . . . . . . . . . . . . . . . . . . . . .  5
   3.  Design Overview  . . . . . . . . . . . . . . . . . . . . . . .  6
   4.  Motivation . . . . . . . . . . . . . . . . . . . . . . . . . .  7
   5.  Requirements . . . . . . . . . . . . . . . .C "-//W3C//DTD HTML 4.01 Transitional//EN">


  


Dear all,



Here is my feedback.

In preparation for this review, I read first the email from Paul and
Gerhard. I will try not to repeat their feedback.



Regards, Benoit.



  
  
  

  
  




IPFIX Working Group                                          B. Trammell
Internet-Draft                                                 E. Boschi
Intended status: Standards Track                          Hitachi Europe
Expires: January 15, 2009                                        L. Mark
                                                         Fraunhofer IFAM
                                                                T. Zseby
                                                        Fraunhofer FOKUS
                                                               A. Wagner
                                                              ETH Zurich
                                                           July 14, 2008


                       An IPFIX-Based File Format
  



Add "Specifications" ?



  
                      draft-ietf-ipfix-file-02.txt

Status of this Memo

   By submitting this Internet-Draft, each author represents that any
   applicable patent or other IPR claims of which he or she is aware
   have been or will be disclosed, and any of which he or she becomes
   aware will be disclosed, in accordance with Section 6 of BCP 79.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF), its areas, and its working groups.  Note that
   other groups may also distribute working documents as Internet-
   Drafts.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   The list of current Internet-Drafts can be accessed at
   http://www.ietf.org/ietf/1id-abstracts.txt.

   The list of Internet-Draft Shadow Directories can be accessed at
   http://www.ietf.org/shadow.html.

   This Internet-Draft will expire on January 15, 2009.

Abstract

   This document describes a file format for the storage of flow data
   based upon the IPFIX Message format.  It proposes a set of
   requirements for flat-file, binary flow data file formats, then
   applies the IPFIX message format to these requirements to build a new
  



Message



  
   file format.  This IPFIX-based file format is designed to facilitate
   interoperability and reusability among a wide variety of flow



Trammell, et al.        Expires January 15, 2009                [Page 1]

Internet-Draft                 IPFIX Files                     July 2008


   storage, processing, and analysis tools.


Table of Contents

   1.  Introduction . . . . . . . . . . . . . . . . . . . . . . . . .  4
     1.1.  IPFIX Documents Overview . . . . . . . . . . . . . . . . .  4
   2.  Terminology  . . . . . . . . . . . . . . . . . . . . . . . . .  5
   3.  Design Overview  . . . . . . . . . . . . . . . . . . . . . . .  6
   4.  Motivation . . . . . . . . . . . . . . . . . . . . . . . . . .  7
   5.  Requirements . . . . . . . . . . . . . . . . . . . . . . . .  9
     5.1.  Record Format Flexibility  . . . . . . . . . . . . . . . . 10
     5.2.  Self Description . . . . . . . . . . . . . . . . . . . . . 10
     5.3.  Data Compression . . . . . . . . . . . . . . . . . . . . . 11
     5.4.  Indexing and Searching . . . . . . . . . . . . . . . . . . 11
     5.5.  Data Integrity . . . . . . . . . . . . . . . . . . . . . . 12
     5.6.  Creator Authentication and Confidentiality . . . . . . . . 12
     5.7.  Anonymization and Obfuscation  . . . . . . . . . . . . . . 13
     5.8.  Session Auditability and Replayability . . . . . . . . . . 13
     5.9.  Performance Characteristics  . . . . . . . . . . . . . . . 14
   6.  Applicability  . . . . . . . . . . . . . . . . . . . . . . . . 14
     6.1.  Storage of IPFIX-collected Flow Data . . . . . . . . . . . 14
     6.2.  Storage of NetFlow V9-collected Flow Data  . . . . . . . . 15
     6.3.  Testing IPFIX Collecting Processes . . . . . . . . . . . . 15
     6.4.  IPFIX Device Diagnostics . . . . . . . . . . . . . . . . . 15
   7.  Detailed Description . . . . . . . . . . . . . . . . . . . . . 16
  



Specifications?



  
     7.1.  File Reader Specification  . . . . . . . . . . . . . . . . 16
     7.2.  File Writer Specification  . . . . . . . . . . . . . . . . 17
     7.3.  Specific File Writer Use Cases . . . . . . . . . . . . . . 18
       7.3.1.  Collocating a File Writer with a Collecting Process  . 18
       7.3.2.  Collocating a File Writer with a Metering Process  . . 19
       7.3.3.  Using IPFIX Files for Archival Storage . . . . . . . . 19
       7.3.4.  Using IPFIX Files as Documents . . . . . . . . . . . . 19
       7.3.5.  Using IPFIX Files for Testing  . . . . . . . . . . . . 20
       7.3.6.  Writing IPFIX Files for Device Diagnostics . . . . . . 21
   8.  File Format Metadata Specification . . . . . . . . . . . . . . 21
  





  
     8.1.  Recommended Options Templates for IPFIX Files  . . . . . . 21
       8.1.1.  Message Checksum Options Template  . . . . . . . . . . 21
       8.1.2.  File Time Window Options Template  . . . . . . . . . . 22
       8.1.3.  Export Session Details Options Template  . . . . . . . 22
       8.1.4.  Message Details Options Template . . . . . . . . . . . 24
     8.2.  Recommended Information Elements for IPFIX Files . . . . . 26
       8.2.1.  collectionTimeMilliseconds . . . . . . . . . . . . . . 26
       8.2.2.  exportSctpStreamId . . . . . . . . . . . . . . . . . . 27
       8.2.3.  maxExportSeconds . . . . . . . . . . . . . . . . . . . 27
       8.2.4.  maxFlowEndSeconds  . . . . . . . . . . . . . . . . . . 28
       8.2.5.  messageMD5Checksum . . . . . . . . . . . . . . . . . . 28
       8.2.6.  messageScope . . . . . . . . . . . . . . . . . . . . . 28



Trammell, et al.        Expires January 15, 2009                [Page 2]

Internet-Draft                 IPFIX Files                     July 2008


       8.2.7.  minExportSeconds . . . . . . . . . . . . . . . . . . . 29
       8.2.8.  minFlowStartSeconds  . . . . . . . . . . . . . . . . . 29
       8.2.9.  opaqueOctets . . . . . . . . . . . . . . . . . . . . . 30
       8.2.10. sessionScope . . . . . . . . . . . . . . . . . . . . . 30
   9.  Recommended Error Resilience Strategies  . . . . . . . . . . . 30
     9.1.  Compression Error Resilience . . . . . . . . . . . . . . . 31
     9.2.  Encryption Error Resilience  . . . . . . . . . . . . . . . 32
   10. Recommended File Integration Strategies  . . . . . . . . . . . 32
     10.1. Encapsulation of Non-IPFIX Data in IPFIX Files . . . . . . 32
     10.2. Encapsulation of IPFIX Files within Other File Formats . . 33
   11. Security Considerations  . . . . . . . . . . . . . . . . . . . 33
   12. IANA Considerations  . . . . . . . . . . . . . . . . . . . . . 34
   13. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 35
   14. References . . . . . . . . . . . . . . . . . . . . . . . . . . .  9
     5.1.  Record Format Flexibility  . . . . . . . . . . . . . . . . 10
     5.2.  Self Description . . . . . . . . . . . . . . . . . . . . . 10
     5.3.  Data Compression . . . . . . . . . . . . . . . . . . . . . 11
     5.4.  Indexing and Searching . . . . . . . . . . . . . . . . . . 11
     5.5.  Data Integrity . . . . . . . . . . . . . . . . . . . . . . 12
     5.6.  Creator Authentication and Confidentiality . . . . . . . . 12
     5.7.  Anonymization and Obfuscation  . . . . . . . . . . . . . . 13
     5.8.  Session Auditability and Replayability . . . . . . . . . . 13
     5.9.  Performance Characteristics  . . . . . . . . . . . . . . . 14
   6.  Applicability  . . . . . . . . . . . . . . . . . . . . . . . . 14
     6.1.  Storage of IPFIX-collected Flow Data . . . . . . . . . . . 14
     6.2.  Storage of NetFlow V9-collected Flow Data  . . . . . . . . 15
     6.3.  Testing IPFIX Collecting Processes . . . . . . . . . . . . 15
     6.4.  IPFIX Device Diagnostics . . . . . . . . . . . . . . . . . 15
   7.  Detailed Description . . . . . . . . . . . . . . . . . . . . . 16
  



Specifications?



  
     7.1.  File Reader Specification  . . . . . . . . . . . . . . . . 16
     7.2.  File Writer Specification  . . . . . . . . . . . . . . . . 17
     7.3.  Specific File Writer Use Cases . . . . . . . . . . . . . . 18
       7.3.1.  Collocating a File Writer with a Collecting Process  . 18
       7.3.2.  Collocating a File Writer with a Metering Process  . . 19
       7.3.3.  Using IPFIX Files for Archival Storage . . . . . . . . 19
       7.3.4.  Using IPFIX Files as Documents . . . . . . . . . . . . 19
       7.3.5.  Using IPFIX Files for Testing  . . . . . . . . . . . . 20
       7.3.6.  Writing IPFIX Files for Device Diagnostics . . . . . . 21
   8.  File Format Metadata Specification . . . . . . . . . . . . . . 21
  





  
     8.1.  Recommended Options Templates for IPFIX Files  . . . . . . 21
       8.1.1.  Message Checksum Options Template  . . . . . . . . . . 21
       8.1.2.  File Time Window Options Template  . . . . . . . . . . 22
       8.1.3.  Export Session Details Options Template  . . . . . . . 22
       8.1.4.  Message Details Options Template . . . . . . . . . . . 24
     8.2.  Recommended Information Elements for IPFIX Files . . . . . 26
       8.2.1.  collectionTimeMilliseconds . . . . . . . . . . . . . . 26
       8.2.2.  exportSctpStreamId . . . . . . . . . . . . . . . . . . 27
       8.2.3.  maxExportSeconds . . . . . . . . . . . . . . . . . . . 27
       8.2.4.  maxFlowEndSeconds  . . . . . . . . . . . . . . . . . . 28
       8.2.5.  messageMD5Checksum . . . . . . . . . . . . . . . . . . 28
       8.2.6.  messageScope . . . . . . . . . . . . . . . . . . . . . 28



Trammell, et al.        Expires January 15, 2009                [Page 2]

Internet-Draft                 IPFIX Files                     July 2008


       8.2.7.  minExportSeconds . . . . . . . . . . . . . . . . . . . 29
       8.2.8.  minFlowStartSeconds  . . . . . . . . . . . . . . . . . 29
       8.2.9.  opaqueOctets . . . . . . . . . . . . . . . . . . . . . 30
       8.2.10. sessionScope . . . . . . . . . . . . . . . . . . . . . 30
   9.  Recommended Error Resilience Strategies  . . . . . . . . . . . 30
     9.1.  Compression Error Resilience . . . . . . . . . . . . . . . 31
     9.2.  Encryption Error Resilience  . . . . . . . . . . . . . . . 32
   10. Recommended File Integration Strategies  . . . . . . . . . . . 32
     10.1. Encapsulation of Non-IPFIX Data in IPFIX Files . . . . . . 32
     10.2. Encapsulation of IPFIX Files within Other File Formats . . 33
   11. Security Considerations  . . . . . . . . . . . . . . . . . . . 33
   12. IANA Considerations  . . . . . . . . . . . . . . . . . . . . . 34
   13. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 35
   14. References . . . . . . . . . . . . . . . . . . . . . . . . . 35
     14.1. Normative References . . . . . . . . . . . . . . . . . . . 35
     14.2. Informative References . . . . . . . . . . . . . . . . . . 35
   Appendix A.  Example IPFIX File  . . . . . . . . . . . . . . . . . 36
     A.1.  Example Options Templates  . . . . . . . . . . . . . . . . 38
     A.2.  Example Supplemental Options Data  . . . . . . . . . . . . 40
     A.3.  Example Message Checksum . . . . . . . . . . . . . . . . . 42
     A.4.  File Example Data Set  . . . . . . . . . . . . . . . . . . 43
     A.5.  Complete File Example  . . . . . . . . . . . . . . . . . . 43
   Appendix B.  Applicability of IPFIX Files to NetFlow V9 flow
                storage . . . . . . . . . . . . . . . . . . . . . . . 45
     B.1.  Comparing NetFlow V9 to IPFIX  . . . . . . . . . . . . . . 45
       B.1.1.  Message Header Format  . . . . . . . . . . . . . . . . 45
       B.1.2.  Set Header Format  . . . . . . . . . . . . . . . . . . 46
       B.1.3.  Template Format  . . . . . . . . . . . . . . . . . . . 47
       B.1.4.  Information Model  . . . . . . . . . . . . . . . . . . 47
       B.1.5.  Template Management  . . . . . . . . . . . . . . . . . 47
       B.1.6.  Transport  . . . . . . . . . . . . . . . . . . . . . . 47
     B.2.  A Method for Transforming NetFlow V9 messages to IPFIX . . 48
     B.3.  NetFlow V9 Transformation Example  . . . . . . . . . . . . 49
   Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 51
   Intellectual Property and Copyright Statements . . . . . . . . . . 53
















Trammell, et al.        Expires January 15, 2009                [Page 3]

Internet-Draft                 IPFIX Files                     July 2008


1.  Introduction

   This document proposes a file format based upon IPFIX, designed to
   facilitiate interoperability and reusability among a wide variety of
   flow storage, processing, and analysis tools.  It begins with an
   overview of the IPFIX File format, and a quick summary of how IPFIX
  



"IPFIX-based file format" in the abstract. " IPFIX File format" here.  



  
   Files work in Section 3.  It then explores the motivation for
   proposing a standardized flow file format and using IPFIX as the
   basis for this new file format in Section 4.  It outlines a set of
   requirements for standardized flow storage in Section 5, and explores
   the applicability of such a format to various specific application
   areas in Section 6.

   The file format specification follows in Section 7, including
  



And new simply "file format". Maybe I should wait for the definition ;-)



  
   specifications of readers and writers of these files, and additional
   specifications that apply in specific use cases.  This format makes
   use of the IPFIX Options mechanism for additional file metadata, in
   order to avoid requiring any protocol or message format extensions,
   and to minimize the effort required to adapt IPFIX implementations to
   use the file format; a detailed definition of the Options Templates
   used for storage metedata appears in Section 8.

   Section 9 and Section 10 provide specific recommendations for error
   resilience during long-term storage and integration of IPFIX File
   data with other formats.  Appendix A contains a detailed example
   IPFIX File.

1.1.  IPFIX Documents Overview

   "Specification of the IPFIX Protocol for the Exchange of IP Traffic
   Flow Information" [RFC5101] and its associated documents define the
   IPFIX Protocol, which provides network engineers and administrators
   with access to IP traffic flow information.

   "Architecture for IP Flow Information Export" [I-D.ietf-ipfix-arch]
   defines the architecture for the export of measured IP flow
   information out of an IPFIX Exporting Process to an IPFIX Collecting
   Process, and the basic terminology used to describe the elements of
   this architect . . . . . . . . . . 35
     14.1. Normative References . . . . . . . . . . . . . . . . . . . 35
     14.2. Informative References . . . . . . . . . . . . . . . . . . 35
   Appendix A.  Example IPFIX File  . . . . . . . . . . . . . . . . . 36
     A.1.  Example Options Templates  . . . . . . . . . . . . . . . . 38
     A.2.  Example Supplemental Options Data  . . . . . . . . . . . . 40
     A.3.  Example Message Checksum . . . . . . . . . . . . . . . . . 42
     A.4.  File Example Data Set  . . . . . . . . . . . . . . . . . . 43
     A.5.  Complete File Example  . . . . . . . . . . . . . . . . . . 43
   Appendix B.  Applicability of IPFIX Files to NetFlow V9 flow
                storage . . . . . . . . . . . . . . . . . . . . . . . 45
     B.1.  Comparing NetFlow V9 to IPFIX  . . . . . . . . . . . . . . 45
       B.1.1.  Message Header Format  . . . . . . . . . . . . . . . . 45
       B.1.2.  Set Header Format  . . . . . . . . . . . . . . . . . . 46
       B.1.3.  Template Format  . . . . . . . . . . . . . . . . . . . 47
       B.1.4.  Information Model  . . . . . . . . . . . . . . . . . . 47
       B.1.5.  Template Management  . . . . . . . . . . . . . . . . . 47
       B.1.6.  Transport  . . . . . . . . . . . . . . . . . . . . . . 47
     B.2.  A Method for Transforming NetFlow V9 messages to IPFIX . . 48
     B.3.  NetFlow V9 Transformation Example  . . . . . . . . . . . . 49
   Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 51
   Intellectual Property and Copyright Statements . . . . . . . . . . 53
















Trammell, et al.        Expires January 15, 2009                [Page 3]

Internet-Draft                 IPFIX Files                     July 2008


1.  Introduction

   This document proposes a file format based upon IPFIX, designed to
   facilitiate interoperability and reusability among a wide variety of
   flow storage, processing, and analysis tools.  It begins with an
   overview of the IPFIX File format, and a quick summary of how IPFIX
  



"IPFIX-based file format" in the abstract. " IPFIX File format" here.  



  
   Files work in Section 3.  It then explores the motivation for
   proposing a standardized flow file format and using IPFIX as the
   basis for this new file format in Section 4.  It outlines a set of
   requirements for standardized flow storage in Section 5, and explores
   the applicability of such a format to various specific application
   areas in Section 6.

   The file format specification follows in Section 7, including
  



And new simply "file format". Maybe I should wait for the definition ;-)



  
   specifications of readers and writers of these files, and additional
   specifications that apply in specific use cases.  This format makes
   use of the IPFIX Options mechanism for additional file metadata, in
   order to avoid requiring any protocol or message format extensions,
   and to minimize the effort required to adapt IPFIX implementations to
   use the file format; a detailed definition of the Options Templates
   used for storage metedata appears in Section 8.

   Section 9 and Section 10 provide specific recommendations for error
   resilience during long-term storage and integration of IPFIX File
   data with other formats.  Appendix A contains a detailed example
   IPFIX File.

1.1.  IPFIX Documents Overview

   "Specification of the IPFIX Protocol for the Exchange of IP Traffic
   Flow Information" [RFC5101] and its associated documents define the
   IPFIX Protocol, which provides network engineers and administrators
   with access to IP traffic flow information.

   "Architecture for IP Flow Information Export" [I-D.ietf-ipfix-arch]
   defines the architecture for the export of measured IP flow
   information out of an IPFIX Exporting Process to an IPFIX Collecting
   Process, and the basic terminology used to describe the elements of
   this architeure, per the requirements defined in "Requirements for
   IP Flow Information Export" [RFC3917].  [RFC5101] then covers the
   details of the method for transporting IPFIX Data Records and
   Templates via a congestion-aware transport protocol from an IPFIX
   Exporting Process to an IPFIX Collecting Process.

   "Information Model for IP Flow Information Export" [RFC5102]
   describes the Information Elements used by IPFIX, including details
   on Information Element naming, numbering, and data type encoding.
  



<CR>?





  
   Finally, "IPFIX Applicability" [I-D.ietf-ipfix-as] describes the
  



Why start with "Finally" while the next section starts with "in
addition"?



  


Trammell, et al.        Expires January 15, 2009                [Page 4]

Internet-Draft                 IPFIX Files                     July 2008


   various applications of the IPFIX protocol and their use of
   information exported via IPFIX, and relates the IPFIX architecture to
   other measurement architectures and frameworks.

   In addition, "Exporting Type Information for IPFIX Information
   Elements" [I-D.ietf-ipfix-exporting-type] specifies a method for
   encoding Information Model properties within an IPFIX Message stream.

   This document references [RFC5101] and the architecture document for
   terminology, defines IPFIX File Writer and IPFIX File Reader in terms
   of the IPFIX Exporting Processes and IPFIX Collecting Process
   definitions from [RFC5101], and extends the IPFIX Information Model
   defined in [RFC5102] to provide new Information Elements for IPFIX
   File metadata.  



I've seen several instances of "metadata". This is a confusing generic
term to me. Maybe it's only me?



  
It uses the method described in "Exporting Type
   Information for IPFIX Information Elements"
   [I-D.ietf-ipfix-exporting-type] document to support the self-
   description of IPFIX Files containing enterprise-specific Information
   Elements.
  



Minor point. The last paragraphs contains interesting information.
Maybe they're not at the right place in the section "IPFIX Documents
Overview"


  

2.  Terminology

   Terms used in this document that are defined in the Terminology
   section of [RFC5101] are to be interpreted as defined there.

   IPFIX File:   An IPFIX File is a serialized stream of IPFIX Messages
      stored on a filesystem.  Any IPFIX Message stream that would be
      considered valid when transported one or more of the specified
      IPFIX transports (SCTP, TCP, or UDP) as defined in [RFC5101] is
      considered an IPFIX File for purposes of this document; however,
      this document extends that definition with recommendations on the
      construction of IPFIX Files that meet the requirements identified
      herein.
  



Your first sentence says: IPFIX file = stream of IPFIX Messages stored
on a filesystem

Your second sentence says: IPFIX file = IPFIX Message stream 

There is a discrepancy.



  
   IPFIX File Reader:   An IPFIX File Reader is a Process which reads
      IPFIX Files from a filesystem, and is analogous to an IPFIX
      Collecting Process.  An IPFIX File Reader MUST behave as an IPFIX
      Collecting Process as outlined in [RFC5101], except as modified by
      this document.
  



Not sure if this is adequate to include sentences with MUST with a
definition.

What about?


   IPFIX File Reader:   An IPFIX File Reader is a Process which reads
      IPFIX Files from a filesystem. An IPFIX File Reader is analogous to an IPFIX
      Collecting Process.
<CR>?

   Finally, "IPFIX Applicability" [I-D.ietf-ipfix-as] describes the
Why start with "Finally" while the next section starts with "in addition"?


Trammell, et al.        Expires January 15, 2009                [Page 4]

Internet-Draft                 IPFIX Files                     July 2008


   various applications of the IPFIX protocol and their use of
   information exported via IPFIX, and relates the IPFIX architecture to
   other measurement architectures and frameworks.

   In addition, "Exporting Type Information for IPFIX Information
   Elements" [I-D.ietf-ipfix-exporting-type] specifies a method for
   encoding Information Model properties within an IPFIX Message stream.

   This document references [RFC5101] and the architecture document for
   terminology, defines IPFIX File Writer and IPFIX File Reader in terms
   of the IPFIX Exporting Processes and IPFIX Collecting Process
   definitions from [RFC5101], and extends the IPFIX Information Model
   defined in [RFC5102] to provide new Information Elements for IPFIX
   File metadata.
I've seen several instances of "metadata". This is a confusing generic term to me. Maybe it's only me?
It uses the method described in "Exporting Type
   Information for IPFIX Information Elements"
   [I-D.ietf-ipfix-exporting-type] document to support the self-
   description of IPFIX Files containing enterprise-specific Information
   Elements.
Minor point. The last paragraphs contains interesting information. Maybe they're not at the right place in the section "IPFIX Documents Overview" 

2.  Terminology

   Terms used in this document that are defined in the Terminology
   section of [RFC5101] are to be interpreted as defined there.

   IPFIX File:   An IPFIX File is a serialized stream of IPFIX Messages
      stored on a filesystem.  Any IPFIX Message stream that would be
      considered valid when transported one or more of the specified
      IPFIX transports (SCTP, TCP, or UDP) as defined in [RFC5101] is
      considered an IPFIX File for purposes of this document; however,
      this document extends that definition with recommendations on the
      construction of IPFIX Files that meet the requirements identified
      herein.
Your first sentence says: IPFIX file = stream of IPFIX Messages stored on a filesystem
Your second sentence says: IPFIX file = IPFIX Message stream
There is a discrepancy.
   IPFIX File Reader:   An IPFIX File Reader is a Process which reads
      IPFIX Files from a filesystem, and is analogous to an IPFIX
      Collecting Process.  An IPFIX File Reader MUST behave as an IPFIX
      Collecting Process as outlined in [RFC5101], except as modified by
      this document.
Not sure if this is adequate to include sentences with MUST with a definition.
What about?
   IPFIX File Reader:   An IPFIX File Reader is a Process which reads
      IPFIX Files from a filesystem. An IPFIX File Reader is analogous to an IPFIX
      Collecting Process.  


   IPFIX File Writer:   An IPFIX File Writer is a process which writes
      IPFIX Files to a filesystem, and is analogous to an IPFIX
      Exporting Process.  An IPFIX File Writer MUST behave as an IPFIX
      Exporting Process as outlined in [RFC5101], except as modified by
      this document.
Ditto.
   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
   "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this



Trammell, et al.        Expires January 15, 2009                [Page 5]

Internet-Draft                 IPFIX Files                     July 2008


   document are to be interpreted as described in [RFC2119].


3.  Design Overview

   An IPFIX File, as defined by this document, is simply a stream
remove "as defined by this document" 
   containing one or more IPFIX Messages serialized to some filesystem.
   Though any set of valid IPFIX Messages can be serialized into an
   IPFIX File, the specification proposes guidelines designed to ease
   storage and retrieval of flow data using the format.
which format?
   IPFIX Files contain only IPFIX Messages; any file metadata such as
   checksums or export session details are stored using Options within
   the IPFIX Message.
I guess that your message is: any file metadata must be encoded in an IPFIX Message.
However, existing ones must be modified or new one created?
This design has several advantages, including
   complete compatibility with the IPFIX Protocol on the wire and free
   manipulability of IPFIX Files through concatenation, appending, and
   splitting (on IPFIX Message boundaries).  A schematic of a typical
   file is shown below:

































Trammell, et al.        Expires January 15, 2009                [Page 6]

Internet-Draft                 IPFIX Files                     July 2008


             +=======================================+
             | IPFIX File                            |
             | +===================================+ |
             | | IPFIX Message                     | |
             | | +-------------------------------+ | |
             | | | Options Template Set          | | |
             | | |   Options Template Record     | | |
             | | |           . . .               | | |
             | | +-------------------------------+ | |
             | | +-------------------------------+ | |
             | | | Template Set                  | | |
             | | |   Template Record             | | |
             | | |            . . .              | | |
             | | +-------------------------------+ | |
             | +===================================+ |
             | | IPFIX Message                     | |
             | | +-------------------------------+ | |
             | | | Data Set                      | | |
             | | |   Data Record                 | | |
             | | |            . . .              | | |
             | | +-------------------------------+ | |
             | | +-------------------------------+ | |
             | | | Data Set                      | | |
             | | |   Data Record                 | | |
             | | |            . . .              | | |
             | | +-------------------------------+ | |
             | |              . . .                | |
             | +===================================+ |
             |                . . .                  |
             +=======================================+

                     Figure 1: Typical File Structure
I had to search to see whether or not kquote cite="mid48B3D434.9030309@cisco.com" type="cite"> 
   IPFIX File Writer:   An IPFIX File Writer is a process which writes
      IPFIX Files to a filesystem, and is analogous to an IPFIX
      Exporting Process.  An IPFIX File Writer MUST behave as an IPFIX
      Exporting Process as outlined in [RFC5101], except as modified by
      this document.
Ditto.
   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
   "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this



Trammell, et al.        Expires January 15, 2009                [Page 5]

Internet-Draft                 IPFIX Files                     July 2008


   document are to be interpreted as described in [RFC2119].


3.  Design Overview

   An IPFIX File, as defined by this document, is simply a stream
remove "as defined by this document" 
   containing one or more IPFIX Messages serialized to some filesystem.
   Though any set of valid IPFIX Messages can be serialized into an
   IPFIX File, the specification proposes guidelines designed to ease
   storage and retrieval of flow data using the format.
which format?
   IPFIX Files contain only IPFIX Messages; any file metadata such as
   checksums or export session details are stored using Options within
   the IPFIX Message.
I guess that your message is: any file metadata must be encoded in an IPFIX Message.
However, existing ones must be modified or new one created?
This design has several advantages, including
   complete compatibility with the IPFIX Protocol on the wire and free
   manipulability of IPFIX Files through concatenation, appending, and
   splitting (on IPFIX Message boundaries).  A schematic of a typical
   file is shown below:

































Trammell, et al.        Expires January 15, 2009                [Page 6]

Internet-Draft                 IPFIX Files                     July 2008


             +=======================================+
             | IPFIX File                            |
             | +===================================+ |
             | | IPFIX Message                     | |
             | | +-------------------------------+ | |
             | | | Options Template Set          | | |
             | | |   Options Template Record     | | |
             | | |           . . .               | | |
             | | +-------------------------------+ | |
             | | +-------------------------------+ | |
             | | | Template Set                  | | |
             | | |   Template Record             | | |
             | | |            . . .              | | |
             | | +-------------------------------+ | |
             | +===================================+ |
             | | IPFIX Message                     | |
             | | +-------------------------------+ | |
             | | | Data Set                      | | |
             | | |   Data Record                 | | |
             | | |            . . .              | | |
             | | +-------------------------------+ | |
             | | +-------------------------------+ | |
             | | | Data Set                      | | |
             | | |   Data Record                 | | |
             | | |            . . .              | | |
             | | +-------------------------------+ | |
             | |              . . .                | |
             | +===================================+ |
             |                . . .                  |
             +=======================================+

                     Figure 1: Typical File Structure
I had to search to see whether or nothe IPFIX Message contains the IPFIX Message Header.
Maybe one extra sentence would be welcome...
>From the definition only, it's not obvious.
   IPFIX Message

      An IPFIX Message is a message originating at the Exporting Process
      that carries the IPFIX records of this Exporting Process and whose
      destination is a Collecting Process.  An IPFIX Message is
      encapsulated at the transport layer.
We have to look at figure B of RFC5101.

   See Section 7 for details of the implementation of this design,
   including specific requirements and guidelines for File Readers and
   File Writers, and Information Elements and Options Templates used for
   file metadata.
I would remove this paragraph. Already covered.

4.  Motivation

   There are a wide variety of applications for the file-based storage
There is 
   of IP flow data, across a continuum of time scales.  Tools used in
   the analysis of flow data and creation of analysis products often use
   files as a convenient unit of work, with an ephemeral lifetime.  A
   set of flows relevant to a security investigation may be stored in a
   file for the duration of that investigation, and further exchanged
   among incident handlers via email or within an external incident



Trammell, et al.        Expires January 15, 2009                [Page 7]

Internet-Draft                 IPFIX Files                     July 2008


   handling workflow application.  Sets of flow data relevant to
Sets is confusing as it begins the sentence. Is this the definition from RFC 5101? I guess no.

   Internet measurement research may be published as files, much as
   libpcap packet trace files are, to provide common data sets for the
Data Sets?
   repeatability of research efforts; these files would have lifetimes
   measured in months or years.  Operational flow measurement systems
   also have a need for long-term, archival storage of flow data, either
   as a primary flow data repository, or as a backing tier for online
   storage in a relational database management system (RDBMS).

   The variety of applications of flow data, and the variety of
   presently deployed storage approaches, would seem to indicate the
   need for a standard approach to flow storage with applicability
   across the continuum of time scales over which flow data is stored.
   A storage format based around flat files would best address the
   variety of storage requirements.  While much work has been done on
   structured storage via RDBMS, relational database systems are not a
   good basis for format standardization owing to the fact that their
   internal data structures are generally private to a single
   implementation and subject to change for internal reasons.  Also,
   there are a wide variety of operations available on flat files, and
   external tools and standards can be leveraged to meet file-based flow
   storage requirements.  Further, flow data is often not very
   semantically complicated, and is managed in very high volume;
   therefore, an RDBMS-based flow storage system would not benefit much
   from the advantages of relational database technology.

   The simplest way to create a new file format is simply to serialize
   some internal data model to disk, with either textual or binary
   representation of data elements, and some framing strategy for
   delimiting fields and records.  "Ad-hoc" file formats such as this
   have several important disadvantages.  They impose the semantics of
   the data model from which they are derived on the file format, and ast the IPFIX Message contains the
IPFIX Message Header.

Maybe one extra sentence would be welcome...

>From the definition only, it's not obvious.


   IPFIX Message

      An IPFIX Message is a message originating at the Exporting Process
      that carries the IPFIX records of this Exporting Process and whose
      destination is a Collecting Process.  An IPFIX Message is
      encapsulated at the transport layer.


We have to look at figure B of RFC5101.





  
   See Section 7 for details of the implementation of this design,
   including specific requirements and guidelines for File Readers and
   File Writers, and Information Elements and Options Templates used for
   file metadata.
  



I would remove this paragraph. Already covered.



  

4.  Motivation

   There are a wide variety of applications for the file-based storage
  



There is 



  
   of IP flow data, across a continuum of time scales.  Tools used in
   the analysis of flow data and creation of analysis products often use
   files as a convenient unit of work, with an ephemeral lifetime.  A
   set of flows relevant to a security investigation may be stored in a
   file for the duration of that investigation, and further exchanged
   among incident handlers via email or within an external incident



Trammell, et al.        Expires January 15, 2009                [Page 7]

Internet-Draft                 IPFIX Files                     July 2008


   handling workflow application.  Sets of flow data relevant to
  



Sets is confusing as it begins the sentence. Is this the definition
from RFC 5101? I guess no.





  
   Internet measurement research may be published as files, much as
   libpcap packet trace files are, to provide common data sets for the
  



Data Sets?



  
   repeatability of research efforts; these files would have lifetimes
   measured in months or years.  Operational flow measurement systems
   also have a need for long-term, archival storage of flow data, either
   as a primary flow data repository, or as a backing tier for online
   storage in a relational database management system (RDBMS).

   The variety of applications of flow data, and the variety of
   presently deployed storage approaches, would seem to indicate the
   need for a standard approach to flow storage with applicability
   across the continuum of time scales over which flow data is stored.
   A storage format based around flat files would best address the
   variety of storage requirements.  While much work has been done on
   structured storage via RDBMS, relational database systems are not a
   good basis for format standardization owing to the fact that their
   internal data structures are generally private to a single
   implementation and subject to change for internal reasons.  Also,
   there are a wide variety of operations available on flat files, and
   external tools and standards can be leveraged to meet file-based flow
   storage requirements.  Further, flow data is often not very
   semantically complicated, and is managed in very high volume;
   therefore, an RDBMS-based flow storage system would not benefit much
   from the advantages of relational database technology.

   The simplest way to create a new file format is simply to serialize
   some internal data model to disk, with either textual or binary
   representation of data elements, and some framing strategy for
   delimiting fields and records.  "Ad-hoc" file formats such as this
   have several important disadvantages.  They impose the semantics of
   the data model from which they are derived on the file format, and 
   such, they are difficult to extend, describe, and standardize.

   Indeed, one de facto standard for the storage of flow data is one of
   these ad-hoc formats.  A common method of storing data collected via
   Cisco NetFlow V5 or V7 is to serialize a stream of raw NetFlow
  



or V8



  
   datagrams into files.  These NetFlow PDU files consist of a
   collection of header-prefixed blocks (corresponding to the datagrams
   as received on the wire) containing fixed-length binary flow records.
   NetFlow V5 and V7 data may be mixed within a given file, as the
  



and V8



  
   header on each datagram defines the NetFlow version of the records
   following; there is indeed very little difference between the two
   record formats.  



Why is this important? The important message is that their format is
fixed.



  
While this NetFlow PDU file format has all the
   disadvantages of an ad-hoc format, and is not extensible to data
   models other than that defined by Cisco NetFlow, it is at least
   reasonably well-understood due to its ubiquity.




Trammell, et al.        Expires January 15, 2009                [Page 8]

Internet-Draft                 IPFIX Files                     July 2008


   Over the past decade XML markup has emerged as a new "universal"
   representation format for structured data.  It is intended to be
   human-readable; indeed, that is one reason for its rapid adoption.
   However XML has limited usefulness for representing network flow
   data.  Network flow data has a simple, repetitive, non-hierarchical
   structure that does not benefit much from XML.  An XML representation
   of flow data would be an essentially flat list of the attributes and
   their values for each flow record.

   The XML approach to data encoding is very heavyweight when compared
   to binary flow encoding.  XML's use of start- and end-tags, and
   plain-text encoding of the actual values, leads to significant
   inefficiency in encoding size.  Typical network flow datasets can
  



data sets or Data Sets? What's a flow dataset?

Throughout the document, my impression is that there is a lack of
consistency for some terms:

    data sets, flow data sets, Data Sets, ....

    flow, flow record, flow data, etc...

    file-based, IPFIX file, etc...



  
   contain millions or billions of flows per hour of traffic
   represented.  Any increase in storage size per record can have
   dramatic impact on flow data storage and transfer sizes.  While data
   compression algorithms can partially remove the redundancy introduced
   by XML encoding, they introduce additional overhead of their own.

   A further problem is that XML processing tools require a full XML
   parser.  XML parsers are fully general and therefore complex,
   resource-intensive and relatively slow, introducing significant
   processing time overhead for large network-flow datasets.  In
   contrast, parsers for typical binary flow data encodings are simply
   structured, since they only need to parse a very small header and
   then have complete knowledge of all following fields for the
   particular flow.  These can then be read in a very efficient linear
   fashion.

   This leads us to propose the IPFIX Message format as the basis for a
   new flow data file format.  



Here is another example: flow data file format


  
The IPFIX working group, in defining the
   IPFIX protocol, has already defined an information model and data
   formatting rules for representation of flow data.  Especially at
   shorter time scales, when a file isas
   such, they are difficult to extend, describe, and standardize.

   Indeed, one de facto standard for the storage of flow data is one of
   these ad-hoc formats.  A common method of storing data collected via
   Cisco NetFlow V5 or V7 is to serialize a stream of raw NetFlow
  



or V8



  
   datagrams into files.  These NetFlow PDU files consist of a
   collection of header-prefixed blocks (corresponding to the datagrams
   as received on the wire) containing fixed-length binary flow records.
   NetFlow V5 and V7 data may be mixed within a given file, as the
  



and V8



  
   header on each datagram defines the NetFlow version of the records
   following; there is indeed very little difference between the two
   record formats.  



Why is this important? The important message is that their format is
fixed.



  
While this NetFlow PDU file format has all the
   disadvantages of an ad-hoc format, and is not extensible to data
   models other than that defined by Cisco NetFlow, it is at least
   reasonably well-understood due to its ubiquity.




Trammell, et al.        Expires January 15, 2009                [Page 8]

Internet-Draft                 IPFIX Files                     July 2008


   Over the past decade XML markup has emerged as a new "universal"
   representation format for structured data.  It is intended to be
   human-readable; indeed, that is one reason for its rapid adoption.
   However XML has limited usefulness for representing network flow
   data.  Network flow data has a simple, repetitive, non-hierarchical
   structure that does not benefit much from XML.  An XML representation
   of flow data would be an essentially flat list of the attributes and
   their values for each flow record.

   The XML approach to data encoding is very heavyweight when compared
   to binary flow encoding.  XML's use of start- and end-tags, and
   plain-text encoding of the actual values, leads to significant
   inefficiency in encoding size.  Typical network flow datasets can
  



data sets or Data Sets? What's a flow dataset?

Throughout the document, my impression is that there is a lack of
consistency for some terms:

    data sets, flow data sets, Data Sets, ....

    flow, flow record, flow data, etc...

    file-based, IPFIX file, etc...



  
   contain millions or billions of flows per hour of traffic
   represented.  Any increase in storage size per record can have
   dramatic impact on flow data storage and transfer sizes.  While data
   compression algorithms can partially remove the redundancy introduced
   by XML encoding, they introduce additional overhead of their own.

   A further problem is that XML processing tools require a full XML
   parser.  XML parsers are fully general and therefore complex,
   resource-intensive and relatively slow, introducing significant
   processing time overhead for large network-flow datasets.  In
   contrast, parsers for typical binary flow data encodings are simply
   structured, since they only need to parse a very small header and
   then have complete knowledge of all following fields for the
   particular flow.  These can then be read in a very efficient linear
   fashion.

   This leads us to propose the IPFIX Message format as the basis for a
   new flow data file format.  



Here is another example: flow data file format


  
The IPFIX working group, in defining the
   IPFIX protocol, has already defined an information model and data
   formatting rules for representation of flow data.  Especially at
   shorter time scales, when a file  a unit of data interchange, the
   filesystem may be viewed as simply another IPFIX Message transport
   between processes.  This format is especially well suited to
   representing flow data, as it was designed specifically for flow data
   export; it is easily extensible unlike ad-hoc serialization, and
   compact unlike XML.  In addition, IPFIX is an IETF standard for the
   export and collection of flow data; using a common format for storage
   and analysis at the collection side allows implementors to use
   substantially the same information model and data formatting
   implementation for transport as well as storage.


5.  Requirements

   In this section, we outline a proposed set of requirements



Trammell, et al.        Expires January 15, 2009                [Page 9]

Internet-Draft                 IPFIX Files                     July 2008


   [SAINT2007] for any persistent storage format for flow data.  First
   and foremost, a flow data file format should support storage across
   the continuum of time scales important to flow storage applications.
   Each of the requirements enumerated in the sections below is broadly
   applicable to flow storage applications, though each may be more
   important at certain time scales.  For each, we first identify the
   requirement, then explain how the IPFIX Message format addresses it,
   or briefly outline the changes that must be made in order for an
   IPFIX-based file format to meet the requirement.

5.1.  Record Format Flexibility

   Due to the wide variety of flow attributes collected by different
   network flow attribute measurement systems, the ideal flow storage
   format will not impose a single data model or a specific record type
   on the flows it stores.  The file format must be flexible and
   extensible; that is, it must support the definition of multiple
   record types within the file itself, and must be able to support new
   field types for data within the records in a graceful way.
  



A generic comment.

So far in the document, you have chosen to use generic terms, not
capitalized.

I'm not too sure how it helps. I even think that this is confusing.
What is a "field types for data"

After all, you wrote a specification, which IMHO should reuse the
terminology 

Something such as:


   Due to the wide variety of Flow attributes collected by different
   IPFIX Device, the ideal Flow storage
   format will not impose a single data model or a specific Record type
   on the flows it stores.  The IPFIX File MUST be flexible and
   extensible; that is, it MUST support the definition of multiple
   Flow Records within the IPFIX File itself, and MUST be able to support new
   Information Elements.

Now, the use of MUST versus must in the requirement section is
debatable.



  
   IPFIX provides record format flexibility through the use of Templates
   to describe each Data Record, through the use of an IANA Registry to
   define its Information Elements, and through the use of enterprise-
   specific Information Elements.

5.2.  Self Description

   Archived data may be read at a time in the future where any external
   reference to the meaning of the data may be lost.  The ideal flow
   storage format should be self-describing; that is, a process reading
   flow data from storage should be able to properly interpret the
   stored flows without reference to anything other than standard
   sources (e.g., the standards document describing the file format) and
   the stored flow data itself.

   The IPFIX Message format is partially self-describing; that is, IPFIX
   Templates containing only IANA-assigned Information Elements can be
   completely interpreted according to the IPFIX Information Model
   without additional external data.

   However, Templates containing private information elements lack
   detailed type and semantic information; a Collecting Process
   receiving data described by a template containing private Inis a unit of data interchange, the
   filesystem may be viewed as simply another IPFIX Message transport
   between processes.  This format is especially well suited to
   representing flow data, as it was designed specifically for flow data
   export; it is easily extensible unlike ad-hoc serialization, and
   compact unlike XML.  In addition, IPFIX is an IETF standard for the
   export and collection of flow data; using a common format for storage
   and analysis at the collection side allows implementors to use
   substantially the same information model and data formatting
   implementation for transport as well as storage.


5.  Requirements

   In this section, we outline a proposed set of requirements



Trammell, et al.        Expires January 15, 2009                [Page 9]

Internet-Draft                 IPFIX Files                     July 2008


   [SAINT2007] for any persistent storage format for flow data.  First
   and foremost, a flow data file format should support storage across
   the continuum of time scales important to flow storage applications.
   Each of the requirements enumerated in the sections below is broadly
   applicable to flow storage applications, though each may be more
   important at certain time scales.  For each, we first identify the
   requirement, then explain how the IPFIX Message format addresses it,
   or briefly outline the changes that must be made in order for an
   IPFIX-based file format to meet the requirement.

5.1.  Record Format Flexibility

   Due to the wide variety of flow attributes collected by different
   network flow attribute measurement systems, the ideal flow storage
   format will not impose a single data model or a specific record type
   on the flows it stores.  The file format must be flexible and
   extensible; that is, it must support the definition of multiple
   record types within the file itself, and must be able to support new
   field types for data within the records in a graceful way.
  



A generic comment.

So far in the document, you have chosen to use generic terms, not
capitalized.

I'm not too sure how it helps. I even think that this is confusing.
What is a "field types for data"

After all, you wrote a specification, which IMHO should reuse the
terminology 

Something such as:


   Due to the wide variety of Flow attributes collected by different
   IPFIX Device, the ideal Flow storage
   format will not impose a single data model or a specific Record type
   on the flows it stores.  The IPFIX File MUST be flexible and
   extensible; that is, it MUST support the definition of multiple
   Flow Records within the IPFIX File itself, and MUST be able to support new
   Information Elements.

Now, the use of MUST versus must in the requirement section is
debatable.



  
   IPFIX provides record format flexibility through the use of Templates
   to describe each Data Record, through the use of an IANA Registry to
   define its Information Elements, and through the use of enterprise-
   specific Information Elements.

5.2.  Self Description

   Archived data may be read at a time in the future where any external
   reference to the meaning of the data may be lost.  The ideal flow
   storage format should be self-describing; that is, a process reading
   flow data from storage should be able to properly interpret the
   stored flows without reference to anything other than standard
   sources (e.g., the standards document describing the file format) and
   the stored flow data itself.

   The IPFIX Message format is partially self-describing; that is, IPFIX
   Templates containing only IANA-assigned Information Elements can be
   completely interpreted according to the IPFIX Information Model
   without additional external data.

   However, Templates containing private information elements lack
   detailed type and semantic information; a Collecting Process
   receiving data described by a template containing private formation
  



Data Records by a Template.

Note: from now on, I won't make this "capitalization" comment again



  
   Elements it does not understand can only treat the data contained
   within those Information Elements as octet arrays.  To be fully self-
   describing, enterprise-specific Information Elements must be
   additionally described via IPFIX Options according to the Information
   Element Type Options Template defined in "Exporting Type Information



Trammell, et al.        Expires January 15, 2009               [Page 10]

Internet-Draft                 IPFIX Files                     July 2008


   for IPFIX Information Elements" [I-D.ietf-ipfix-exporting-type].

5.3.  Data Compression

   Regardless of the representation format, flow data describing traffic
   on real networks tends to be highly compressible.  Compression tends
   to improve the scalability of flow collection systems, by reducing
   the disk storage and I/O bandwidth requirement for a given workload.
   The ideal flow storage format should support applications which wish
   to leverage this fact by supporting compression of stored data.

   The IPFIX Message format has no support for data compression, as the
   IPFIX protocol was designed for speed and simplicity of export.  Of
   course, any flat file is readily compressible using a wide variety of
   external data compression tools, formats, and algorithms; therefore,
   this requirement can be met externally.

   However, a couple of simple optimizations can be made by File Writers
   to increase the integrity and usability of compressed IPFIX data;
   these are outlined in Section 9.1.

5.4.  Indexing and Searching

   Binary, record stream oriented file formats natively support only one
   form of searching, sequential scan in file order.  By choosing the
   order of records in a file carefully (e.g., by flow start or flow end
   time), a file can be indexed by a single key.

   Beyond this, properly addressing indexing is an application-specific
   problem, as it inherently involves tradeoffs between storage
   complexity 



In this section you might want to develop on the "storage complexity"

What I have in mind is the difficulty for an IPFIX Device to support
the "order of records in a file" option.

Based on the timeouts described in [IPFIX-ARCH], it's very difficult to
do order by flow start without huge buffers or constantly inserting
flow records at the right place in the file. Both options are not
practical in real-time. Neither in an IPFIX device or a collector.



  
and retrieval speed, and requirements vary widely based on
   time scales and the types of queries used from site to site.
   However, a generic standard flow storage format may provide limited
   direct support for indexing and searching.

   The ideal flow storage format will support a limited table of
   contents facility noting that the records in a file contain data
   relating only to certain keys or values of keys, in order to keep
   multi-file search implementations from having to scan a file for data
   it does not contain.

   The IPFIX Message format has no direct support for indexing.
   However, its template mechanism and the technique described in
   "Reducing Redundancy in IPFIX and PSAMP Reports"
   [I-D.ietf-ipfix-reducing-redundancy] can be used to describe the
   contents of a file in a limited way.  Additionally, as flow data is
   often sorted and divided by time, the start and end time of the flows
   in a file may be declared using the File Time Window Options Template



Trammell, et al.        Expires January 15, 2009               [Page 11]

Internet-Draft                 IPFIX Files                     July 2008


   defined in Section 8.1.2.

5.5.  Data Integrity

   When storing flow data over long time scales, especially for archival
   purposes, it is important to ensure that hardwaInformation
  



Data Records by a Template.

Note: from now on, I won't make this "capitalization" comment again



  
   Elements it does not understand can only treat the data contained
   within those Information Elements as octet arrays.  To be fully self-
   describing, enterprise-specific Information Elements must be
   additionally described via IPFIX Options according to the Information
   Element Type Options Template defined in "Exporting Type Information



Trammell, et al.        Expires January 15, 2009               [Page 10]

Internet-Draft                 IPFIX Files                     July 2008


   for IPFIX Information Elements" [I-D.ietf-ipfix-exporting-type].

5.3.  Data Compression

   Regardless of the representation format, flow data describing traffic
   on real networks tends to be highly compressible.  Compression tends
   to improve the scalability of flow collection systems, by reducing
   the disk storage and I/O bandwidth requirement for a given workload.
   The ideal flow storage format should support applications which wish
   to leverage this fact by supporting compression of stored data.

   The IPFIX Message format has no support for data compression, as the
   IPFIX protocol was designed for speed and simplicity of export.  Of
   course, any flat file is readily compressible using a wide variety of
   external data compression tools, formats, and algorithms; therefore,
   this requirement can be met externally.

   However, a couple of simple optimizations can be made by File Writers
   to increase the integrity and usability of compressed IPFIX data;
   these are outlined in Section 9.1.

5.4.  Indexing and Searching

   Binary, record stream oriented file formats natively support only one
   form of searching, sequential scan in file order.  By choosing the
   order of records in a file carefully (e.g., by flow start or flow end
   time), a file can be indexed by a single key.

   Beyond this, properly addressing indexing is an application-specific
   problem, as it inherently involves tradeoffs between storage
   complexity 



In this section you might want to develop on the "storage complexity"

What I have in mind is the difficulty for an IPFIX Device to support
the "order of records in a file" option.

Based on the timeouts described in [IPFIX-ARCH], it's very difficult to
do order by flow start without huge buffers or constantly inserting
flow records at the right place in the file. Both options are not
practical in real-time. Neither in an IPFIX device or a collector.



  
and retrieval speed, and requirements vary widely based on
   time scales and the types of queries used from site to site.
   However, a generic standard flow storage format may provide limited
   direct support for indexing and searching.

   The ideal flow storage format will support a limited table of
   contents facility noting that the records in a file contain data
   relating only to certain keys or values of keys, in order to keep
   multi-file search implementations from having to scan a file for data
   it does not contain.

   The IPFIX Message format has no direct support for indexing.
   However, its template mechanism and the technique described in
   "Reducing Redundancy in IPFIX and PSAMP Reports"
   [I-D.ietf-ipfix-reducing-redundancy] can be used to describe the
   contents of a file in a limited way.  Additionally, as flow data is
   often sorted and divided by time, the start and end time of the flows
   in a file may be declared using the File Time Window Options Template



Trammell, et al.        Expires January 15, 2009               [Page 11]

Internet-Draft                 IPFIX Files                     July 2008


   defined in Section 8.1.2.

5.5.  Data Integrity

   When storing flow data over long time scales, especially for archival
   purposes, it is important to ensure that hardre or software faults
   do not introduce errors into the data over time.  The ideal flow
   storage format will support the detection and correction of encoding-
   level errors in the data.

   Note that more advanced error correction is almost certainly best
   handled at a layer below that addressed by this document.  Error
   correction is a topic well addressed by the storage industry in
   general (e.g. by RAID and other technologies), and by specifying a
   flow storage format based upon files, we can leverage these features
   to meet this requirement.

   However, the ideal flow storage format will be resilient against
   errors, providing an internal facility for the detection of errors
   and the ability to isolate errors to as few data records as possible.

   Note that this requirement interacts with the choice of data
   compression or encryption algorithm.  The use of block compression
   algorithms can serve to isolate errors to a single compression block,
   unlike stream compressors, which may fail to resynchronize after a
   single bit error, invalidating the entire message stream.  Similarly,
   the use of a stream cipher can serve to isolate errors in the
   plaintext without amplifying them as, for example, a cipher in CBC
   mode can.  See the "Recommended Compression Error Resilience
   Strategy" and "Recommended Encryption Error Resilience Strategy"
   sections below for more on this interaction.

   The IPFIX Message format does not support data integrity assurance.
  



the IPFIX specifications [RFC 5101] does not ...



  
   It is assumed that advanced error correction will be provided
   externally.  For simple error detection support, checksums may be
   attached to messages via IPFIX Options according to the Message
   Checksum Options Template defined in Section 8.1.1.

5.6.  Creator Authentication and Confidentiality

   Storage of flow data across long time scales may also require
  



remove "long time scales"



  
   assurance that no unauthorized entity can read or modify the stored
   data.  Asymmetric-key cryptography can be applied to this problem, by
   signing flow data with the private key of the creator, and encrypting
   it with the public keys of those authorized to read it.  The ideal
   flow storage format will support the encryption and signing of flow
   data.




Trammell, et al.        Expires January 15, 2009               [Page 12]

Internet-Draft                 IPFIX Files                     July 2008


   As with error correction, this problem has been addressed well at a
   layer below that addressed by this document.  Instead of specifying a
   particular choice of encryption technology, we can leverage the fact
   that existing cryptographic technologies work quite well on data
   stored in files to meet this requirement.

   Beyond support for the use of TLS for transport over TCP or DTLS for
   transport over SCTP or UDP, both of which provide transient
   authentication and confidentiality, the IPFIX protocol does not
   support this requirement directly.  It is assumed that this
   requirement will be met externally.

5.7.  Anonymization and Obfuscation

   To ensure the privacy of individuals and organizations at the
   endpoints of communications represented by flow records, it is often
   necessary to obfuscate or anonymize stored and exported flow data.
   The ideal flow storage format will provide for a notation that a
   given information element on a given record type represents
   anonymized, rather than real, data.

   The IPFIX Message format presently has no support for anonymization
   notation.  



the IPFIX specifications [RFC 5101] does not ...


  
It should be noted that anonymization is one of the
   requirements given for IPFIX in [RFC3917].  The decision to qualiware or software faults
   do not introduce errors into the data over time.  The ideal flow
   storage format will support the detection and correction of encoding-
   level errors in the data.

   Note that more advanced error correction is almost certainly best
   handled at a layer below that addressed by this document.  Error
   correction is a topic well addressed by the storage industry in
   general (e.g. by RAID and other technologies), and by specifying a
   flow storage format based upon files, we can leverage these features
   to meet this requirement.

   However, the ideal flow storage format will be resilient against
   errors, providing an internal facility for the detection of errors
   and the ability to isolate errors to as few data records as possible.

   Note that this requirement interacts with the choice of data
   compression or encryption algorithm.  The use of block compression
   algorithms can serve to isolate errors to a single compression block,
   unlike stream compressors, which may fail to resynchronize after a
   single bit error, invalidating the entire message stream.  Similarly,
   the use of a stream cipher can serve to isolate errors in the
   plaintext without amplifying them as, for example, a cipher in CBC
   mode can.  See the "Recommended Compression Error Resilience
   Strategy" and "Recommended Encryption Error Resilience Strategy"
   sections below for more on this interaction.

   The IPFIX Message format does not support data integrity assurance.
  



the IPFIX specifications [RFC 5101] does not ...



  
   It is assumed that advanced error correction will be provided
   externally.  For simple error detection support, checksums may be
   attached to messages via IPFIX Options according to the Message
   Checksum Options Template defined in Section 8.1.1.

5.6.  Creator Authentication and Confidentiality

   Storage of flow data across long time scales may also require
  



remove "long time scales"



  
   assurance that no unauthorized entity can read or modify the stored
   data.  Asymmetric-key cryptography can be applied to this problem, by
   signing flow data with the private key of the creator, and encrypting
   it with the public keys of those authorized to read it.  The ideal
   flow storage format will support the encryption and signing of flow
   data.




Trammell, et al.        Expires January 15, 2009               [Page 12]

Internet-Draft                 IPFIX Files                     July 2008


   As with error correction, this problem has been addressed well at a
   layer below that addressed by this document.  Instead of specifying a
   particular choice of encryption technology, we can leverage the fact
   that existing cryptographic technologies work quite well on data
   stored in files to meet this requirement.

   Beyond support for the use of TLS for transport over TCP or DTLS for
   transport over SCTP or UDP, both of which provide transient
   authentication and confidentiality, the IPFIX protocol does not
   support this requirement directly.  It is assumed that this
   requirement will be met externally.

5.7.  Anonymization and Obfuscation

   To ensure the privacy of individuals and organizations at the
   endpoints of communications represented by flow records, it is often
   necessary to obfuscate or anonymize stored and exported flow data.
   The ideal flow storage format will provide for a notation that a
   given information element on a given record type represents
   anonymized, rather than real, data.

   The IPFIX Message format presently has no support for anonymization
   notation.  



the IPFIX specifications [RFC 5101] does not ...


  
It should be noted that anonymization is one of the
   requirements given for IPFIX in [RFC3917].  The decision to quafy
   this requirement with 'MAY' and not 'MUST' in the requirements
   document, and its subsequent lack of specification in the current
   version of the IPFIX protocol, is due to the fact that anonymization
   algorithms are still an open area of research, and that there
   currently exist no standardized methods for anonymization.

   No support is presently defined in [RFC5101] or this IPFIX-based File
   Format for anonymization, as anonymization notation is an area of
   open work for the IPFIX working group.

5.8.  Session Auditability and Replayability

   Certain use cases for archival flow storage require the storage of
   collection infrastructure details alongside the data itself.  These
   details include information about how and when data was received, and
   where it was received from, and are useful for auditing as well as
   for the replaying received data for testing purposes.

   The IPFIX Message format contains no direct support for auditability
  



the IPFIX specifications [RFC 5101] does not ...


  
   and replayability, though the IPFIX Information Model does define
   various Information Elements required to represent collection
   infrastructure details.  These details may be stored in IPFIX Files
   using the Export Session Details Options Template defined in
   Section 8.1.3 and the Message Details Options Template defined in



Trammell, et al.        Expires January 15, 2009               [Page 13]

Internet-Draft                 IPFIX Files                     July 2008


   Section 8.1.4.

5.9.  Performance Characteristics

   The ideal standard flow storage format will not have a significant
   negative impact on the performance of the application generating or
   processing flow data stored in the format.  This is a non-functional
   requirement, but it is important to note that a standard that implies
   a significant performance penalty is unlikely to be widely
   implemented and adopted.

   A static analysis of the IPFIX Message format would seem to suggest
   that implementations of it are not particularly prone to slowness;
  



What is a static analysis?



Not prone to slowness... at the condition that there is no re-ordering
of  flow records, compression, or anonymization before you write into
the file.

In other words, at the condition that you write in sequence what you
receive/want to export.



  
   indeed, a template-based data representation is more easily subject
   to optimization for common cases than representations that embed
   structural information directly in the data stream (e.g.  XML).
   However, a full analysis of the impact of using IPFIX Messages as a
   basis for flow data storage on read/write performance will require
   more implementation experience and performance measurement.
  



I think the section order was discussed before. I don't remember the
conclusion

Here I see the applicability after the requirements.



  

6.  Applicability

   This section describes the specific applicability of IPFIX Files to
   various use cases.  IPFIX Files are particularly useful in a flow
   collection and processing infrastructure using IPFIX for flow export.
   We explore the applicability and provide guidelines for using IPFIX
   files for the storage of flow data collected by IPFIX Collecting
   Processes and NetFlow V9 collectors, the testing of IPFIX Collecting
   Processes, and diagnostics of IPFIX Devices.

6.1.  Storage of IPFIX-collected Flow Data

   IPFIX Files can naturally be used to store flow data collected by an
   IPFIX Collecting Process; indeed, this was one of the primary initial
   motivations behind the file format described within this document.
   Using IPFIX Files as such provides a single, standard, well-
   understood encoding to be used for fllify
   this requirement with 'MAY' and not 'MUST' in the requirements
   document, and its subsequent lack of specification in the current
   version of the IPFIX protocol, is due to the fact that anonymization
   algorithms are still an open area of research, and that there
   currently exist no standardized methods for anonymization.

   No support is presently defined in [RFC5101] or this IPFIX-based File
   Format for anonymization, as anonymization notation is an area of
   open work for the IPFIX working group.

5.8.  Session Auditability and Replayability

   Certain use cases for archival flow storage require the storage of
   collection infrastructure details alongside the data itself.  These
   details include information about how and when data was received, and
   where it was received from, and are useful for auditing as well as
   for the replaying received data for testing purposes.

   The IPFIX Message format contains no direct support for auditability
  



the IPFIX specifications [RFC 5101] does not ...


  
   and replayability, though the IPFIX Information Model does define
   various Information Elements required to represent collection
   infrastructure details.  These details may be stored in IPFIX Files
   using the Export Session Details Options Template defined in
   Section 8.1.3 and the Message Details Options Template defined in



Trammell, et al.        Expires January 15, 2009               [Page 13]

Internet-Draft                 IPFIX Files                     July 2008


   Section 8.1.4.

5.9.  Performance Characteristics

   The ideal standard flow storage format will not have a significant
   negative impact on the performance of the application generating or
   processing flow data stored in the format.  This is a non-functional
   requirement, but it is important to note that a standard that implies
   a significant performance penalty is unlikely to be widely
   implemented and adopted.

   A static analysis of the IPFIX Message format would seem to suggest
   that implementations of it are not particularly prone to slowness;
  



What is a static analysis?



Not prone to slowness... at the condition that there is no re-ordering
of  flow records, compression, or anonymization before you write into
the file.

In other words, at the condition that you write in sequence what you
receive/want to export.



  
   indeed, a template-based data representation is more easily subject
   to optimization for common cases than representations that embed
   structural information directly in the data stream (e.g.  XML).
   However, a full analysis of the impact of using IPFIX Messages as a
   basis for flow data storage on read/write performance will require
   more implementation experience and performance measurement.
  



I think the section order was discussed before. I don't remember the
conclusion

Here I see the applicability after the requirements.



  

6.  Applicability

   This section describes the specific applicability of IPFIX Files to
   various use cases.  IPFIX Files are particularly useful in a flow
   collection and processing infrastructure using IPFIX for flow export.
   We explore the applicability and provide guidelines for using IPFIX
   files for the storage of flow data collected by IPFIX Collecting
   Processes and NetFlow V9 collectors, the testing of IPFIX Collecting
   Processes, and diagnostics of IPFIX Devices.

6.1.  Storage of IPFIX-collected Flow Data

   IPFIX Files can naturally be used to store flow data collected by an
   IPFIX Collecting Process; indeed, this was one of the primary initial
   motivations behind the file format described within this document.
   Using IPFIX Files as such provides a single, standard, well-
   understood encoding to be used for ow data on disk and on the wire,
   and allows IPFIX implementations to leverage substantially the same
   code for flow export and flow storage.  In addition, the storage of
   single Transport Sessions in IPFIX Files is particularly important
   for network measurement research, allowing repeatability of
   experiments by providing a format for the storage and exchange of
   IPFIX flow trace data much as the libpcap format is used for
   experiments on packet trace data.






Trammell, et al.        Expires January 15, 2009               [Page 14]

Internet-Draft                 IPFIX Files                     July 2008


6.2.  Storage of NetFlow V9-collected Flow Data

   Although the IPFIX protocol is based on the Cisco Netflow Services,
   Version 9 (NetFlow V9) protocol [RFC3954], the two have diverged
   since work began on IPFIX.  However, since the NetFlow V9 information
   model is a compatible subset of the IPFIX information model, it is
   possible to use IPFIX files to store collected NetFlow V9 flow data.
   This approach may be particularly useful in multi-vendor, multi-
   protocol collection infrastructures using both NetFlow V9 and IPFIX
   to export flow data.

   The applicability of IPFIX Files to this use case is outlined in
   Appendix B.

6.3.  Testing IPFIX Collecting Processes

   IPFIX Files can be used to store IPFIX Messages for the testing of
   IPFIX Collecting Processes.  A variety of test cases may be stored in
   IPFIX Files.  First, IPFIX data sets collected in real network
  



Data Sets



  
   environments and stored in an IPFIX File can be used as input to
   check the behavior of new or extended implementations of IPFIX
   Collectors.  Furthermore, IPFIX Files can be used to validate the
   operation of a given IPFIX Collecting Process in a new environment,
   i.e., to test with recorded IPFIX data from the target network before
   installing the Collecting Process in the network.

   The IPFIX File format can also be used to store artificial, non-
   compliant reference messages for specific Collecting Process test
   cases.  Examples for such test cases are sets of IPFIX records with
   undefined Information Elements, Data Records described by missing
   Templates, or incorrectly framed messages or data sets.
   Representative error handling test cases are defined in "IPFIX
   Testing" [I-D.ietf-ipfix-testing].

   Furthermore, fast replay of IPFIX records stored in a file can be
  



IPFIX records -> IPFIX Messages



  
   used for stress/load tests (e.g., high rate of incoming Data Records,
   large Templates with high Information Element counts), as described
   in "IPFIX Testing" [I-D.ietf-ipfix-testing].  The provisioning and
   use of a set of reference files for testing simplifies the
   performance of tests and increases the comparability of test results.

6.4.  IPFIX Device Diagnostics

   As an IPFIX File can be used store any collection of flows, the
   format may also be used for dumping and storing various types of flow
   data for IPFIX Device diagnostics (e.g., the open flow cache of a
   Metering Process or the flow backlog of an Exporting or Collecting
   Process at the time of a process reset or crash).  File-based storage



Trammell, et al.        Expires January 15, 2009               [Page 15]

Internet-Draft                 IPFIX Files                     July 2008


   is preferable to remote transmission in such error-recovery
   situations.


7.  Detailed Description

   An IPFIX File, as introduced in Section 3 and elaborated below, is at
   its core simply an IPFIX Message stream serialized to some
   filesystem.  



Why re-explain the definition here?



  
Any valid serialized IPFIX Message stream MUST be
   accepted by a File Reader as a valid IPFIX file.  In this wayflow data on disk and on the wire,
   and allows IPFIX implementations to leverage substantially the same
   code for flow export and flow storage.  In addition, the storage of
   single Transport Sessions in IPFIX Files is particularly important
   for network measurement research, allowing repeatability of
   experiments by providing a format for the storage and exchange of
   IPFIX flow trace data much as the libpcap format is used for
   experiments on packet trace data.






Trammell, et al.        Expires January 15, 2009               [Page 14]

Internet-Draft                 IPFIX Files                     July 2008


6.2.  Storage of NetFlow V9-collected Flow Data

   Although the IPFIX protocol is based on the Cisco Netflow Services,
   Version 9 (NetFlow V9) protocol [RFC3954], the two have diverged
   since work began on IPFIX.  However, since the NetFlow V9 information
   model is a compatible subset of the IPFIX information model, it is
   possible to use IPFIX files to store collected NetFlow V9 flow data.
   This approach may be particularly useful in multi-vendor, multi-
   protocol collection infrastructures using both NetFlow V9 and IPFIX
   to export flow data.

   The applicability of IPFIX Files to this use case is outlined in
   Appendix B.

6.3.  Testing IPFIX Collecting Processes

   IPFIX Files can be used to store IPFIX Messages for the testing of
   IPFIX Collecting Processes.  A variety of test cases may be stored in
   IPFIX Files.  First, IPFIX data sets collected in real network
  



Data Sets



  
   environments and stored in an IPFIX File can be used as input to
   check the behavior of new or extended implementations of IPFIX
   Collectors.  Furthermore, IPFIX Files can be used to validate the
   operation of a given IPFIX Collecting Process in a new environment,
   i.e., to test with recorded IPFIX data from the target network before
   installing the Collecting Process in the network.

   The IPFIX File format can also be used to store artificial, non-
   compliant reference messages for specific Collecting Process test
   cases.  Examples for such test cases are sets of IPFIX records with
   undefined Information Elements, Data Records described by missing
   Templates, or incorrectly framed messages or data sets.
   Representative error handling test cases are defined in "IPFIX
   Testing" [I-D.ietf-ipfix-testing].

   Furthermore, fast replay of IPFIX records stored in a file can be
  



IPFIX records -> IPFIX Messages



  
   used for stress/load tests (e.g., high rate of incoming Data Records,
   large Templates with high Information Element counts), as described
   in "IPFIX Testing" [I-D.ietf-ipfix-testing].  The provisioning and
   use of a set of reference files for testing simplifies the
   performance of tests and increases the comparability of test results.

6.4.  IPFIX Device Diagnostics

   As an IPFIX File can be used store any collection of flows, the
   format may also be used for dumping and storing various types of flow
   data for IPFIX Device diagnostics (e.g., the open flow cache of a
   Metering Process or the flow backlog of an Exporting or Collecting
   Process at the time of a process reset or crash).  File-based storage



Trammell, et al.        Expires January 15, 2009               [Page 15]

Internet-Draft                 IPFIX Files                     July 2008


   is preferable to remote transmission in such error-recovery
   situations.


7.  Detailed Description

   An IPFIX File, as introduced in Section 3 and elaborated below, is at
   its core simply an IPFIX Message stream serialized to some
   filesystem.  



Why re-explain the definition here?



  
Any valid serialized IPFIX Message stream MUST be
   accepted by a File Reader as a valid IPFIX file.  In this w, the
   filesystem is simply treated as another IPFIX transport alongside
   SCTP, TCP, and UDP.  In contrast to normal IPFIX operation, 



what does it mean "normal IPFIX operation"?



  
the time
   between a File Writer writing an IPFIX Message stream to a File and a
   File Reader reading it can be extremely variable.  In other words,
   this notional file transport has unusually high latency, as the File
   Reader and File Writer do not necessarily run at the same time.
  



Is this important, since the IPFIX Message contains the correct time
anyway?

Or do you want to say: don't rely on the write time in the file?



  
   This section specifies the detailed actions of File Readers and File
   Writers in handling IPFIX Files, and further specifies actions of
   File Writers in specific use cases.  Unless otherwise specified
   herein, where appropriate IPFIX File Writers MUST behave as IPFIX
   Exporting Processes, and IPFIX File Readers MUST behave as IPFIX
   Collecting Processes.

7.1.  File Reader Specification

   An IPFIX File Reader MUST accept as valid any serialized IPFIX
   Message stream that would be considered valid by one or more of the
   other defined IPFIX transport layers.  



You explained that already a few paragraphs before.

>From a spec. point of view, can we say that an IPFIX File Reader must
comply to the Collecting Process specifications from RFC5101?



  
Practically, this means that
   the union of template management features supported by SCTP, TCP, and
   UDP MUST be supported in IPFIX Files.  The following requirements
  



I have a problem with "requirement" since the requirements were treated
in the previous section.

I would remove this sentence.



  
   apply to IPFIX File Readers:

   o  File Readers MUST accept IPFIX Messages containing Template Sets,
      Options Template Sets, and Data Sets within the same message, as
      with IPFIX over TCP or UDP.

   o  File Readers MUST accept Template Sets that define templates
  



Template



  
      already defined within the file, as may occur with template
  



IPFIX File

Template



  
      retransmission when using IPFIX over UDP as described in section
      10.3.6 of [RFC5101].  In the event of a conflict between a resent
      definition and a previous definition, the File Reader MUST assume
      that the new template replaces the old, as consistent with UDP
      template expiration and ID reuse.
  



What does the File Reader must do in that case?

Replace or simply write in sequence?

Btw, it depends on the transport. For SCTP, that's impossible:


   Template IDs are unique per SCTP association and per Observation
   Domain.  If the Collecting Process receives a Template that has
   already been received but that has not previously been withdrawn
   (i.e., a Template Record from the same Exporter Observation Domain
   with the same Template ID received on the SCTP association), then the
   Collecting Process MUST shut down the association.




What if the IPFIX Message is malformed? Write or not? Useful for
diagnostic but ... So basically the answer is: it depends.


   If the Collecting Process receives a malformed IPFIX Message, it MUST
   reset the SCTP association, discard the IPFIX Message, and SHOULD log
   the error.  Note that non-zero Set padding does not constitute a
   malformed IPFIX Message.

	










  
   ay, the
   filesystem is simply treated as another IPFIX transport alongside
   SCTP, TCP, and UDP.  In contrast to normal IPFIX operation, 



what does it mean "normal IPFIX operation"?



  
the time
   between a File Writer writing an IPFIX Message stream to a File and a
   File Reader reading it can be extremely variable.  In other words,
   this notional file transport has unusually high latency, as the File
   Reader and File Writer do not necessarily run at the same time.
  



Is this important, since the IPFIX Message contains the correct time
anyway?

Or do you want to say: don't rely on the write time in the file?



  
   This section specifies the detailed actions of File Readers and File
   Writers in handling IPFIX Files, and further specifies actions of
   File Writers in specific use cases.  Unless otherwise specified
   herein, where appropriate IPFIX File Writers MUST behave as IPFIX
   Exporting Processes, and IPFIX File Readers MUST behave as IPFIX
   Collecting Processes.

7.1.  File Reader Specification

   An IPFIX File Reader MUST accept as valid any serialized IPFIX
   Message stream that would be considered valid by one or more of the
   other defined IPFIX transport layers.  



You explained that already a few paragraphs before.

>From a spec. point of view, can we say that an IPFIX File Reader must
comply to the Collecting Process specifications from RFC5101?



  
Practically, this means that
   the union of template management features supported by SCTP, TCP, and
   UDP MUST be supported in IPFIX Files.  The following requirements
  



I have a problem with "requirement" since the requirements were treated
in the previous section.

I would remove this sentence.



  
   apply to IPFIX File Readers:

   o  File Readers MUST accept IPFIX Messages containing Template Sets,
      Options Template Sets, and Data Sets within the same message, as
      with IPFIX over TCP or UDP.

   o  File Readers MUST accept Template Sets that define templates
  



Template



  
      already defined within the file, as may occur with template
  



IPFIX File

Template



  
      retransmission when using IPFIX over UDP as described in section
      10.3.6 of [RFC5101].  In the event of a conflict between a resent
      definition and a previous definition, the File Reader MUST assume
      that the new template replaces the old, as consistent with UDP
      template expiration and ID reuse.
  



What does the File Reader must do in that case?

Replace or simply write in sequence?

Btw, it depends on the transport. For SCTP, that's impossible:


   Template IDs are unique per SCTP association and per Observation
   Domain.  If the Collecting Process receives a Template that has
   already been received but that has not previously been withdrawn
   (i.e., a Template Record from the same Exporter Observation Domain
   with the same Template ID received on the SCTP association), then the
   Collecting Process MUST shut down the association.




What if the IPFIX Message is malformed? Write or not? Useful for
diagnostic but ... So basically the answer is: it depends.


   If the Collecting Process receives a malformed IPFIX Message, it MUST
   reset the SCTP association, discard the IPFIX Message, and SHOULD log
   the error.  Note that non-zero Set padding does not constitute a
   malformed IPFIX Message.

	










  
 o  File Readers MUST accept Template Withdrawals as described in
      section 8 of [RFC5101], provided that the Template to be withdrawn
      is defined, as is the case with IPFIX over TCP and SCTP.
  



So the IPFIX Writer MUST write the TWM into the IPFIX File.



  


Trammell, et al.        Expires January 15, 2009               [Page 16]

Internet-Draft                 IPFIX Files                     July 2008


   Note that some applications, particularly those storing large
   collections of data over long periods of time, may benefit from the
   ability to treat a collection of IPFIX Files as a single Transport
   Session.  A File Reader MAY be configurable to treat a collection of
   Files (e.g., all the files in a directory) as a single Transport
   Session.  However, a File Reader MUST NOT treat a single IPFIX File
   as containing multiple Transport Sessions.
  



I disagree.  This was discussed already on the list.



  
   If an IPFIX File uses the technique described in "Reducing Redundancy
   in IPFIX and PSAMP Reports" [I-D.ietf-ipfix-reducing-redundancy] AND
   all of the non-Options Templates in the File contain the
   commonPropertiesId Information Element, a File Reader MAY assume the
   set of commonPropertiesId definitions provides a complete table of
   contents for the File for searching purposes.

7.2.  File Writer Specification

   While any valid serialized IPFIX Message stream is a valid IPFIX
   File, the following recommendations will improve representation
   simplicity and read performance in the general case, where possible.

   File Writers SHOULD emit each Template Set or Options Template Set to
   appear in the file before any Data Set described by the Templates
   within that Set, to ensure the File Reader can decode every Data Set
   without waiting to process subsequent Templates or Options Templates.
  



If the File Write is on a Collector,

    - Not possible with UDP.

    - Not possible with PR-SCTP without the [IPFIX-PER-STREAM] draft,
which you should mention.

      Obviously, removing the restriction "one file = one stream" 



  
   File Writers SHOULD emit Data Records described by Options Templates
   to appear in the file before any Data Records which depend on the
   scopes defined by those options.
  



If the File Write is on a Collector,

    - Not possible with UDP.

    - Not possible with PR-SCTP without the [IPFIX-PER-STREAM] draft,
which you should mention

      Obviously, removing the restriction "one file = one stream" 



  
   File Writers SHOULD use Template Withdrawals to withdraw Templates if
   template IDs need to be reused.  In this case, the new Templates
   reusing those IDs SHOULD appear directly in the file after the
   Template Withdrawals making the IDs available for reuse.  



If the File Write is on a Collector,

    - Not possible with UDP.

    - Not possible with PR-SCTP without the [IPFIX-PER-STREAM] draft,
which you should mention

      Obviously, removing the restriction "one file = one stream" 



  
Template
   Withdrawals SHOULD NOT be used unless necessary to reuse template
   IDs.
  



I don't understand what the File Writer should do with that statement?
... because I was assuming that the File Writer would simply write what
it receives...





  
   Each IPFIX File is generall  o  File Readers MUST accept Template Withdrawals as described in
      section 8 of [RFC5101], provided that the Template to be withdrawn
      is defined, as is the case with IPFIX over TCP and SCTP.
  



So the IPFIX Writer MUST write the TWM into the IPFIX File.



  


Trammell, et al.        Expires January 15, 2009               [Page 16]

Internet-Draft                 IPFIX Files                     July 2008


   Note that some applications, particularly those storing large
   collections of data over long periods of time, may benefit from the
   ability to treat a collection of IPFIX Files as a single Transport
   Session.  A File Reader MAY be configurable to treat a collection of
   Files (e.g., all the files in a directory) as a single Transport
   Session.  However, a File Reader MUST NOT treat a single IPFIX File
   as containing multiple Transport Sessions.
  



I disagree.  This was discussed already on the list.



  
   If an IPFIX File uses the technique described in "Reducing Redundancy
   in IPFIX and PSAMP Reports" [I-D.ietf-ipfix-reducing-redundancy] AND
   all of the non-Options Templates in the File contain the
   commonPropertiesId Information Element, a File Reader MAY assume the
   set of commonPropertiesId definitions provides a complete table of
   contents for the File for searching purposes.

7.2.  File Writer Specification

   While any valid serialized IPFIX Message stream is a valid IPFIX
   File, the following recommendations will improve representation
   simplicity and read performance in the general case, where possible.

   File Writers SHOULD emit each Template Set or Options Template Set to
   appear in the file before any Data Set described by the Templates
   within that Set, to ensure the File Reader can decode every Data Set
   without waiting to process subsequent Templates or Options Templates.
  



If the File Write is on a Collector,

    - Not possible with UDP.

    - Not possible with PR-SCTP without the [IPFIX-PER-STREAM] draft,
which you should mention.

      Obviously, removing the restriction "one file = one stream" 



  
   File Writers SHOULD emit Data Records described by Options Templates
   to appear in the file before any Data Records which depend on the
   scopes defined by those options.
  



If the File Write is on a Collector,

    - Not possible with UDP.

    - Not possible with PR-SCTP without the [IPFIX-PER-STREAM] draft,
which you should mention

      Obviously, removing the restriction "one file = one stream" 



  
   File Writers SHOULD use Template Withdrawals to withdraw Templates if
   template IDs need to be reused.  In this case, the new Templates
   reusing those IDs SHOULD appear directly in the file after the
   Template Withdrawals making the IDs available for reuse.  



If the File Write is on a Collector,

    - Not possible with UDP.

    - Not possible with PR-SCTP without the [IPFIX-PER-STREAM] draft,
which you should mention

      Obviously, removing the restriction "one file = one stream" 



  
Template
   Withdrawals SHOULD NOT be used unless necessary to reuse template
   IDs.
  



I don't understand what the File Writer should do with that statement?
... because I was assuming that the File Writer would simply write what
it receives...





  
   Each IPFIX File is generay synonymous with a single Transport
   Session.  File Writers SHOULD store the Templates and Options
   required to decode the data within the File in the File itself, and
   File Readers SHOULD NOT use Templates or Options defined in one file
   to decode or interpret Data Sets in another.
  



So the IPFIX File should keep in memory the (Options) Template for that
active Transport Session in case where it lasts a long time, where
several IPFIX Files are required in order to insert the (Options)
Template in the subsequent IPFIX Files?

A simple example: [IPFIX-RED-RED] used across IPFIX Files



  
   File Writers SHOULD write IPFIX Messages within an IPFIX File in
   ascending Export Time order.

   File Writers MAY write Data Records to an IPFIX File in any order.
   However, File Writers that write flow records to an IPFIX File in



Trammell, et al.        Expires January 15, 2009               [Page 17]

Internet-Draft                 IPFIX Files                     July 2008


   flowStartTime or flowEndTime order SHOULD be consistent in this
   ordering within each File.

7.3.  Specific File Writer Use Cases
  



Why not put this section as 7.2.1?

Some of my remarks above might be treated below...



  
   The specifications in this section apply to specific situations.
   Each section below extends or modifies the base File Writer
   specification in Section 7.2.  Considerations for collocation of a
   File Writer with IPFIX Collecting Processes and Metering Processes
   are given, as are specific guidelines for using IPFIX Files for
   archival storage, or as documents.  Also covered are the use of IPFIX
   Files in the testing and diagnostics of IPFIX Devices.

7.3.1.  Collocating a File Writer with a Collecting Process

   When collocating a File Writer with an IPFIX Collecting Process for
   archival storage of collected data in IPFIX Files as described in
   Section 6.1, the following recommendations may improve the usefulness
   of the stored data.

   The simplest way for a to store the data collected in a single
   Transport Session is to simply write the incoming IPFIX Messages to
   an IPFIX File as they are collected.  However, the resulting files
  



IPFIX File



  
   will lack information about the IPFIX Transport Session used to
   export them, such as the network addresses of the Exporting and
   Collecting Processes and the protocols used to transport them.  In
   this case, if information about the Transport Session is required,
   the File Writer SHOULD store a single IPFIX Transport Session in an
   IPFIX File and SHOULD record information about the Transport Session
   using the Export Session Details Options Template described in
   Section 8.1.3.

   Additional per-Message information MAY be recorded by the File Writer
   using the Message Details Options Template described in
   Section 8.1.4.  Per-message information includes the time at which
  



per-message or per-Message (2 lines above)



  
   each IPFIX Message was received at the Collecting Process, and can be
   used to resend IPFIX Messages while keeping the original measurement
   plane traffic profile.

   When collocating a File Writer with a Collecting Process, the Export
   Time of each Message SHOULD be the Export Time of the Message
   received by the Collecting Process containing the first Data Record
   in the Message.  Note that File Writers storing IPFIX data collected
   from an IPFIX Collecting Process using SCTP as the transport protocol
   SHOULD interleave messages from multiple streams in order to preserve
   Export Time order, and SHOULD reorder the written messages as
   necessary to ensure that each Template Set or Options Templatlly synonymous with a single Transport
   Session.  File Writers SHOULD store the Templates and Options
   required to decode the data within the File in the File itself, and
   File Readers SHOULD NOT use Templates or Options defined in one file
   to decode or interpret Data Sets in another.
  



So the IPFIX File should keep in memory the (Options) Template for that
active Transport Session in case where it lasts a long time, where
several IPFIX Files are required in order to insert the (Options)
Template in the subsequent IPFIX Files?

A simple example: [IPFIX-RED-RED] used across IPFIX Files



  
   File Writers SHOULD write IPFIX Messages within an IPFIX File in
   ascending Export Time order.

   File Writers MAY write Data Records to an IPFIX File in any order.
   However, File Writers that write flow records to an IPFIX File in



Trammell, et al.        Expires January 15, 2009               [Page 17]

Internet-Draft                 IPFIX Files                     July 2008


   flowStartTime or flowEndTime order SHOULD be consistent in this
   ordering within each File.

7.3.  Specific File Writer Use Cases
  



Why not put this section as 7.2.1?

Some of my remarks above might be treated below...



  
   The specifications in this section apply to specific situations.
   Each section below extends or modifies the base File Writer
   specification in Section 7.2.  Considerations for collocation of a
   File Writer with IPFIX Collecting Processes and Metering Processes
   are given, as are specific guidelines for using IPFIX Files for
   archival storage, or as documents.  Also covered are the use of IPFIX
   Files in the testing and diagnostics of IPFIX Devices.

7.3.1.  Collocating a File Writer with a Collecting Process

   When collocating a File Writer with an IPFIX Collecting Process for
   archival storage of collected data in IPFIX Files as described in
   Section 6.1, the following recommendations may improve the usefulness
   of the stored data.

   The simplest way for a to store the data collected in a single
   Transport Session is to simply write the incoming IPFIX Messages to
   an IPFIX File as they are collected.  However, the resulting files
  



IPFIX File



  
   will lack information about the IPFIX Transport Session used to
   export them, such as the network addresses of the Exporting and
   Collecting Processes and the protocols used to transport them.  In
   this case, if information about the Transport Session is required,
   the File Writer SHOULD store a single IPFIX Transport Session in an
   IPFIX File and SHOULD record information about the Transport Session
   using the Export Session Details Options Template described in
   Section 8.1.3.

   Additional per-Message information MAY be recorded by the File Writer
   using the Message Details Options Template described in
   Section 8.1.4.  Per-message information includes the time at which
  



per-message or per-Message (2 lines above)



  
   each IPFIX Message was received at the Collecting Process, and can be
   used to resend IPFIX Messages while keeping the original measurement
   plane traffic profile.

   When collocating a File Writer with a Collecting Process, the Export
   Time of each Message SHOULD be the Export Time of the Message
   received by the Collecting Process containing the first Data Record
   in the Message.  Note that File Writers storing IPFIX data collected
   from an IPFIX Collecting Process using SCTP as the transport protocol
   SHOULD interleave messages from multiple streams in order to preserve
   Export Time order, and SHOULD reorder the written messages as
   necessary to ensure that each Template Set or Options Temple Set
   appears in the file before any Data Set described by the Templates
  



That's very difficult without the [IPFIX-PER-STREAM] with PR-SCTP





  


Trammell, et al.        Expires January 15, 2009               [Page 18]

Internet-Draft                 IPFIX Files                     July 2008


   within that Set.

7.3.2.  Collocating a File Writer with a Metering Process

   Note that File Writers may also be collocated directly with IPFIX
   Metering Processes, for writing measured information directly to disk
   without intermediate IPFIX Exporting or Collecting Processes.  This
   arrangement may be particularly useful when providing data to an
   analysis environment with an IPFIX File based workflow, or when
   testing Metering Processes during development.

   When collocating a File Writer with a Metering Process, note that
   Information Elements associated with Exporting or Collecting
   Processes are meaningless, and SHOULD NOT appear in the Export
   Session Details Options Template described in Section 8.1.3 or the
   Message Details Options Template described in Section 8.1.4.

   When collocating a File Writer with an Exporting Process, the Export
   Time of each Message SHOULD be the time at which the first Data
   Record in the Message was received from the Exporting Process.
  



>From the EP or MP.

I would say EP, but your first sentence in this section says "without
intermediate IPFIX Exporting "


  
7.3.3.  Using IPFIX Files for Archival Storage

   While in the general case File Writers should store one Transport
   Session per IPFIX File, some applications storing large collections
   of data over long periods of time may benefit from the ability to
   treat a collection of IPFIX Files as a single Transport Session.  A
   File Writer MAY be configurable to write data from a single Transport
   Session into multiple IPFIX Files; however, File Writers supporting
   such a configuration option MUST provide a configuration option to
   support one-file-per-session behavior for interoperability purposes.

   File Writers compressing or encrypting archival data and File Readers
   reading compressed or encrypted archival data SHOULD follow the
   recommendations in Section 9.

7.3.4.  Using IPFIX Files as Documents

   When IPFIX Files are used as documents, to store a set of flows
   relevant to query, investigation, or other common context, or for the
   publication of flow data sets relevant to network research, each File
   MUST be readable as a single Transport Session, self-contained and
   making no reference to metadata stored in separate Files, in order to
   ensure interoperability.

   When writing Files to be used as documents, File Writers may emit the
   special Data Records described by Options Templates before any other
   Data Records in the File, in the following order, to ease the



Trammell, et al.        Expires January 15, 2009               [Page 19]

Internet-Draft                 IPFIX Files                     July 2008


   inspection and use of documents by File Readers:
  



I would use two sentences, otherwise it's confusing.

The may concerns "may emit" or "may emit in order".

Btw isn't it MAY?



  
   o  Time Window records described by the File Time Window Options
      Template as defined in Section 8.1.2 below; followed by

   o  commonPropertiesId definitions as described in "Reducing
      Redundancy in IPFIX and PSAMP Reports"
      [I-D.ietf-ipfix-reducing-redundancy]; followed by

   o  Information Element Type Records as described in "Exporting Type
      Information for IPFIX Information Elements"
      [I-D.ietf-ipfix-exporting-type]; followed by

   o  Export Session details records described by the Export Session
      Details Options Template as defined in ate Set
   appears in the file before any Data Set described by the Templates
  



That's very difficult without the [IPFIX-PER-STREAM] with PR-SCTP





  


Trammell, et al.        Expires January 15, 2009               [Page 18]

Internet-Draft                 IPFIX Files                     July 2008


   within that Set.

7.3.2.  Collocating a File Writer with a Metering Process

   Note that File Writers may also be collocated directly with IPFIX
   Metering Processes, for writing measured information directly to disk
   without intermediate IPFIX Exporting or Collecting Processes.  This
   arrangement may be particularly useful when providing data to an
   analysis environment with an IPFIX File based workflow, or when
   testing Metering Processes during development.

   When collocating a File Writer with a Metering Process, note that
   Information Elements associated with Exporting or Collecting
   Processes are meaningless, and SHOULD NOT appear in the Export
   Session Details Options Template described in Section 8.1.3 or the
   Message Details Options Template described in Section 8.1.4.

   When collocating a File Writer with an Exporting Process, the Export
   Time of each Message SHOULD be the time at which the first Data
   Record in the Message was received from the Exporting Process.
  



>From the EP or MP.

I would say EP, but your first sentence in this section says "without
intermediate IPFIX Exporting "


  
7.3.3.  Using IPFIX Files for Archival Storage

   While in the general case File Writers should store one Transport
   Session per IPFIX File, some applications storing large collections
   of data over long periods of time may benefit from the ability to
   treat a collection of IPFIX Files as a single Transport Session.  A
   File Writer MAY be configurable to write data from a single Transport
   Session into multiple IPFIX Files; however, File Writers supporting
   such a configuration option MUST provide a configuration option to
   support one-file-per-session behavior for interoperability purposes.

   File Writers compressing or encrypting archival data and File Readers
   reading compressed or encrypted archival data SHOULD follow the
   recommendations in Section 9.

7.3.4.  Using IPFIX Files as Documents

   When IPFIX Files are used as documents, to store a set of flows
   relevant to query, investigation, or other common context, or for the
   publication of flow data sets relevant to network research, each File
   MUST be readable as a single Transport Session, self-contained and
   making no reference to metadata stored in separate Files, in order to
   ensure interoperability.

   When writing Files to be used as documents, File Writers may emit the
   special Data Records described by Options Templates before any other
   Data Records in the File, in the following order, to ease the



Trammell, et al.        Expires January 15, 2009               [Page 19]

Internet-Draft                 IPFIX Files                     July 2008


   inspection and use of documents by File Readers:
  



I would use two sentences, otherwise it's confusing.

The may concerns "may emit" or "may emit in order".

Btw isn't it MAY?



  
   o  Time Window records described by the File Time Window Options
      Template as defined in Section 8.1.2 below; followed by

   o  commonPropertiesId definitions as described in "Reducing
      Redundancy in IPFIX and PSAMP Reports"
      [I-D.ietf-ipfix-reducing-redundancy]; followed by

   o  Information Element Type Records as described in "Exporting Type
      Information for IPFIX Information Elements"
      [I-D.ietf-ipfix-exporting-type]; followed by

   o  Export Session details records described by the Export Session
      Details Options Template as defined iSection 8.1.3 below.

   The Export Time of each Message within a File used as a document
   SHOULD be the time at which the Message was written by the File
   Writer.

   If an IPFIX File used as a document uses the technique described in
   "Reducing Redundancy in IPFIX and PSAMP Reports"
   [I-D.ietf-ipfix-reducing-redundancy] AND all of the non-Options
   Templates in the File contain the commonPropertiesId Information
   Element, a File Reader MAY assume the set of commonPropertiesId
   definitions provides a complete table of contents for the File for
   searching purposes.

7.3.5.  Using IPFIX Files for Testing

   IPFIX Files can be used for testing IPFIX Collecting Processes in two
   ways.  First, IPFIX Files can be used to store specific flow data for
   regression and stress testing of Collectors; there are no special
   considerations for IPFIX Files used in this way.

   Second, IPFIX Files are useful for storing reference messages which
   do not comply to the IPFIX Protocol in order to test the error
   handling and recovery behavior of Collectors.  Of course, IPFIX Files
   intended to be used in this application necessarily MAY violate any
   of of the specifications in this document or in [RFC5101], and such
  



of



  
   files MUST NOT be transmitted to Collecting Processes or given as
   input File Readers not under test.

   Note that an extremely simple IPFIX Exporting Process may be crafted
   for testing purposes by simply reading an IPFIX File and transmitting
   it directly to a Collecting Process.  Similarly, an extremely simple
   Collecting Process may be crafted for testing purposes by simply
   accepting connections and/or IPFIX Messages from Exporting Processes



Trammell, et al.        Expires January 15, 2009               [Page 20]

Internet-Draft                 IPFIX Files                     July 2008


   and writing the session's message stream to an IPFIX File.

7.3.6.  Writing IPFIX Files for Device Diagnostics

   IPFIX Files can be used in the debugging of devices which use flow
   data as internal state, as a common format for the representation of
   flow tables.  In such situations, the opaueOctets information element
   can be used to store additional non-IPFIX encoded, non-flow
   information (e.g., stack backtraces, process state, etc.) within the
   IPFIX File as in Section 10.1; the IPFIX flow table information could
   also be embedded in a larger proprietary diagnostic format using
   delimiters as in Section 10.2


8.  File Format Metadata Specification

   This section defines the Options Templates used for IPFIX File
   metadata, and the Information Elements they require.

8.1.  Recommended Options Templates for IPFIX Files

   The following Options Templates allow IPFIX Message streams to meet
   the requirements outlined above without extension to the message
   format or protocol.  They are defined in terms of existing
   Information Elements defined in [RFC5102], the Information Elements
   defined in "Exporting Type Information for IPFIX Information
   Elements" [I-D.ietf-ipfix-exporting-type], as well as Information
   Elements defined in Section 8.2.  IPFIX File Readers and Writers
   SHOULD support these options templates as defined below.
  



There is a SHOULD here. See my comments about may versus MAY above.



  
   In addition, IPFIX File Readers and Writers SHOULD support the
   Options Templates defined in "Exporting Type Information for IPFIX
   Information Elements" [I-D.ietf-ipfix-exporting-type] in order to
   support self-description of enterprise-specific Information Elements.

8.1.1.  Message Checksum Options Template

   The Message Checksum Options Template specifies the structure of a
   Data Record for attaching an MD5 message checksum to an IPFIX
   Message.  An MD5 message checksum as described MAY be used if long-
   term data integrity is importan Section 8.1.3 below.

   The Export Time of each Message within a File used as a document
   SHOULD be the time at which the Message was written by the File
   Writer.

   If an IPFIX File used as a document uses the technique described in
   "Reducing Redundancy in IPFIX and PSAMP Reports"
   [I-D.ietf-ipfix-reducing-redundancy] AND all of the non-Options
   Templates in the File contain the commonPropertiesId Information
   Element, a File Reader MAY assume the set of commonPropertiesId
   definitions provides a complete table of contents for the File for
   searching purposes.

7.3.5.  Using IPFIX Files for Testing

   IPFIX Files can be used for testing IPFIX Collecting Processes in two
   ways.  First, IPFIX Files can be used to store specific flow data for
   regression and stress testing of Collectors; there are no special
   considerations for IPFIX Files used in this way.

   Second, IPFIX Files are useful for storing reference messages which
   do not comply to the IPFIX Protocol in order to test the error
   handling and recovery behavior of Collectors.  Of course, IPFIX Files
   intended to be used in this application necessarily MAY violate any
   of of the specifications in this document or in [RFC5101], and such
  



of



  
   files MUST NOT be transmitted to Collecting Processes or given as
   input File Readers not under test.

   Note that an extremely simple IPFIX Exporting Process may be crafted
   for testing purposes by simply reading an IPFIX File and transmitting
   it directly to a Collecting Process.  Similarly, an extremely simple
   Collecting Process may be crafted for testing purposes by simply
   accepting connections and/or IPFIX Messages from Exporting Processes



Trammell, et al.        Expires January 15, 2009               [Page 20]

Internet-Draft                 IPFIX Files                     July 2008


   and writing the session's message stream to an IPFIX File.

7.3.6.  Writing IPFIX Files for Device Diagnostics

   IPFIX Files can be used in the debugging of devices which use flow
   data as internal state, as a common format for the representation of
   flow tables.  In such situations, the opaueOctets information element
   can be used to store additional non-IPFIX encoded, non-flow
   information (e.g., stack backtraces, process state, etc.) within the
   IPFIX File as in Section 10.1; the IPFIX flow table information could
   also be embedded in a larger proprietary diagnostic format using
   delimiters as in Section 10.2


8.  File Format Metadata Specification

   This section defines the Options Templates used for IPFIX File
   metadata, and the Information Elements they require.

8.1.  Recommended Options Templates for IPFIX Files

   The following Options Templates allow IPFIX Message streams to meet
   the requirements outlined above without extension to the message
   format or protocol.  They are defined in terms of existing
   Information Elements defined in [RFC5102], the Information Elements
   defined in "Exporting Type Information for IPFIX Information
   Elements" [I-D.ietf-ipfix-exporting-type], as well as Information
   Elements defined in Section 8.2.  IPFIX File Readers and Writers
   SHOULD support these options templates as defined below.
  



There is a SHOULD here. See my comments about may versus MAY above.



  
   In addition, IPFIX File Readers and Writers SHOULD support the
   Options Templates defined in "Exporting Type Information for IPFIX
   Information Elements" [I-D.ietf-ipfix-exporting-type] in order to
   support self-description of enterprise-specific Information Elements.

8.1.1.  Message Checksum Options Template

   The Message Checksum Options Template specifies the structure of a
   Data Record for attaching an MD5 message checksum to an IPFIX
   Message.  An MD5 message checksum as described MAY be used if long-
   term data integrity is impornt to the application.  The described
   Data Record MUST appear only once per IPFIX Message, but MAY appear
   anywhere within the Message. 
  



We observed an IPFIX Message, with its own length. Now, we have to add
the Message Checksum Options Template.

- Is this a new IPFIX Message? Then we have to say if this comes after
or before the IPFIX Message it refers to.

- Or we have to modify the observed IPFIX Message: changing the length
of the IPFIX Message header. If this is the case (and I guess it is,
but it's not clearly mentioned), then there is a problem. I export an
IPFIX Message with the maximum length, the IPFIX Writer adds this
Options Template before writing it into the file. Now, I transfer this
file. Will the other IPFIX Reader reject it because it's not IPFIX
compliant, i.e. too big?

In other words, should the File Writer strictly follow RFC5101 or not? 





  
   The template SHOULD contain the following Information Elements:






Trammell, et al.        Expires January 15, 2009               [Page 21]

Internet-Draft                 IPFIX Files                     July 2008


   +--------------------+----------------------------------------------+
   | IE                 | Description                                  |
   +--------------------+----------------------------------------------+
   | messageScope       | A marker denoting this Option applies to the |
   | [scope]            | whole IPFIX Message; content is ignored.     |
   |                    | This Information Element MUST be defined as  |
   |                    | a Scope Field.                               |
   | messageMD5Checksum | The MD5 checksum of the containing IPFIX     |
   |                    | Message.                                     |
   +--------------------+----------------------------------------------+

8.1.2.  File Time Window Options Template

   The File Time Window Options Template specifies the structure of a
   Data Record for attaching a time window to an IPFIX File; this Data
   Record is referred to as a time window record.  A time window record
   defines the earliest flow start time and the latest flow end time of
   the flow records within a File.  One and only one time window record
   MAY appear within an IPFIX File if the time window information is
   available; a File Writer MUST NOT write more than one time window
   record to an IPFIX File.  A File Writer that writes a time window
   record to a File MUST NOT write any Flow with a start time before the
   beginning of the window or an end time after the end of the window to
   that File.
  



Where should we add this one?

- in a new IPFIX Message?

- in an existing one? If yes, which one?



  
   The template SHOULD contain the following Information Elements:

   +---------------------+---------------------------------------------+
   | IE                  | Description                                 |
   +---------------------+---------------------------------------------+
   | sessionScope        | A marker denoting this Option applies to    |
   | [scope]             | the whole IPFIX Transport Session (i.e.,    |
   |                     | IPFIX File); content is ignored.  This      |
  



Why do you say: Transport Session and not IPFIX File?



  
   |                     | Information Element MUST be defined as a    |
   |                     | Scope Field.                                |
   | minFlowStartSeconds | The start time of the earliest flow in the  |
   |                     | Transport Session (i.e., File) in epoch     |
   |                     | seconds.                                    |
   | maxFlowEndSeconds   | The end time of the latest flow in the      |
   |                     | Transport Session (i.e., File) in epoctant to the application.  The described
   Data Record MUST appear only once per IPFIX Message, but MAY appear
   anywhere within the Message. 
  



We observed an IPFIX Message, with its own length. Now, we have to add
the Message Checksum Options Template.

- Is this a new IPFIX Message? Then we have to say if this comes after
or before the IPFIX Message it refers to.

- Or we have to modify the observed IPFIX Message: changing the length
of the IPFIX Message header. If this is the case (and I guess it is,
but it's not clearly mentioned), then there is a problem. I export an
IPFIX Message with the maximum length, the IPFIX Writer adds this
Options Template before writing it into the file. Now, I transfer this
file. Will the other IPFIX Reader reject it because it's not IPFIX
compliant, i.e. too big?

In other words, should the File Writer strictly follow RFC5101 or not? 





  
   The template SHOULD contain the following Information Elements:






Trammell, et al.        Expires January 15, 2009               [Page 21]

Internet-Draft                 IPFIX Files                     July 2008


   +--------------------+----------------------------------------------+
   | IE                 | Description                                  |
   +--------------------+----------------------------------------------+
   | messageScope       | A marker denoting this Option applies to the |
   | [scope]            | whole IPFIX Message; content is ignored.     |
   |                    | This Information Element MUST be defined as  |
   |                    | a Scope Field.                               |
   | messageMD5Checksum | The MD5 checksum of the containing IPFIX     |
   |                    | Message.                                     |
   +--------------------+----------------------------------------------+

8.1.2.  File Time Window Options Template

   The File Time Window Options Template specifies the structure of a
   Data Record for attaching a time window to an IPFIX File; this Data
   Record is referred to as a time window record.  A time window record
   defines the earliest flow start time and the latest flow end time of
   the flow records within a File.  One and only one time window record
   MAY appear within an IPFIX File if the time window information is
   available; a File Writer MUST NOT write more than one time window
   record to an IPFIX File.  A File Writer that writes a time window
   record to a File MUST NOT write any Flow with a start time before the
   beginning of the window or an end time after the end of the window to
   that File.
  



Where should we add this one?

- in a new IPFIX Message?

- in an existing one? If yes, which one?



  
   The template SHOULD contain the following Information Elements:

   +---------------------+---------------------------------------------+
   | IE                  | Description                                 |
   +---------------------+---------------------------------------------+
   | sessionScope        | A marker denoting this Option applies to    |
   | [scope]             | the whole IPFIX Transport Session (i.e.,    |
   |                     | IPFIX File); content is ignored.  This      |
  



Why do you say: Transport Session and not IPFIX File?



  
   |                     | Information Element MUST be defined as a    |
   |                     | Scope Field.                                |
   | minFlowStartSeconds | The start time of the earliest flow in the  |
   |                     | Transport Session (i.e., File) in epoch     |
   |                     | seconds.                                    |
   | maxFlowEndSeconds   | The end time of the latest flow in the      |
   |                     | Transport Session (i.e., File) in eph     |
   |                     | seconds.                                    |
   +---------------------+---------------------------------------------+
  



>From our email exchange:




Benoit: - 7.1.2 and 7.1.3. Why
minFlowStartSeconds, maxFlowStartSeconds,
minExportSeconds, maxExportSeconds? Why not any of the time related
I.E.? Not to duplicate the I.Es?







Brian: Basically,
yes; the idea is that when you're using the first two IEs to attach
time windows to files, second resolution is sufficient, and there's no
need to get more precise than that. This may be an overly restrictive
assumption. The export time window information, however, is explicitly
in seconds, because it describes the IPFIX Export Time, which is also
in seconds.





My feedback is:

- it's not specified that you meant the IPFIX Export Time.

- "there's no
need to get more precise than that". I disagree, the answer is: it
depends.







  
8.1.3.  Export Session Details Options Template

   The Export Session Details Options Template specifies the structure
   of a Data Record for recording the details of an IPFIX Transport
   Session in an IPFIX File.  It is intended for use in storing a single



Trammell, et al.        Expires January 15, 2009               [Page 22]

Internet-Draft                 IPFIX Files                     July 2008


   complete IPFIX Transport Session in a single IPFIX File.  The
   described Data Record SHOULD appear only once in a given IPFIX File.
  



Same remark: where?



  
   The template SHOULD contain the following Information Elements,
   subject to applicability as noted on each Information Element:
  



MAY contain more.

For example, new I.E. about SCTP streams number, or stream ID, or
whatever in the future.



  
   +----------------------------+--------------------------------------+
   | IE                         | Description                          |
   +----------------------------+--------------------------------------+
   | sessionScope [scope]       | A marker denoting this Option        |
   |                            | applies to the whole IPFIX Transport |
   |                            | Session (i.e., IPFIX File); content  |
   |                            | is ignored.  This Information        |
   |                            | Element MUST be defined as a Scope   |
   |                            | Field.                               |
   | exporterIPv4Address        | IPv4 address of the IPFIX Exporting  |
   |                            | Process from which the Messages in   |
   |                            | this Transport Session were          |
   |                            | received.  Present only for          |
   |                            | Exporting Processes with an IPv4     |
   |                            | interface.  For multi-homed SCTP     |
   |                            | associations, this SHOULD be the     |
   |                            | primary path endpoint address of the |
   |                            | Exporting Process.                   |
   | exporterIPv6Address        | IPv6 address of the IPFIX Exporting  |
   |                            | Process from which the Messages in   |
   |                            | this Transport Session were          |
   |                            | received.  Present only for          |
   |                            | Exporting Processes with an IPv6     |
   |                            | interface.  For multi-homed SCTP     |
   |                            | associations, this SHOULD be the     |
   |                            | primary path endpoint address of the |
   |                 och     |
   |                     | seconds.                                    |
   +---------------------+---------------------------------------------+
  



>From our email exchange:




Benoit: - 7.1.2 and 7.1.3. Why
minFlowStartSeconds, maxFlowStartSeconds,
minExportSeconds, maxExportSeconds? Why not any of the time related
I.E.? Not to duplicate the I.Es?







Brian: Basically,
yes; the idea is that when you're using the first two IEs to attach
time windows to files, second resolution is sufficient, and there's no
need to get more precise than that. This may be an overly restrictive
assumption. The export time window information, however, is explicitly
in seconds, because it describes the IPFIX Export Time, which is also
in seconds.





My feedback is:

- it's not specified that you meant the IPFIX Export Time.

- "there's no
need to get more precise than that". I disagree, the answer is: it
depends.







  
8.1.3.  Export Session Details Options Template

   The Export Session Details Options Template specifies the structure
   of a Data Record for recording the details of an IPFIX Transport
   Session in an IPFIX File.  It is intended for use in storing a single



Trammell, et al.        Expires January 15, 2009               [Page 22]

Internet-Draft                 IPFIX Files                     July 2008


   complete IPFIX Transport Session in a single IPFIX File.  The
   described Data Record SHOULD appear only once in a given IPFIX File.
  



Same remark: where?



  
   The template SHOULD contain the following Information Elements,
   subject to applicability as noted on each Information Element:
  



MAY contain more.

For example, new I.E. about SCTP streams number, or stream ID, or
whatever in the future.



  
   +----------------------------+--------------------------------------+
   | IE                         | Description                          |
   +----------------------------+--------------------------------------+
   | sessionScope [scope]       | A marker denoting this Option        |
   |                            | applies to the whole IPFIX Transport |
   |                            | Session (i.e., IPFIX File); content  |
   |                            | is ignored.  This Information        |
   |                            | Element MUST be defined as a Scope   |
   |                            | Field.                               |
   | exporterIPv4Address        | IPv4 address of the IPFIX Exporting  |
   |                            | Process from which the Messages in   |
   |                            | this Transport Session were          |
   |                            | received.  Present only for          |
   |                            | Exporting Processes with an IPv4     |
   |                            | interface.  For multi-homed SCTP     |
   |                            | associations, this SHOULD be the     |
   |                            | primary path endpoint address of the |
   |                            | Exporting Process.                   |
   | exporterIPv6Address        | IPv6 address of the IPFIX Exporting  |
   |                            | Process from which the Messages in   |
   |                            | this Transport Session were          |
   |                            | received.  Present only for          |
   |                            | Exporting Processes with an IPv6     |
   |                            | interface.  For multi-homed SCTP     |
   |                            | associations, this SHOULD be the     |
   |                            | primary path endpoint address of the |
   |                          | Exporting Process.                   |
   | exporterTransportPort      | The source port from which the       |
   |                            | Messages in this Transport Session   |
   |                            | were received.                       |
   | collectorIPv4Address       | IPv4 address of the IPFIX Collecting |
   |                            | Process which received the Messages  |
   |                            | in this Transport Session.  Present  |
   |                            | only for Collecting Processes with   |
   |                            | an IPv4 interface.  For multi-homed  |
   |                            | SCTP associations, this SHOULD be    |
   |                            | the primary path endpoint address of |
   |                            | the Collecting Process.              |







Trammell, et al.        Expires January 15, 2009               [Page 23]

Internet-Draft                 IPFIX Files                     July 2008


   | collectorIPv6Address       | IPv6 address of the IPFIX Collecting |
   |                            | Process which received the Messages  |
   |                            | in this Transport Session.  Present  |
   |                            | only for Collecting Processes with   |
   |                            | an IPv6 interface.  For multi-homed  |
   |                            | SCTP associations, this SHOULD be    |
   |                            | the primary path endpoint address of |
   |                            | the Collecting Process.              |
   | collectorTransportPort     | The destination port on which the    |
   |                            | Messages in this Transport Session   |
   |                            | were received.                       |
   | collectorTransportProtocol | The IP Protocol Identifier of the    |
   |                            | transport protocol used to transport |
   |                            | Messages within this Transport       |
   |                            | Session.                             |
   | collectorProtocolVersion   | The version of the IPFIX Protocol    |
   |                            | used to transport Messages within    |
   |                            | this Transport Session.              |
   | minExportSeconds           | The Export Time of the first Message |
   |                            | in the Transport Session.            |
   | maxExportSeconds           | The Export Time of the last Message  |
   |                            | in the Transport Session.            |
   +----------------------------+--------------------------------------+

8.1.4.  Message Details Options Template

   The Message Details Options Template specifies the structure of a
   Data Record for attaching additional export details to an IPFIX
   Message.  These details include the time at which a message was
  



IPFIX Message?



  
   received and information about the export and collection
   infrastructure used to transport the Message.  This Options Template
   also allows the storage of the export session metadata provided the
   Export Session Details Options Template, for storing information from
   multiple Transport Sessions in the same IPFIX File.

   The template SHOULD contain the following Information Elements,
   subject to applicability as noted for each Information Element.  Note
   that when used in conjunction with the Export Session Details Options
   Template, when storing a single complete IPFIX Transport Session in
   an IPFIX File, this template SHOULD contain only the messageScope and
   collectionTimeMilliseconds Information Elements, and the
   exportSctpStreamId Information Element for Messages transported via
   SCTP.








Trammell, et al.        Expires January 15, 2009               [Page 24]

Internet-Draft                 IPFIX Files                     July 2008


   +----------------------------+--------------------             | Exporting Process.                   |
   | exporterTransportPort      | The source port from which the       |
   |                            | Messages in this Transport Session   |
   |                            | were received.                       |
   | collectorIPv4Address       | IPv4 address of the IPFIX Collecting |
   |                            | Process which received the Messages  |
   |                            | in this Transport Session.  Present  |
   |                            | only for Collecting Processes with   |
   |                            | an IPv4 interface.  For multi-homed  |
   |                            | SCTP associations, this SHOULD be    |
   |                            | the primary path endpoint address of |
   |                            | the Collecting Process.              |







Trammell, et al.        Expires January 15, 2009               [Page 23]

Internet-Draft                 IPFIX Files                     July 2008


   | collectorIPv6Address       | IPv6 address of the IPFIX Collecting |
   |                            | Process which received the Messages  |
   |                            | in this Transport Session.  Present  |
   |                            | only for Collecting Processes with   |
   |                            | an IPv6 interface.  For multi-homed  |
   |                            | SCTP associations, this SHOULD be    |
   |                            | the primary path endpoint address of |
   |                            | the Collecting Process.              |
   | collectorTransportPort     | The destination port on which the    |
   |                            | Messages in this Transport Session   |
   |                            | were received.                       |
   | collectorTransportProtocol | The IP Protocol Identifier of the    |
   |                            | transport protocol used to transport |
   |                            | Messages within this Transport       |
   |                            | Session.                             |
   | collectorProtocolVersion   | The version of the IPFIX Protocol    |
   |                            | used to transport Messages within    |
   |                            | this Transport Session.              |
   | minExportSeconds           | The Export Time of the first Message |
   |                            | in the Transport Session.            |
   | maxExportSeconds           | The Export Time of the last Message  |
   |                            | in the Transport Session.            |
   +----------------------------+--------------------------------------+

8.1.4.  Message Details Options Template

   The Message Details Options Template specifies the structure of a
   Data Record for attaching additional export details to an IPFIX
   Message.  These details include the time at which a message was
  



IPFIX Message?



  
   received and information about the export and collection
   infrastructure used to transport the Message.  This Options Template
   also allows the storage of the export session metadata provided the
   Export Session Details Options Template, for storing information from
   multiple Transport Sessions in the same IPFIX File.

   The template SHOULD contain the following Information Elements,
   subject to applicability as noted for each Information Element.  Note
   that when used in conjunction with the Export Session Details Options
   Template, when storing a single complete IPFIX Transport Session in
   an IPFIX File, this template SHOULD contain only the messageScope and
   collectionTimeMilliseconds Information Elements, and the
   exportSctpStreamId Information Element for Messages transported via
   SCTP.








Trammell, et al.        Expires January 15, 2009               [Page 24]

Internet-Draft                 IPFIX Files                     July 2008


   +----------------------------+------------------------------------+
   | IE                         | Description                          |
   +----------------------------+--------------------------------------+
   | messageScope [scope]       | A marker denoting this Option        |
   |                            | applies to the whole IPFIX message;  |
   |                            | content is ignored.  This            |
   |                            | Information Element MUST be defined  |
   |                            | as a Scope Field.                    |
   | collectionTimeMilliseconds | The absolute time at which this      |
   |                            | Message was received by the IPFIX    |
   |                            | Collecting Process.                  |
  



same remark about the timing. Why not be flexible.



  
   | exporterIPv4Address        | IPv4 address of the IPFIX Exporting  |
   |                            | Process from which this Message was  |
   |                            | received.  Present only for          |
   |                            | Exporting Processes with an IPv4     |
   |                            | interface, and if this information   |
   |                            | is not available via the Export      |
   |                            | Session Details Options Template.    |
   |                            | For multi-homed SCTP associations,   |
   |                            | this SHOULD be the primary path      |
   |                            | endpoint address of the Exporting    |
   |                            | Process.                             |
   | exporterIPv6Address        | IPv6 address of the IPFIX Exporting  |
   |                            | Process this Message was received.   |
   |                            | Present only for Exporting Processes |
   |                            | with an IPv6 interface, and if this  |
   |                            | information is not available via the |
   |                            | Export Session Details Options       |
   |                            | Template.  For multi-homed SCTP      |
   |                            | associations, this SHOULD be the     |
   |                            | primary path endpoint address of the |
   |                            | Exporting Process.                   |
   | exporterTransportPort      | The source port from which this      |
   |                            | Message received.  Present only if   |
   |                            | this information is not available    |
   |                            | via the Export Session Details       |
   |                            | Options Template.                    |
   | collectorIPv4Address       | IPv4 address of the IPFIX Collecting |
   |                            | Process which received this Message. |
   |                            | Present only for Collecting          |
   |                            | Processes with an IPv4 interface,    |
   |                            | and if this information is not       |
   |                            | available via the Export Session     |
   |                            | Details Options Template.  For       |
   |                            | multi-homed SCTP associations, this  |
   |                            | SHOULD be the primary path endpoint  |
   |                            | address of the Collecting Process.   |




Trammell, et al.        Expires January 15, 2009               [Page 25]

Internet-Draft                 IPFIX Files                     July 2008


   | collectorIPv6Address       | IPv6 address of the IPFIX Collecting |
   |                            | Process which received this Message. |
   |                            | Present only for Collecting          |
   |                            | Processes with an IPv6 interface,    |
   |                            | and if this information is not       |
   |                            | available --------------------+
   | IE                         | Description                          |
   +----------------------------+--------------------------------------+
   | messageScope [scope]       | A marker denoting this Option        |
   |                            | applies to the whole IPFIX message;  |
   |                            | content is ignored.  This            |
   |                            | Information Element MUST be defined  |
   |                            | as a Scope Field.                    |
   | collectionTimeMilliseconds | The absolute time at which this      |
   |                            | Message was received by the IPFIX    |
   |                            | Collecting Process.                  |
  



same remark about the timing. Why not be flexible.



  
   | exporterIPv4Address        | IPv4 address of the IPFIX Exporting  |
   |                            | Process from which this Message was  |
   |                            | received.  Present only for          |
   |                            | Exporting Processes with an IPv4     |
   |                            | interface, and if this information   |
   |                            | is not available via the Export      |
   |                            | Session Details Options Template.    |
   |                            | For multi-homed SCTP associations,   |
   |                            | this SHOULD be the primary path      |
   |                            | endpoint address of the Exporting    |
   |                            | Process.                             |
   | exporterIPv6Address        | IPv6 address of the IPFIX Exporting  |
   |                            | Process this Message was received.   |
   |                            | Present only for Exporting Processes |
   |                            | with an IPv6 interface, and if this  |
   |                            | information is not available via the |
   |                            | Export Session Details Options       |
   |                            | Template.  For multi-homed SCTP      |
   |                            | associations, this SHOULD be the     |
   |                            | primary path endpoint address of the |
   |                            | Exporting Process.                   |
   | exporterTransportPort      | The source port from which this      |
   |                            | Message received.  Present only if   |
   |                            | this information is not available    |
   |                            | via the Export Session Details       |
   |                            | Options Template.                    |
   | collectorIPv4Address       | IPv4 address of the IPFIX Collecting |
   |                            | Process which received this Message. |
   |                            | Present only for Collecting          |
   |                            | Processes with an IPv4 interface,    |
   |                            | and if this information is not       |
   |                            | available via the Export Session     |
   |                            | Details Options Template.  For       |
   |                            | multi-homed SCTP associations, this  |
   |                            | SHOULD be the primary path endpoint  |
   |                            | address of the Collecting Process.   |




Trammell, et al.        Expires January 15, 2009               [Page 25]

Internet-Draft                 IPFIX Files                     July 2008


   | collectorIPv6Address       | IPv6 address of the IPFIX Collecting |
   |                            | Process which received this Message. |
   |                            | Present only for Collecting          |
   |                            | Processes with an IPv6 interface,    |
   |                            | and if this information is not       |
   |                            | availablvia the Export Session     |
   |                            | Details Options Template.  For       |
   |                            | multi-homed SCTP associations, this  |
   |                            | SHOULD be the primary path endpoint  |
   |                            | address of the Collecting Process.   |
   | collectorTransportPort     | The destination port on which this   |
   |                            | Message was received.  Present only  |
   |                            | if this information is not available |
   |                            | via the Export Session Details       |
   |                            | Options Template.                    |
   | collectorTransportProtocol | The IP Protocol Identifier of the    |
   |                            | transport protocol used to transport |
   |                            | this Message.  Present only if this  |
   |                            | information is not available via the |
   |                            | Export Session Details Options       |
   |                            | Template.                            |
   | collectorProtocolVersion   | The version of the IPFIX Protocol    |
   |                            | used to transport this Message.      |
   |                            | Present only if this information is  |
   |                            | not available via the Export Session |
   |                            | Details Options Template.            |
   | exportSctpStreamId         | The SCTP stream used to transport    |
   |                            | this Message.  Present only if the   |
   |                            | Message was transported via SCTP.    |
   +----------------------------+--------------------------------------+

8.2.  Recommended Information Elements for IPFIX Files

   The following Information Elements are used by the options templates
   in Section 8.1 to allow IPFIX Message streams to meet the
   requirements outlined above without extension of the message format
   or protocol.  IPFIX File Readers and Writers SHOULD support these
   Information Elements as defined below.

   In addition, IPFIX File Readers and Writers SHOULD support the
   Information Elements defined in "Exporting Type Information for IPFIX
   Information Elements" [I-D.ietf-ipfix-exporting-type] in order to
   support full self-description of Information Elements.

8.2.1.  collectionTimeMilliseconds






Trammell, et al.        Expires January 15, 2009               [Page 26]

Internet-Draft                 IPFIX Files                     July 2008


   Description:   The absolute timestamp at which the data within the
      scope containing this Information Element was received by a
      Collecting Process.  This Information Element SHOULD be bound to
      its containing IPFIX Message via IPFIX Options and the
      messageScope Information Element, as defined below.

   Abstract Data Type:   dateTimeMilliseconds

   ElementId:   TBD1

   Status:   current

8.2.2.  exportSctpStreamId

   Description:   The value of the SCTP Stream Identifier used by the
      Exporting Process for exporting IPFIX Message data.  This is
      carried in the Stream Identifier field of the header of the SCTP
      DATA chunk containing the IPFIX Message(s).

   Abstract Data Type:   unsigned16

   Data Type Semantics:   identifier

   ElementId:   TBD2

   Status:   current

8.2.3.  maxExportSeconds

   Description:   The absolute Export Time of the latest IPFIX Message
      within the scope containing this Information Element.  This
      Information Element SHOULD be bound to its containing IPFIX
      Transport Session (i.e., File) via an options record and the
      sessionScope Information Element, as defined below, and SHOULD
      appear only once in a given IPFIX File.

   Abstract Data Type:   dateTimeSeconds

   ElementId:   TBD3

   Status:   current

   Units:   seconds








Trammell, et al.        Expires January 15, 2009               [Page 27]

Internet-Draft                 IPFIX Files                 e via the Export Session     |
   |                            | Details Options Template.  For       |
   |                            | multi-homed SCTP associations, this  |
   |                            | SHOULD be the primary path endpoint  |
   |                            | address of the Collecting Process.   |
   | collectorTransportPort     | The destination port on which this   |
   |                            | Message was received.  Present only  |
   |                            | if this information is not available |
   |                            | via the Export Session Details       |
   |                            | Options Template.                    |
   | collectorTransportProtocol | The IP Protocol Identifier of the    |
   |                            | transport protocol used to transport |
   |                            | this Message.  Present only if this  |
   |                            | information is not available via the |
   |                            | Export Session Details Options       |
   |                            | Template.                            |
   | collectorProtocolVersion   | The version of the IPFIX Protocol    |
   |                            | used to transport this Message.      |
   |                            | Present only if this information is  |
   |                            | not available via the Export Session |
   |                            | Details Options Template.            |
   | exportSctpStreamId         | The SCTP stream used to transport    |
   |                            | this Message.  Present only if the   |
   |                            | Message was transported via SCTP.    |
   +----------------------------+--------------------------------------+

8.2.  Recommended Information Elements for IPFIX Files

   The following Information Elements are used by the options templates
   in Section 8.1 to allow IPFIX Message streams to meet the
   requirements outlined above without extension of the message format
   or protocol.  IPFIX File Readers and Writers SHOULD support these
   Information Elements as defined below.

   In addition, IPFIX File Readers and Writers SHOULD support the
   Information Elements defined in "Exporting Type Information for IPFIX
   Information Elements" [I-D.ietf-ipfix-exporting-type] in order to
   support full self-description of Information Elements.

8.2.1.  collectionTimeMilliseconds






Trammell, et al.        Expires January 15, 2009               [Page 26]

Internet-Draft                 IPFIX Files                     July 2008


   Description:   The absolute timestamp at which the data within the
      scope containing this Information Element was received by a
      Collecting Process.  This Information Element SHOULD be bound to
      its containing IPFIX Message via IPFIX Options and the
      messageScope Information Element, as defined below.

   Abstract Data Type:   dateTimeMilliseconds

   ElementId:   TBD1

   Status:   current

8.2.2.  exportSctpStreamId

   Description:   The value of the SCTP Stream Identifier used by the
      Exporting Process for exporting IPFIX Message data.  This is
      carried in the Stream Identifier field of the header of the SCTP
      DATA chunk containing the IPFIX Message(s).

   Abstract Data Type:   unsigned16

   Data Type Semantics:   identifier

   ElementId:   TBD2

   Status:   current

8.2.3.  maxExportSeconds

   Description:   The absolute Export Time of the latest IPFIX Message
      within the scope containing this Information Element.  This
      Information Element SHOULD be bound to its containing IPFIX
      Transport Session (i.e., File) via an options record and the
      sessionScope Information Element, as defined below, and SHOULD
      appear only once in a given IPFIX File.

   Abstract Data Type:   dateTimeSeconds

   ElementId:   TBD3

   Status:   current

   Units:   seconds








Trammell, et al.        Expires January 15, 2009               [Page 27]

Internet-Draft                 IPFIX Files                   July 2008


8.2.4.  maxFlowEndSeconds

   Description:   The latest absolute timestamp of the last packet
      within any Flow within the scope containing this Information
      Element, rounded up to the second.  This Information Element
      SHOULD be bound to its containing IPFIX Transport Session (i.e.,
      File) via an IPFIX Options and the sessionScope Information
      Element, as defined below, and SHOULD appear only once in a given
      IPFIX File.

   Abstract Data Type:   dateTimeSeconds

   ElementId:   TBD4

   Status:   current

   Units:   seconds

8.2.5.  messageMD5Checksum

   Description:   The MD5 checksum of the IPFIX Message containing this
      record.  This Information Element SHOULD be bound to its
      containing IPFIX Message via an options record and the
      messageScope Information Element, as defined below, and SHOULD
      appear only once in a given IPFIX Message.  To calculate the value
      of this Information Element, first buffer the containing IPFIX
      Message, setting the value of this Information Element to all
      zeroes.  Then caluclate the MD5 checksum of the resulting buffer
      as defined in [RFC1321], place the resulting value in this
      Information Element, and export the buffered message.

   Abstract Data Type:   octetArray (16 bytes)

   ElementId:   TBD5

   Status:   current

   Reference:   RFC 1321, The MD5 Message-Digest Algorithm [RFC1321]

8.2.6.  messageScope

   Description:   The presence of this Information Element as scope in
      an Options Template signifies that the options described by the
      Template apply to the IPFIX Message that contains them.  It is
      defined for general purpose message scoping of options, and
      proposed specifically to allow the attachment a checksum to a
      message via IPFIX Options.  The value of this Information Element
      MUST be written as 0 by the File Writer or Exporting Process.  The



Trammell, et al.        Expires January 15, 2009               [Page 28]

Internet-Draft                 IPFIX Files                     July 2008


      value of this Information Element MUST be ignored by the File
      Reader or the Collecting Process.

   Abstract Data Type:   octet

   ElementId:   TBD6

   Status:   current

8.2.7.  minExportSeconds

   Description:   The absolute Export Time of the earliest IPFIX Message
      within the scope containing this Information Element.  This
      Information Element SHOULD be bound to its containing IPFIX
      Transport Session (i.e., File) via an options record and the
      sessionScope Information Element, as defined below, and SHOULD
      appear only once in a given IPFIX File.

   Abstract Data Type:   dateTimeSeconds

   ElementId:   TBD7

   Status:   current

   Units:   seconds

8.2.8.  minFlowStartSeconds

   Description:   The earliest absolute timestamp of the first packet
      within any Flow within the scope containing this Information
      Element, rounded down to the second.  This Information Element
      SHOULD be bound to its containing IPFIX Transport Session (i.e.,
      File) via an options record and the sessionScope Information
      Element, as defined below, and SHOULD appear only once in a given
      IPFIX File.

   Abstract Data Type:   dateTimeSeconds

   ElementId:   TBD8

   Status:   current

   Units:   seconds








Trammell, et al.        Expires January 15, 2009               [Page 29]

Internet-Draft                 IPFIX Files                     July 2008


8.2.9.  opaqueOctets

   Description:   This Information Element is used to encapsulate non-
      IPFIX data into an IPFIX Message stream, for the purpose of
      allowing a non-IPFIX data processor to store a data stream inline
      within an IPFIX file.  A Collecting Process or File Writer MUST
      NOT try to interpret this binary data.  This Information Element
      differs from paddingOctets as its contents are meaningful in some
      non-IPFIX context, while the contents of paddingOctets MUST be
      0x00 and are intended only for Informa      July 2008


8.2.4.  maxFlowEndSeconds

   Description:   The latest absolute timestamp of the last packet
      within any Flow within the scope containing this Information
      Element, rounded up to the second.  This Information Element
      SHOULD be bound to its containing IPFIX Transport Session (i.e.,
      File) via an IPFIX Options and the sessionScope Information
      Element, as defined below, and SHOULD appear only once in a given
      IPFIX File.

   Abstract Data Type:   dateTimeSeconds

   ElementId:   TBD4

   Status:   current

   Units:   seconds

8.2.5.  messageMD5Checksum

   Description:   The MD5 checksum of the IPFIX Message containing this
      record.  This Information Element SHOULD be bound to its
      containing IPFIX Message via an options record and the
      messageScope Information Element, as defined below, and SHOULD
      appear only once in a given IPFIX Message.  To calculate the value
      of this Information Element, first buffer the containing IPFIX
      Message, setting the value of this Information Element to all
      zeroes.  Then caluclate the MD5 checksum of the resulting buffer
      as defined in [RFC1321], place the resulting value in this
      Information Element, and export the buffered message.

   Abstract Data Type:   octetArray (16 bytes)

   ElementId:   TBD5

   Status:   current

   Reference:   RFC 1321, The MD5 Message-Digest Algorithm [RFC1321]

8.2.6.  messageScope

   Description:   The presence of this Information Element as scope in
      an Options Template signifies that the options described by the
      Template apply to the IPFIX Message that contains them.  It is
      defined for general purpose message scoping of options, and
      proposed specifically to allow the attachment a checksum to a
      message via IPFIX Options.  The value of this Information Element
      MUST be written as 0 by the File Writer or Exporting Process.  The



Trammell, et al.        Expires January 15, 2009               [Page 28]

Internet-Draft                 IPFIX Files                     July 2008


      value of this Information Element MUST be ignored by the File
      Reader or the Collecting Process.

   Abstract Data Type:   octet

   ElementId:   TBD6

   Status:   current

8.2.7.  minExportSeconds

   Description:   The absolute Export Time of the earliest IPFIX Message
      within the scope containing this Information Element.  This
      Information Element SHOULD be bound to its containing IPFIX
      Transport Session (i.e., File) via an options record and the
      sessionScope Information Element, as defined below, and SHOULD
      appear only once in a given IPFIX File.

   Abstract Data Type:   dateTimeSeconds

   ElementId:   TBD7

   Status:   current

   Units:   seconds

8.2.8.  minFlowStartSeconds

   Description:   The earliest absolute timestamp of the first packet
      within any Flow within the scope containing this Information
      Element, rounded down to the second.  This Information Element
      SHOULD be bound to its containing IPFIX Transport Session (i.e.,
      File) via an options record and the sessionScope Information
      Element, as defined below, and SHOULD appear only once in a given
      IPFIX File.

   Abstract Data Type:   dateTimeSeconds

   ElementId:   TBD8

   Status:   current

   Units:   seconds








Trammell, et al.        Expires January 15, 2009               [Page 29]

Internet-Draft                 IPFIX Files                     July 2008


8.2.9.  opaqueOctets

   Description:   This Information Element is used to encapsulate non-
      IPFIX data into an IPFIX Message stream, for the purpose of
      allowing a non-IPFIX data processor to store a data stream inline
      within an IPFIX file.  A Collecting Process or File Writer MUST
      NOT try to interpret this binary data.  This Information Element
      differs from paddingOctets as its contents are meaningful in some
      non-IPFIX context, while the contents of paddingOctets MUST be
      0x00 and are intended only for Infortion Element alignment.

   Abstract Data Type:   octet

   ElementId:   TBD9

   Status:   current

8.2.10.  sessionScope

   Description:   The presence of this Information Element as scope in
      an Options Template signifies that the options described by the
      Template apply to the IPFIX Transport Session that contains them.
      Note that as all options are implicitly scoped to Transport
      Session and Observation Domain, this Information Element is
      equivalent to a "null" scope.  It is defined for general purpose
      session scoping of options, and proposed specifically to allow the
      attachment of time window to a file via IPFIX Options.  The value
      of this Information Element MUST be written as 0 by the File
      Writer or Exporting Process.  The value of this Information
      Element MUST be ignored by the File Reader or the Collecting
      Process.

   Abstract Data Type:   octet

   ElementId:   TBD10

   Status:   current


9.  Recommended Error Resilience Strategies

   This section describes recommended methods for making IPFIX Files
   resilient to errors during storage.  It is intended primarily for
   applications using IPFIX Files for long-term archival storage of flow
   data.






Trammell, et al.        Expires January 15, 2009               [Page 30]

Internet-Draft                 IPFIX Files                     July 2008


9.1.  Compression Error Resilience

   Note that, since any file may be compressed and decompressed with a
   variety of widely available tools implementing a variety of
   compression standards (both specified and de facto), compression of
   IPFIX File data can be accomplished externally.  However, compression
   at the file level is not particularly resilient to errors; in the
   worst case, a single bit error in a stream-compressed file may result
   in the loss of the entire file.

   To limit the impact of errors on the recoverability of compressed
   data, we recommend the use of block compression where possible.
   Ideally, the block compression algorithm should support the
   identification and isolation of blocks containing errors; bzip2 is an
   example of such a block compressor.

   Since the block boundary of a block-compressed IPFIX File may fall in
   the middle of an IPFIX Message, resynchronization of an IPFIX Message
   stream by a File Reader after a compression error requires some care.
   The beginning of an IPFIX Message may be identified by its header
   signature (the Version field of the Message Header, 0x00 0x0A,
   followed by a 16-bit Message Length), but simply searching for the
   first occurance of the Version field is insufficient, since these two
   bytes may occur in valid IPFIX Template or Data Sets.

   Therefore, we propose the following algorithm for File Readers to
   resynchronize an IPFIX Message Stream after skipping a compressed
   block containing errors:

   1.  Search after the error for the first occurrence of the octet
       string 0x00, 0x0A (the IPFIX Message Header Version field.)

   2.  Treat this field as the beginning of a candidate IPFIX Message.
       Read the two bytes following the Version field as a Message
       Length, and seek to that offset from the beginning of the
       candidate IPFIX Message.

   3.  If the first two octets after the candidate IPFIX Message are
       0x00, 0x0A (i.e., the IPFIX Message Header Version field of the
       next message in the stream), or if the end of the file is reached
       precisely at the end of the candidate IPFIX Message, presume that
       the candidate IPFIX Message is valid, and begin reading the IPFIX
       File from the start of the candidate IPFIX Message.

   4.  If not, or if the seek reaches end-of-file or another block
       containing errors before finding the end of the candidate
       message, go back to step 1, starting the search two bytes from
       the start of the candidate IPFIX Message.



Trammell, et al.        Expires January 15, 2009               [Page 31]

Internet-Draft                 IPFIX Files        mation Element alignment.

   Abstract Data Type:   octet

   ElementId:   TBD9

   Status:   current

8.2.10.  sessionScope

   Description:   The presence of this Information Element as scope in
      an Options Template signifies that the options described by the
      Template apply to the IPFIX Transport Session that contains them.
      Note that as all options are implicitly scoped to Transport
      Session and Observation Domain, this Information Element is
      equivalent to a "null" scope.  It is defined for general purpose
      session scoping of options, and proposed specifically to allow the
      attachment of time window to a file via IPFIX Options.  The value
      of this Information Element MUST be written as 0 by the File
      Writer or Exporting Process.  The value of this Information
      Element MUST be ignored by the File Reader or the Collecting
      Process.

   Abstract Data Type:   octet

   ElementId:   TBD10

   Status:   current


9.  Recommended Error Resilience Strategies

   This section describes recommended methods for making IPFIX Files
   resilient to errors during storage.  It is intended primarily for
   applications using IPFIX Files for long-term archival storage of flow
   data.






Trammell, et al.        Expires January 15, 2009               [Page 30]

Internet-Draft                 IPFIX Files                     July 2008


9.1.  Compression Error Resilience

   Note that, since any file may be compressed and decompressed with a
   variety of widely available tools implementing a variety of
   compression standards (both specified and de facto), compression of
   IPFIX File data can be accomplished externally.  However, compression
   at the file level is not particularly resilient to errors; in the
   worst case, a single bit error in a stream-compressed file may result
   in the loss of the entire file.

   To limit the impact of errors on the recoverability of compressed
   data, we recommend the use of block compression where possible.
   Ideally, the block compression algorithm should support the
   identification and isolation of blocks containing errors; bzip2 is an
   example of such a block compressor.

   Since the block boundary of a block-compressed IPFIX File may fall in
   the middle of an IPFIX Message, resynchronization of an IPFIX Message
   stream by a File Reader after a compression error requires some care.
   The beginning of an IPFIX Message may be identified by its header
   signature (the Version field of the Message Header, 0x00 0x0A,
   followed by a 16-bit Message Length), but simply searching for the
   first occurance of the Version field is insufficient, since these two
   bytes may occur in valid IPFIX Template or Data Sets.

   Therefore, we propose the following algorithm for File Readers to
   resynchronize an IPFIX Message Stream after skipping a compressed
   block containing errors:

   1.  Search after the error for the first occurrence of the octet
       string 0x00, 0x0A (the IPFIX Message Header Version field.)

   2.  Treat this field as the beginning of a candidate IPFIX Message.
       Read the two bytes following the Version field as a Message
       Length, and seek to that offset from the beginning of the
       candidate IPFIX Message.

   3.  If the first two octets after the candidate IPFIX Message are
       0x00, 0x0A (i.e., the IPFIX Message Header Version field of the
       next message in the stream), or if the end of the file is reached
       precisely at the end of the candidate IPFIX Message, presume that
       the candidate IPFIX Message is valid, and begin reading the IPFIX
       File from the start of the candidate IPFIX Message.

   4.  If not, or if the seek reaches end-of-file or another block
       containing errors before finding the end of the candidate
       message, go back to step 1, starting the search two bytes from
       the start of the candidate IPFIX Message.



Trammell, et al.        Expires January 15, 2009               [Page 31]

Internet-Draft                 IPFIX Files                   July 2008


   The algorithm above will improperly identify a non-message as a
   message approximately 1 in 2^32 times, assuming random IPFIX data.
   It may be expanded to consider multiple candidate IPFIX Messages in
   order to increase reliability.

   In applications (e.g. archival storage) in which error resilience is
   very important, File Writers SHOULD use block compression algorithms,
   and MAY attempt to align IPFIX Messages within compression blocks to
   ease resynchronization after errors, if such is supported by the
   chosen block compressor.  File Readers SHOULD use the
   resynchronization algorithm above to minimize data loss due to
   compression errors.

9.2.  Encryption Error Resilience

   File-level encryption has error resilience issues similar to file-
   level compression.  Single bit errors in the encrypted data stream
   can result in unreadability of the entire remaining file, dependent
   on the encryption method used.  The use of CBC (Cipher Block
   Chaining) mode, which suffers from this low error resilience, is
   relatively common.

   In applications (e.g. archival storage) in which error resilience is
   very important, File Writers SHOULD use a stream cipher, for example
   a block cipher in OFB (Output Feedback) mode (often referred to as
   stream mode) instead of modes like CBC when encrypting, since errors
   are not amplified by stream ciphers: A single-bit error in the
   ciphertext results in a single bit error in the plaintext.
   Alternatively File Writers SHOULD use any other cipher which can
   resynchronize after bit errors.  An example is a block cipher in CBC
   mode that is reinitialized after a specific amount of data has been
   encrypted.  The maximum data loss per bit-error is then up to the
   next reinitialization point.  In this case, File Writers SHOULD also
   use the Message Checksum Options Template to attach a checksum to
   each IPFIX Message in the IPFIX File, in order to support the
   recognition of errors in the decrypted data.


10.  Recommended File Integration Strategies

   This section describes methods for integrating IPFIX File data with
   other file formats.

10.1.  Encapsulation of Non-IPFIX Data in IPFIX Files

   At times it may be useful to export or store non-IPFIX data inline in
   an IPFIX File or Message stream.  To do this cleanly, this data must
   be encapsulated into IPFIX Messages so that an IPFIX File Reader or



Trammell, et al.        Expires January 15, 2009               [Page 32]

Internet-Draft                 IPFIX Files                     July 2008


   Collecting Process can handle it without any need to interpret it.
   At the same time, this data must not be changed during transmission
   or storage.  The opaqueOctets Information Element as defined in
   Section 8.2.9 is provided for this encapsulation.
  



A new IPFIX Message is required or not?

Each time a new IPFIX Message is required (maybe the case for your
metadata cases), what do we do with the Sequence Number?

Is the File Writer consider as a mediation function then?



  
   Processing the encapsulated non-IPFIX data is left to a separate
   processing mechanisms that can identify encapsulated non-IPFIX data
   in an IPFIX message stream, but need not have any other IPFIX
   handling capability, except the ability to skip over all IPFIX
   messages that do not encapsulate non-IPFIX data.

   The Message Checksum Options Template, described in Section 8.1.1 may
   be used as a uniform mechanism to identify errors within encapsulated
   data.

   Note that this mechanism can only encapsulate data objects up to
   65,515 octets in length.  If the space available in one IPFIX Message
   is not enough for the amount of data to be encapsulated, then the
   data must be broken into smaller segments that are encapsulated into
   consecutive IPFIX Messages.  



Is this a generic statement (see my remark above in the metadata
secti               July 2008


   The algorithm above will improperly identify a non-message as a
   message approximately 1 in 2^32 times, assuming random IPFIX data.
   It may be expanded to consider multiple candidate IPFIX Messages in
   order to increase reliability.

   In applications (e.g. archival storage) in which error resilience is
   very important, File Writers SHOULD use block compression algorithms,
   and MAY attempt to align IPFIX Messages within compression blocks to
   ease resynchronization after errors, if such is supported by the
   chosen block compressor.  File Readers SHOULD use the
   resynchronization algorithm above to minimize data loss due to
   compression errors.

9.2.  Encryption Error Resilience

   File-level encryption has error resilience issues similar to file-
   level compression.  Single bit errors in the encrypted data stream
   can result in unreadability of the entire remaining file, dependent
   on the encryption method used.  The use of CBC (Cipher Block
   Chaining) mode, which suffers from this low error resilience, is
   relatively common.

   In applications (e.g. archival storage) in which error resilience is
   very important, File Writers SHOULD use a stream cipher, for example
   a block cipher in OFB (Output Feedback) mode (often referred to as
   stream mode) instead of modes like CBC when encrypting, since errors
   are not amplified by stream ciphers: A single-bit error in the
   ciphertext results in a single bit error in the plaintext.
   Alternatively File Writers SHOULD use any other cipher which can
   resynchronize after bit errors.  An example is a block cipher in CBC
   mode that is reinitialized after a specific amount of data has been
   encrypted.  The maximum data loss per bit-error is then up to the
   next reinitialization point.  In this case, File Writers SHOULD also
   use the Message Checksum Options Template to attach a checksum to
   each IPFIX Message in the IPFIX File, in order to support the
   recognition of errors in the decrypted data.


10.  Recommended File Integration Strategies

   This section describes methods for integrating IPFIX File data with
   other file formats.

10.1.  Encapsulation of Non-IPFIX Data in IPFIX Files

   At times it may be useful to export or store non-IPFIX data inline in
   an IPFIX File or Message stream.  To do this cleanly, this data must
   be encapsulated into IPFIX Messages so that an IPFIX File Reader or



Trammell, et al.        Expires January 15, 2009               [Page 32]

Internet-Draft                 IPFIX Files                     July 2008


   Collecting Process can handle it without any need to interpret it.
   At the same time, this data must not be changed during transmission
   or storage.  The opaqueOctets Information Element as defined in
   Section 8.2.9 is provided for this encapsulation.
A new IPFIX Message is required or not?
Each time a new IPFIX Message is required (maybe the case for your metadata cases), what do we do with the Sequence Number?
Is the File Writer consider as a mediation function then?
   Processing the encapsulated non-IPFIX data is left to a separate
   processing mechanisms that can identify encapsulated non-IPFIX data
   in an IPFIX message stream, but need not have any other IPFIX
   handling capability, except the ability to skip over all IPFIX
   messages that do not encapsulate non-IPFIX data.

   The Message Checksum Options Template, described in Section 8.1.1 may
   be used as a uniform mechanism to identify errors within encapsulated
   data.

   Note that this mechanism can only encapsulate data objects up to
   65,515 octets in length.  If the space available in one IPFIX Message
   is not enough for the amount of data to be encapsulated, then the
   data must be broken into smaller segments that are encapsulated into
   consecutive IPFIX Messages.
Is this a generic statement (see my remark above in the metadata secon) or only applicable here?
Any additional structuring or semantics
   of the raw data is outside the scope of IPFIX and must be implemented
   within the encapsulated binary data itself.  Furthermore, the raw
   encapsulated data cannot be assumed by an IPFIX File Reader to have
   any specific format.

10.2.  Encapsulation of IPFIX Files within Other File Formats

   Consequently, it may also be useful to reverse the encapsulation,
   that is, to export or store IPFIX data inline within a non-IPFIX file
   or data stream.  This makes sense when the other file format is not
   compatible with the encapsulation described above in Section 10.1.
   Generally speaking, the encapsulation here will be specific to the
   format of the containing file.  For example, IPFIX files may be
   embedded in XML elements using hex or Base64 encoding, or in raw
   binary files using start and end delimiters or some form of run-
   length encoding.  As there are as many potential encapsulations here
   as there are potential file formats, the specifics of each are out of
   scope for this specification.


11.  Security Considerations

   The IPFIX-based file format itself does not directly introduce
   security issues.  Rather it is used to store information which may
   for privacy or business reasons be considered sensitive.  The file
   format must therefore provide appropriate procedures to guarantee the
   integrity and confidentiality of the stored information.




Trammell, et al.        Expires January 15, 2009               [Page 33]

Internet-Draft                 IPFIX Files                     July 2008


   The underlying protocol used to exchange the information that will be
   stored using the format proposed in this document must as well apply
   appropriate procedures to guarantee the integrity and confidentiality
   of the exported information.  Such issues are addressed in [RFC5101].

   Implementors of IPFIX File Writers which store data taken from an
   IPFIX Collecting Process using TLS or DTLS for transport security
   should note that IPFIX Files may present a potential breach of
   confidentiality if IPFIX data collected using TLS or DTLS is stored
   in unencrypted files, and should consider providing an external file
   encryption option to mitigate this risk.


12.  IANA Considerations

   This document specifies the creation of several new IPFIX Information
   Elements in the IPFIX Information Element registry located at
   http://www.iana.org/assignments/ipfix, as defined in Section 8.2
   above.  IANA has assigned the following Information Element numbers
   for their respective Information Elements as specified below:

   o  Information Element number TBD1 for the collectionTimeMilliseconds
      Information Element.

   o  Information Element number TBD2 for the exportSctpStreamId
      Information Element.

   o  Information Element number TBD3 for the maxExportSeconds
      Information Element.

   o  Information Element number TBD4 for the maxFlowEndSeconds
      Information Element.

   o  Information Element number TBD5 for the messageMD5Checksum
      Information Element.

   o  Information Element number TBD6 for the messageScope Information
      Element.

   o  Information Element number TBD7 for the minExportSeconds
      Information Element.

   o  Information Element number TBD8 for the minFlowStartSeconds
      Information Element.

   o  Information Element number TBD9 for the opaqueOctets Information
      Element.




Trammell, et al.        Expires January 15, 2009               [Page 34]

Internet-Draft                 IPFIX Files                     July 2008


   o  Information Element number TBD10 for the sessionScope Information
      Element.

   [NOTE for IANA: The text TBDn should be replaced with the respective
   assigned Information Element numbers where they appear in this
   document.]


13.  Acknowledgements

   tion) or only applicable here?



  
Any additional structuring or semantics
   of the raw data is outside the scope of IPFIX and must be implemented
   within the encapsulated binary data itself.  Furthermore, the raw
   encapsulated data cannot be assumed by an IPFIX File Reader to have
   any specific format.

10.2.  Encapsulation of IPFIX Files within Other File Formats

   Consequently, it may also be useful to reverse the encapsulation,
   that is, to export or store IPFIX data inline within a non-IPFIX file
   or data stream.  This makes sense when the other file format is not
   compatible with the encapsulation described above in Section 10.1.
   Generally speaking, the encapsulation here will be specific to the
   format of the containing file.  For example, IPFIX files may be
   embedded in XML elements using hex or Base64 encoding, or in raw
   binary files using start and end delimiters or some form of run-
   length encoding.  As there are as many potential encapsulations here
   as there are potential file formats, the specifics of each are out of
   scope for this specification.


11.  Security Considerations

   The IPFIX-based file format itself does not directly introduce
   security issues.  Rather it is used to store information which may
   for privacy or business reasons be considered sensitive.  The file
   format must therefore provide appropriate procedures to guarantee the
   integrity and confidentiality of the stored information.




Trammell, et al.        Expires January 15, 2009               [Page 33]

Internet-Draft                 IPFIX Files                     July 2008


   The underlying protocol used to exchange the information that will be
   stored using the format proposed in this document must as well apply
   appropriate procedures to guarantee the integrity and confidentiality
   of the exported information.  Such issues are addressed in [RFC5101].

   Implementors of IPFIX File Writers which store data taken from an
   IPFIX Collecting Process using TLS or DTLS for transport security
   should note that IPFIX Files may present a potential breach of
   confidentiality if IPFIX data collected using TLS or DTLS is stored
   in unencrypted files, and should consider providing an external file
   encryption option to mitigate this risk.


12.  IANA Considerations

   This document specifies the creation of several new IPFIX Information
   Elements in the IPFIX Information Element registry located at
   http://www.iana.org/assignments/ipfix, as defined in Section 8.2
   above.  IANA has assigned the following Information Element numbers
   for their respective Information Elements as specified below:

   o  Information Element number TBD1 for the collectionTimeMilliseconds
      Information Element.

   o  Information Element number TBD2 for the exportSctpStreamId
      Information Element.

   o  Information Element number TBD3 for the maxExportSeconds
      Information Element.

   o  Information Element number TBD4 for the maxFlowEndSeconds
      Information Element.

   o  Information Element number TBD5 for the messageMD5Checksum
      Information Element.

   o  Information Element number TBD6 for the messageScope Information
      Element.

   o  Information Element number TBD7 for the minExportSeconds
      Information Element.

   o  Information Element number TBD8 for the minFlowStartSeconds
      Information Element.

   o  Information Element number TBD9 for the opaqueOctets Information
      Element.




Trammell, et al.        Expires January 15, 2009               [Page 34]

Internet-Draft                 IPFIX Files                     July 2008


   o  Information Element number TBD10 for the sessionScope Information
      Element.

   [NOTE for IANA: The text TBDn should be replaced with the respective
   assigned Information Element numbers where they appear in this
   document.]


13.  Acknowledgements

 Thanks to Maurizio Molina, Tom Kosnar, and Andreas Kind for technical
   assistance with the requirements for a standard flow storage format.
   Thanks to Benoit Claise, Paul Aitken, and Andrew Johnson for their
   reviews and feedback.


14.  References

14.1.  Normative References

   [RFC5101]  Claise, B., "Specification of the IP Flow Information
              Export (IPFIX) Protocol for the Exchange of IP Traffic
              Flow Information", RFC 5101, January 2008.

   [RFC5102]  Quittek, J., Bryant, S., Claise, B., Aitken, P., and J.
              Meyer, "Information Model for IP Flow Information Export",
              RFC 5102, January 2008.

   [I-D.ietf-ipfix-reducing-redundancy]
              Boschi, E., "Reducing Redundancy in IP Flow Information
              Export (IPFIX) and Packet  Sampling (PSAMP) Reports",
              draft-ietf-ipfix-reducing-redundancy-04 (work in
              progress), May 2007.

   [I-D.ietf-ipfix-exporting-type]
              Boschi, E., Trammell, B., Mark, L., and T. Zseby,
              "Exporting Type Information for IPFIX Information
              Elements", draft-ietf-ipfix-exporting-type-01 (work in
              progress), February 2008.

   [RFC1321]  Rivest, R., "The MD5 Message-Digest Algorithm", RFC 1321,
              April 1992.

14.2.  Informative References

   [I-D.ietf-ipfix-arch]
              Sadasivan, G. and N. Brownlee, "Architecture Model for IP
              Flow Information Export", draft-ietf-ipfix-arch-02 (work



Trammell, et al.        Expires January 15, 2009               [Page 35]

Internet-Draft                 IPFIX Files                     July 2008


              in progress), October 2003.

   [I-D.ietf-ipfix-as]
              Zseby, T., "IPFIX Applicability", draft-ietf-ipfix-as-12
              (work in progress), July 2007.

   [RFC5103]  Trammell, B. and E. Boschi, "Bidirectional Flow Export
              Using IP Flow Information Export (IPFIX)", RFC 5103,
              January 2008.

   [I-D.ietf-ipfix-testing]
              Schmoll, C., Aitken, P., and B. Claise, "Guidelines for IP
              Flow Information eXport (IPFIX) Testing",
              draft-ietf-ipfix-testing-05 (work in progress),
              April 2008.

   [RFC3954]  Claise, B., "Cisco Systems NetFlow Services Export Version
              9", RFC 3954, October 2004.

   [RFC3917]  Quittek, J., Zseby, T., Claise, B., and S. Zander,
              "Requirements for IP Flow Information Export (IPFIX)",
              RFC 3917, October 2004.

   [RFC2119]  Bradner, S., "Key words for use in RFCs to Indicate
              Requirement Levels", BCP 14, RFC 2119, March 1997.

   [SAINT2007]
              Trammell, B., Boschi, E., Mark, L., and T. Zseby,
              "Requirements for a standardized flow storage solution",
               in Proceedings of the SAINT 2007 workshop on Internet
              Measurement Technology, Hiroshima, Japan, January 2007.


Appendix A.  Example IPFIX File

   In this section we will explore an example IPFIX File which
   demonstrates the various features of the IPFIX File format.  This
   file contains flow records described by a single Template.  This file
   also contains a File Time Window record to note the start and end
   time of the data, and an Export Session Details record to record
   collection infrastructure information.  Each Message within this File
   also contains a Message Checksum record, as this file may be
   externally encrypted and/or stored as an archive.  The structure of
   this file is shown in Figure 2.







Trammell, et al.        Expires January 15, 2009               [Page 36]

Internet-Draft                 IPFIX Files                     July 2008


             +=================================================+
             | IPFIX Message                       seq. 0      |
             | +---------------------------------------------+ |
             | | Template Set (id 2)                  1 rec  | |
             | |   Data Tmpl. id 256                         | |
             | +-----------  Thanks to Maurizio Molina, Tom Kosnar, and Andreas Kind for technical
   assistance with the requirements for a standard flow storage format.
   Thanks to Benoit Claise, Paul Aitken, and Andrew Johnson for their
   reviews and feedback.


14.  References

14.1.  Normative References

   [RFC5101]  Claise, B., "Specification of the IP Flow Information
              Export (IPFIX) Protocol for the Exchange of IP Traffic
              Flow Information", RFC 5101, January 2008.

   [RFC5102]  Quittek, J., Bryant, S., Claise, B., Aitken, P., and J.
              Meyer, "Information Model for IP Flow Information Export",
              RFC 5102, January 2008.

   [I-D.ietf-ipfix-reducing-redundancy]
              Boschi, E., "Reducing Redundancy in IP Flow Information
              Export (IPFIX) and Packet  Sampling (PSAMP) Reports",
              draft-ietf-ipfix-reducing-redundancy-04 (work in
              progress), May 2007.

   [I-D.ietf-ipfix-exporting-type]
              Boschi, E., Trammell, B., Mark, L., and T. Zseby,
              "Exporting Type Information for IPFIX Information
              Elements", draft-ietf-ipfix-exporting-type-01 (work in
              progress), February 2008.

   [RFC1321]  Rivest, R., "The MD5 Message-Digest Algorithm", RFC 1321,
              April 1992.

14.2.  Informative References

   [I-D.ietf-ipfix-arch]
              Sadasivan, G. and N. Brownlee, "Architecture Model for IP
              Flow Information Export", draft-ietf-ipfix-arch-02 (work



Trammell, et al.        Expires January 15, 2009               [Page 35]

Internet-Draft                 IPFIX Files                     July 2008


              in progress), October 2003.

   [I-D.ietf-ipfix-as]
              Zseby, T., "IPFIX Applicability", draft-ietf-ipfix-as-12
              (work in progress), July 2007.

   [RFC5103]  Trammell, B. and E. Boschi, "Bidirectional Flow Export
              Using IP Flow Information Export (IPFIX)", RFC 5103,
              January 2008.

   [I-D.ietf-ipfix-testing]
              Schmoll, C., Aitken, P., and B. Claise, "Guidelines for IP
              Flow Information eXport (IPFIX) Testing",
              draft-ietf-ipfix-testing-05 (work in progress),
              April 2008.

   [RFC3954]  Claise, B., "Cisco Systems NetFlow Services Export Version
              9", RFC 3954, October 2004.

   [RFC3917]  Quittek, J., Zseby, T., Claise, B., and S. Zander,
              "Requirements for IP Flow Information Export (IPFIX)",
              RFC 3917, October 2004.

   [RFC2119]  Bradner, S., "Key words for use in RFCs to Indicate
              Requirement Levels", BCP 14, RFC 2119, March 1997.

   [SAINT2007]
              Trammell, B., Boschi, E., Mark, L., and T. Zseby,
              "Requirements for a standardized flow storage solution",
               in Proceedings of the SAINT 2007 workshop on Internet
              Measurement Technology, Hiroshima, Japan, January 2007.


Appendix A.  Example IPFIX File

   In this section we will explore an example IPFIX File which
   demonstrates the various features of the IPFIX File format.  This
   file contains flow records described by a single Template.  This file
   also contains a File Time Window record to note the start and end
   time of the data, and an Export Session Details record to record
   collection infrastructure information.  Each Message within this File
   also contains a Message Checksum record, as this file may be
   externally encrypted and/or stored as an archive.  The structure of
   this file is shown in Figure 2.







Trammell, et al.        Expires January 15, 2009               [Page 36]

Internet-Draft                 IPFIX Files                     July 2008


             +=================================================+
             | IPFIX Message                       seq. 0      |
             | +---------------------------------------------+ |
             | | Template Set (id 2)                  1 rec  | |
             | |   Data Tmpl. id 256                         | |
             | +-------------------------------------------+ |
             | | Options Template Set (id 3)          3 recs | |
             | |   File Time Window Opt. Tmpl. id 257        | |
             | |   Message Checksum Opt. Tmpl. id 259        | |
             | |   Export Session Details Opt. Tmpl. id 258  | |
             | +---------------------------------------------+ |
             | | Data Set (id 259) [Message Checksum] 1 rec  | |
             | +---------------------------------------------+ |
             +=================================================+
             | IPFIX Message                       seq. 1      |
             | +---------------------------------------------+ |
             | | Data Set (id 257) [File Time Window] 1 rec  | |
             | +---------------------------------------------+ |
             | | Data Set (id 258) [Export Session]   1 rec  | |
             | +---------------------------------------------+ |
             | | Data Set (id 259) [Message Checksum] 1 rec  | |
             | +---------------------------------------------+ |
             +=================================================+
             | IPFIX Message                       seq. 4      |
             | +---------------------------------------------+ |
             | | Data Set (id 256)                   50 recs | |
             | |  contains flow data                         | |
             | +---------------------------------------------+ |
             | | Data Set (id 259) [Message Checksum] 1 rec  | |
             | +---------------------------------------------+ |
             +=================================================+
             | IPFIX Message                       seq. 55     |
             |                    . . .                        |

                     Figure 2: File Example Structure

   The template describing the data records contains a flow start
   timestamp, an IPv4 5-tuple, and packet and octet total counts.  The
   Template Set defining this is as shown in Figure 3 below:












Trammell, et al.        Expires January 15, 2009               [Page 37]

Internet-Draft                 IPFIX Files                     July 2008


                        1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |          Set ID = 2           |          Length =  40         |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |      Template ID = 256        |        Field Count = 8        |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |0| flowStartSeconds      = 150 |       Field Length =  4       |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |0| sourceIPv4Address     =   8 |       Field Length =  4       |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |0| dest.IPv4Address      =  12 |       Field Length =  4       |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |0| sourceTransportPort   =   7 |       Field Length =  2       |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |0| dest.TransportPort    =  11 |       Field Length =  2       |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |0| protocolIdentifier    =   4 |       Field Length =  1       |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |0| octetTotalCount       =  85 |       Field Length =  4       |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |0| packetTotalCount      =  86 |       Field Length =  4       |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

                   Figure 3: File Example Data Template

A.1.  Example Options Templates

   This is followed by an Options Template Set containing the options
   templates required to read the File: the File Time Window Options
   Template defined in Section 8.1.2 above, the Export Session Details
   ------------------------------------+ |
             | | Options Template Set (id 3)          3 recs | |
             | |   File Time Window Opt. Tmpl. id 257        | |
             | |   Message Checksum Opt. Tmpl. id 259        | |
             | |   Export Session Details Opt. Tmpl. id 258  | |
             | +---------------------------------------------+ |
             | | Data Set (id 259) [Message Checksum] 1 rec  | |
             | +---------------------------------------------+ |
             +=================================================+
             | IPFIX Message                       seq. 1      |
             | +---------------------------------------------+ |
             | | Data Set (id 257) [File Time Window] 1 rec  | |
             | +---------------------------------------------+ |
             | | Data Set (id 258) [Export Session]   1 rec  | |
             | +---------------------------------------------+ |
             | | Data Set (id 259) [Message Checksum] 1 rec  | |
             | +---------------------------------------------+ |
             +=================================================+
             | IPFIX Message                       seq. 4      |
             | +---------------------------------------------+ |
             | | Data Set (id 256)                   50 recs | |
             | |  contains flow data                         | |
             | +---------------------------------------------+ |
             | | Data Set (id 259) [Message Checksum] 1 rec  | |
             | +---------------------------------------------+ |
             +=================================================+
             | IPFIX Message                       seq. 55     |
             |                    . . .                        |

                     Figure 2: File Example Structure

   The template describing the data records contains a flow start
   timestamp, an IPv4 5-tuple, and packet and octet total counts.  The
   Template Set defining this is as shown in Figure 3 below:












Trammell, et al.        Expires January 15, 2009               [Page 37]

Internet-Draft                 IPFIX Files                     July 2008


                        1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |          Set ID = 2           |          Length =  40         |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |      Template ID = 256        |        Field Count = 8        |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |0| flowStartSeconds      = 150 |       Field Length =  4       |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |0| sourceIPv4Address     =   8 |       Field Length =  4       |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |0| dest.IPv4Address      =  12 |       Field Length =  4       |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |0| sourceTransportPort   =   7 |       Field Length =  2       |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |0| dest.TransportPort    =  11 |       Field Length =  2       |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |0| protocolIdentifier    =   4 |       Field Length =  1       |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |0| octetTotalCount       =  85 |       Field Length =  4       |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |0| packetTotalCount      =  86 |       Field Length =  4       |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

                   Figure 3: File Example Data Template

A.1.  Example Options Templates

   This is followed by an Options Template Set containing the options
   templates required to read the File: the File Time Window Options
   Template defined in Section 8.1.2 above, the Export Session Details
 Options Template defined in Section 8.1.3 above, and the Message
   Checksum Options Template defined in Section 8.1.1 above.  This
   Options Template Set is shown in Figure 4 and Figure 5 below:

















Trammell, et al.        Expires January 15, 2009               [Page 38]

Internet-Draft                 IPFIX Files                     July 2008


                        1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |          Set ID = 3           |          Length =  80         |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |      Template ID = 257        |        Field Count = 3        |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |    Scope Field Count = 1      |0| sessionScope        = TBD10 |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |       Field Length =  1       |0| minFlowStartSeconds  = TBD8 |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |       Field Length =  4       |0| maxFlowEndSeconds    = TBD4 |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |       Field Length = 4        |      Template ID = 259        |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |       Field Count = 2         |    Scope Field Count = 1      |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |0| messageScope         = TBD6 |       Field Length =  1       |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |0| messageMD5Checksum   = TBD5 |       Field Length = 16       |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

    Figure 4: File Example Options Templates (Time Window and Checksum)




























Trammell, et al.        Expires January 15, 2009               [Page 39]

Internet-Draft                 IPFIX Files                     July 2008


                        1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |       Template ID = 258       |         Field Count = 9       |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |    Scope Field Count = 1      |0| sessionScope        = TBD10 |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |       Field Length =  1       |0| exporterIPv4Address   = 130 |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |       Field Length =  4       |0| collectorIPv4Address  = 211 |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |       Field Length =  4       |0| exporterTransportPort = 217 |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |       Field Length =  2       |0| col.TransportPort     = 216 |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |       Field Length =  2       |0| col.TransportProtocol = 215 |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |       Field Length =  1       |0| col.ProtocolVersion   = 214 |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |       Field Length =  1       |0| minExportSeconds     = TBD7 |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |       Field Length =  4       |0| maxExportSeconds     = TBD3 |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |       Field Length =  4       |     set padding (2 octets)    |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

   Figure 5: File Example Options Templates, Continued (Session Details)

A.2.  Example Supplemental Options Data

   Following the templates required to decode the file is the
   supplemental options information used to describe the file's contents
   and type information.  First comes the File Time Window recor  Options Template defined in Section 8.1.3 above, and the Message
   Checksum Options Template defined in Section 8.1.1 above.  This
   Options Template Set is shown in Figure 4 and Figure 5 below:

















Trammell, et al.        Expires January 15, 2009               [Page 38]

Internet-Draft                 IPFIX Files                     July 2008


                        1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |          Set ID = 3           |          Length =  80         |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |      Template ID = 257        |        Field Count = 3        |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |    Scope Field Count = 1      |0| sessionScope        = TBD10 |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |       Field Length =  1       |0| minFlowStartSeconds  = TBD8 |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |       Field Length =  4       |0| maxFlowEndSeconds    = TBD4 |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |       Field Length = 4        |      Template ID = 259        |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |       Field Count = 2         |    Scope Field Count = 1      |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |0| messageScope         = TBD6 |       Field Length =  1       |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |0| messageMD5Checksum   = TBD5 |       Field Length = 16       |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

    Figure 4: File Example Options Templates (Time Window and Checksum)




























Trammell, et al.        Expires January 15, 2009               [Page 39]

Internet-Draft                 IPFIX Files                     July 2008


                        1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |       Template ID = 258       |         Field Count = 9       |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |    Scope Field Count = 1      |0| sessionScope        = TBD10 |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |       Field Length =  1       |0| exporterIPv4Address   = 130 |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |       Field Length =  4       |0| collectorIPv4Address  = 211 |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |       Field Length =  4       |0| exporterTransportPort = 217 |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |       Field Length =  2       |0| col.TransportPort     = 216 |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |       Field Length =  2       |0| col.TransportProtocol = 215 |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |       Field Length =  1       |0| col.ProtocolVersion   = 214 |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |       Field Length =  1       |0| minExportSeconds     = TBD7 |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |       Field Length =  4       |0| maxExportSeconds     = TBD3 |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |       Field Length =  4       |     set padding (2 octets)    |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

   Figure 5: File Example Options Templates, Continued (Session Details)

A.2.  Example Supplemental Options Data

   Following the templates required to decode the file is the
   supplemental options information used to describe the file's contents
   and type information.  First comes the File Time Window recd; it
   notes that the file contains data from 9 October 2007 between
   00:01:13 and 23:56:27 UTC, and appears as in Figure 6:
















Trammell, et al.        Expires January 15, 2009               [Page 40]

Internet-Draft                 IPFIX Files                     July 2008


                        1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |          Set ID = 257         |          Length =  13         |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   | sessionScope  |           minFlowStartSeconds
   |       0       |         2007-10-09 00:01:13 UTC           . . .
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
                   |            maxFlowEndSeconds
   . . .           |         2007-10-09 23:56:27 UTC           . . .
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
                   |
   . . .           |
   +-+-+-+-+-+-+-+-+

                    Figure 6: File Example Time Window

   This is followed by information about how the data in the file was
   collected, in the Export Session Details record.  This record notes
   that the session stored in this file was sent via SCTP from an
   exporter at 192.0.2.30 port 32769 to an collector at 192.0.2.40 port
   4739, and contains messages exported between 00:01:57 and 23:57:12
   UTC on 9 October 2007; it is represented in its Data Set as in
   Figure 7:



























Trammell, et al.        Expires January 15, 2009               [Page 41]

Internet-Draft                 IPFIX Files                     July 2008


                       1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |          Set ID = 258         |          Length =  27         |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   | sessionScope  |           exporterIPv4Address
   |       0       |               192.0.2.30                  . . .
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
                   |           collectorIPv4Address
   . . .           |               192.0.2.31                  . . .
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
                   |     exporterTransportPort     |   cTPort
   . . .           |             32769             |    4739   . . .
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
                   |   cTProtocol  |  cPVersion    |
   . . .           |      132      |     10        |           . . .
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
                minExportSeconds                   |
   . . .     2007-10-09 00:01:57 UTC               |           . . .
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
                maxExportSeconds                   |
   . . .     2007-10-09 23:57:12 UTC               |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

               Figure 7: File Example Export Session Details

A.3.  Example Message Checksum

   Each IPFIX Message within the file is completed with a Message
   Checksum record; the structure of this record within its Data Set is
   as in Figure 8:

    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |          Set ID = 259         |          Length =  24         |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   | messageScope  |                                               |
   |       0       |                                               |
   +-+-+-+-+-+-+-+-+                                               |
   |                       messageMD5Checksum                      |
   |           (16 byte MD5 checksum of options message)           |
   |                     ord; it
   notes that the file contains data from 9 October 2007 between
   00:01:13 and 23:56:27 UTC, and appears as in Figure 6:
















Trammell, et al.        Expires January 15, 2009               [Page 40]

Internet-Draft                 IPFIX Files                     July 2008


                        1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |          Set ID = 257         |          Length =  13         |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   | sessionScope  |           minFlowStartSeconds
   |       0       |         2007-10-09 00:01:13 UTC           . . .
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
                   |            maxFlowEndSeconds
   . . .           |         2007-10-09 23:56:27 UTC           . . .
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
                   |
   . . .           |
   +-+-+-+-+-+-+-+-+

                    Figure 6: File Example Time Window

   This is followed by information about how the data in the file was
   collected, in the Export Session Details record.  This record notes
   that the session stored in this file was sent via SCTP from an
   exporter at 192.0.2.30 port 32769 to an collector at 192.0.2.40 port
   4739, and contains messages exported between 00:01:57 and 23:57:12
   UTC on 9 October 2007; it is represented in its Data Set as in
   Figure 7:



























Trammell, et al.        Expires January 15, 2009               [Page 41]

Internet-Draft                 IPFIX Files                     July 2008


                       1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |          Set ID = 258         |          Length =  27         |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   | sessionScope  |           exporterIPv4Address
   |       0       |               192.0.2.30                  . . .
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
                   |           collectorIPv4Address
   . . .           |               192.0.2.31                  . . .
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
                   |     exporterTransportPort     |   cTPort
   . . .           |             32769             |    4739   . . .
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
                   |   cTProtocol  |  cPVersion    |
   . . .           |      132      |     10        |           . . .
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
                minExportSeconds                   |
   . . .     2007-10-09 00:01:57 UTC               |           . . .
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
                maxExportSeconds                   |
   . . .     2007-10-09 23:57:12 UTC               |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

               Figure 7: File Example Export Session Details

A.3.  Example Message Checksum

   Each IPFIX Message within the file is completed with a Message
   Checksum record; the structure of this record within its Data Set is
   as in Figure 8:

    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |          Set ID = 259         |          Length =  24         |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   | messageScope  |                                               |
   |       0       |                                               |
   +-+-+-+-+-+-+-+-+                                               |
   |                       messageMD5Checksum                      |
   |           (16 byte MD5 checksum of options message)           |
   |                                                             |
   |                                                               |
   |               +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |               |              set padding (3 octets)           |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

                  Figure 8: File Example Message Checksum



Trammell, et al.        Expires January 15, 2009               [Page 42]

Internet-Draft                 IPFIX Files                     July 2008


A.4.  File Example Data Set

   After the templates and supplemental options information comes the
   data itself.  The first record of an example Data Set is shown with
   its message and set headers in Figure 9:

                        1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |     Version = 10              |         Length = 1296         |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |          Export Time = 2007-10-09 00:01:57 UTC                |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                      Sequence Number = 4                      |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                   Observation Domain ID = 1                   |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |       Set ID = 256           |          Length = 1254         |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                      flowStartSeconds                         |
   |                    2007-10-09 00:01:13 UTC                    |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                      sourceIPv4Address                        |
   |                          192.0.2.2                            |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                    destinationIPv4Address                     |
   |                          192.0.2.3                            |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |      sourceTransportPort      |   destinationTransportPort    |
   |             32770             |               80              |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |  protocolId   |             totalOctetCount
   |       6       |                  18000                    . . .
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
                   |             totalPacketCount
   . . .           |                    65                     . . .
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
                   |             (49 more records)
   . . .           |
   +-+-+-+-+-+-+-+-+

                      Figure 9: File Example Data Set

A.5.  Complete File Example

   Bringing together the examples above and adding message headers as
   appropriate, a hex dump of the first 317 bytes of the example file
   constructed above would appear as in the annotated Figure 10 below.



Trammell, et al.        Expires January 15, 2009               [Page 43]

Internet-Draft                 IPFIX Files                     July 2008


   [EDITOR'S NOTE: In this figure, xx refers to unassigned IANA IE
   numbers as in the IANA Considerations section above; cs refers to
   message checksum bytes that depend on the rest of the message
   contents.  These will have to be replaced if we keep this example
   once the IE numbers are assigned.]

     0:|00 0A 00 A0 47 0A B6 E5 00 00 00 00 00 00 00 01
      [^ first message header (length 160 bytes) -->
    16:|00 02 00 28 01 00 00 08 00 96 00 04 00 08 00 04
      [^ data template set -->
    32: 00 0C 00 04 00 07 00 02 00 0B 00 02 00 04 00 01

    48: 00 55 00 04 00 56 00 04|00 03 00 50 01 01 00 03
                              [^ opt template set -->
    64: 00 01 xx xx 00 01 xx x                                            |
   |                                                               |
   |               +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |               |              set padding (3 octets)           |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

                  Figure 8: File Example Message Checksum



Trammell, et al.        Expires January 15, 2009               [Page 42]

Internet-Draft                 IPFIX Files                     July 2008


A.4.  File Example Data Set

   After the templates and supplemental options information comes the
   data itself.  The first record of an example Data Set is shown with
   its message and set headers in Figure 9:

                        1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |     Version = 10              |         Length = 1296         |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |          Export Time = 2007-10-09 00:01:57 UTC                |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                      Sequence Number = 4                      |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                   Observation Domain ID = 1                   |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |       Set ID = 256           |          Length = 1254         |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                      flowStartSeconds                         |
   |                    2007-10-09 00:01:13 UTC                    |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                      sourceIPv4Address                        |
   |                          192.0.2.2                            |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                    destinationIPv4Address                     |
   |                          192.0.2.3                            |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |      sourceTransportPort      |   destinationTransportPort    |
   |             32770             |               80              |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |  protocolId   |             totalOctetCount
   |       6       |                  18000                    . . .
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
                   |             totalPacketCount
   . . .           |                    65                     . . .
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
                   |             (49 more records)
   . . .           |
   +-+-+-+-+-+-+-+-+

                      Figure 9: File Example Data Set

A.5.  Complete File Example

   Bringing together the examples above and adding message headers as
   appropriate, a hex dump of the first 317 bytes of the example file
   constructed above would appear as in the annotated Figure 10 below.



Trammell, et al.        Expires January 15, 2009               [Page 43]

Internet-Draft                 IPFIX Files                     July 2008


   [EDITOR'S NOTE: In this figure, xx refers to unassigned IANA IE
   numbers as in the IANA Considerations section above; cs refers to
   message checksum bytes that depend on the rest of the message
   contents.  These will have to be replaced if we keep this example
   once the IE numbers are assigned.]

     0:|00 0A 00 A0 47 0A B6 E5 00 00 00 00 00 00 00 01
      [^ first message header (length 160 bytes) -->
    16:|00 02 00 28 01 00 00 08 00 96 00 04 00 08 00 04
      [^ data template set -->
    32: 00 0C 00 04 00 07 00 02 00 0B 00 02 00 04 00 01

    48: 00 55 00 04 00 56 00 04|00 03 00 50 01 01 00 03
                              [^ opt template set -->
    64: 00 01 xx xx 00 01 xxx 00 04 xx xx 00 04 01 03

    80: 00 02 00 01 xx xx 00 01 xx xx 00 10 01 02 00 09

    96: 00 01 xx xx 00 01 00 82 00 04 00 D3 00 04 00 D9

   112: 00 02 00 D8 00 02 00 D7 00 01 00 D0 00 01 xx xx

   128: 00 04 xx xx 00 04 00 00|01 03 00 18 00 cs cs cs
                              [^ checksum record -->
   144: cs cs cs cs cs cs cs cs cs cs cs cs cs 00 00 00

   176:|00 0A 00 50 47 0A B6 E5 00 00 00 01 00 00 00 01
      [^ second message header (length 80 bytes) -->
   192:|01 01 00 0E 00 47 0A B6 B9 47 0C 07 1B 00|01 02
      [^ time window rec -> [ session detail rec ^ -->
   208: 00 1C 00 C0 00 02 1E 0C 00 02 1F 80 01 12 83 84

   224: 0A 47 0A B6 E5 47 0C 07 48 00|01 03 00 18 00 cs
              [ message checksum rec ^ -->
   240: cs cs cs cs cs cs cs cs cs cs cs cs cs cs cs 00

   256:|00 0A 05 10 47 0A B6 E5 00 00 00 06 00 00 00 01
      [^ third message header (length 1296 bytes) -->
   272:|01 00 04 E6|47 0A B6 B9 C0 00 02 02 C0 00 02 03
      [^ set hdr ][^ first data rec -->
   288: 80 02 00 50 06 00 00 46 50 00 00 00 41

                     Figure 10: File Example Hex Dump








Trammell, et al.        Expires January 15, 2009               [Page 44]

Internet-Draft                 IPFIX Files                     July 2008


Appendix B.  Applicability of IPFIX Files to NetFlow V9 flow storage

   As the IPFIX Message format is nearly a superset of the NetFlow V9
   packet format, IPFIX Files can be used for store NetFlow V9 data
   relatively easily.  This section describes a method for doing so.
   The differences between the two protocols are outlined in
   Appendix B.1 below.  A simple, lightweight, message-for-message
   translation method for transforming V9 Packets into IPFIX Messages
   for storage within IPFIX Files is described in Appendix B.2.  An
   example of this translation method is given in Appendix B.3.

B.1.  Comparing NetFlow V9 to IPFIX

   With a few caveats, the IPFIX Protocol is a superset of the NetFlow
   V9 protocol, having evolved from it largely through a process of
   feature addition to bring it into compliance with the IPFIX
   Requirements and the needs of stakeholders within the IPFIX Working
   Group.  This appendix outlines the differences between the two
   protocols.  It is informative only, and presented as an exploration
   of the two protocols to motivate the usage of IPFIX Files to store
   V9-collected flow data.

B.1.1.  Message Header Format

   Both NetFlow V9 and IPFIX use streams of messages prefixed by a
   message header, though the message header differs significantly
   between the two.  Note that in NetFlow V9 terminology, these messages
   are called packets, and messages must be delimited by datagram
   boundaries.  IPFIX does not have this constraint.  The header formats
   are detailed below:

    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |       Version Number          |            Count              |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                           sysUpTime                           |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                           UNIX Secs                           |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                       Sequence Number                         |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                        Source ID                              |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

                Figure 11: NetFlow V9 Packet Header Format





Trammell, et al.        Expires January 15, 2009               [Page 45]

Internet-Draft                 IPFIX Files                     July 2008


    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+ xx 00 04 xx xx 00 04 01 03

    80: 00 02 00 01 xx xx 00 01 xx xx 00 10 01 02 00 09

    96: 00 01 xx xx 00 01 00 82 00 04 00 D3 00 04 00 D9

   112: 00 02 00 D8 00 02 00 D7 00 01 00 D0 00 01 xx xx

   128: 00 04 xx xx 00 04 00 00|01 03 00 18 00 cs cs cs
                              [^ checksum record -->
   144: cs cs cs cs cs cs cs cs cs cs cs cs cs 00 00 00

   176:|00 0A 00 50 47 0A B6 E5 00 00 00 01 00 00 00 01
      [^ second message header (length 80 bytes) -->
   192:|01 01 00 0E 00 47 0A B6 B9 47 0C 07 1B 00|01 02
      [^ time window rec -> [ session detail rec ^ -->
   208: 00 1C 00 C0 00 02 1E 0C 00 02 1F 80 01 12 83 84

   224: 0A 47 0A B6 E5 47 0C 07 48 00|01 03 00 18 00 cs
              [ message checksum rec ^ -->
   240: cs cs cs cs cs cs cs cs cs cs cs cs cs cs cs 00

   256:|00 0A 05 10 47 0A B6 E5 00 00 00 06 00 00 00 01
      [^ third message header (length 1296 bytes) -->
   272:|01 00 04 E6|47 0A B6 B9 C0 00 02 02 C0 00 02 03
      [^ set hdr ][^ first data rec -->
   288: 80 02 00 50 06 00 00 46 50 00 00 00 41

                     Figure 10: File Example Hex Dump








Trammell, et al.        Expires January 15, 2009               [Page 44]

Internet-Draft                 IPFIX Files                     July 2008


Appendix B.  Applicability of IPFIX Files to NetFlow V9 flow storage

   As the IPFIX Message format is nearly a superset of the NetFlow V9
   packet format, IPFIX Files can be used for store NetFlow V9 data
   relatively easily.  This section describes a method for doing so.
   The differences between the two protocols are outlined in
   Appendix B.1 below.  A simple, lightweight, message-for-message
   translation method for transforming V9 Packets into IPFIX Messages
   for storage within IPFIX Files is described in Appendix B.2.  An
   example of this translation method is given in Appendix B.3.

B.1.  Comparing NetFlow V9 to IPFIX

   With a few caveats, the IPFIX Protocol is a superset of the NetFlow
   V9 protocol, having evolved from it largely through a process of
   feature addition to bring it into compliance with the IPFIX
   Requirements and the needs of stakeholders within the IPFIX Working
   Group.  This appendix outlines the differences between the two
   protocols.  It is informative only, and presented as an exploration
   of the two protocols to motivate the usage of IPFIX Files to store
   V9-collected flow data.

B.1.1.  Message Header Format

   Both NetFlow V9 and IPFIX use streams of messages prefixed by a
   message header, though the message header differs significantly
   between the two.  Note that in NetFlow V9 terminology, these messages
   are called packets, and messages must be delimited by datagram
   boundaries.  IPFIX does not have this constraint.  The header formats
   are detailed below:

    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |       Version Number          |            Count              |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                           sysUpTime                           |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                           UNIX Secs                           |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                       Sequence Number                         |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                        Source ID                              |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

                Figure 11: NetFlow V9 Packet Header Format





Trammell, et al.        Expires January 15, 2009               [Page 45]

Internet-Draft                 IPFIX Files                     July 2008


    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |       Version Number          |            Length             |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                           Export Time                         |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                       Sequence Number                         |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                    Observation Domain ID                      |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

                  Figure 12: IPFIX Message Header Format

   Version Number:   The IPFIX Version Number MUST be 10, while the
      NetFlow V9 Version Number MUST be 9.

   Length vs. Count:   The Count field in the NetFlow V9 packet header
      counts records in the message (including data and template
      records), while the Length field in the IPFIX Message Header
      counts octets in the message.  Note that this implies that NetFlow
      V9 collectors must rely on datagram boundaries or some other
      external delimeter; or otherwise must completely consume a message
      before finding its end.

   System Uptime:   System uptime in milliseconds is exported in the
      NetFlow V9 packet header.  This field is not present in the IPFIX
      Message Header, and must be exported using an IPFIX Option if
      required.

   Export Time:   Aside from being called UNIX Secs in the NetFlow V9
      packet header specification, the export time in seconds since 1
      January 1970 at 0000 UTC appears in both NetFlow V9 and IPFIX
      message headers.

   Sequence Number:   The NetFlow V9 Sequence Number counts packets,
      while the IPFIX Sequence Number counts records in Data Sets.  Both
      are scoped to Observation Domain.

   Observation Domain ID:   Similarly, the NetFlow V9 sourceID has
      become the IPFIX Observation Domain ID.

B.1.2.  Set Header Format

   Set headers are identical between NetFlow V9 and IPFIX; that is, each
   Set (FlowSet in NetFlow V9 terminology) is prefixed by a 4-byte set
   header containing the Set ID and the length of the set in octets.




Trammell, et al.        Expires January 15, 2009               [Page 46]

Internet-Draft                 IPFIX Files                     July 2008


   Note that the special Set IDs are different between IPFIX and NetFlow
   V9.  IPFIX Template Sets are identified by Set ID 2, while NetFlow V9
   Template FlowSets are identified by Set ID 0.  Similarly, IPFIX
   Options Template Sets are identified by Set ID 3, while NetFlow V9
   Options Template FlowSets are identified by Set ID 1.

   Both protocols reserve Set IDs 0-255, and use Set IDs 256-65535 for
   Data Sets (or FlowSets, in NetFlow V9 terminology).

B.1.3.  Template Format

   Template FlowSets in NetFlow V9 support a subset of functionality of
   those in IPFIX.  Specifically, NetFlow V9 does not have any support
   for vendor-specific Information Elements as IPFIX does, so there is
   no enterprise bit or facility for associating a private enterprise
   number with an information element.

   Options Template FlowSets in NetFlow V9 are similar to Options
   Template Sets in IPFIX in the same way.

B.1.4.  Information Model

   The NetFlow V9 field type definitions are a compatible subset of, and
   have evolved in concert with, the IPFIX Information Model.  IPFIX
   Information Element numbers in the range 1-127 are defined by the
   IPFIX Information Model [RFC5102] to be compatible with the
   corresponding NetFlow V9 field types.

B.1.5.  Template Management

   NetFlow V9 has no concept of a Transport Session as in IPFIX, as
   NetFlow V9 was designed with a connectionless transport in mind.
   Template IDs are therefore scoped to an Exporting Process lifetime
   (i.e., an Exporting Process instance between restarts).  There is no
   facility in NetFlow V9 as in IPFIX for Template withdrawal or
   Template ID reuse.  Template retransmission at the Exporter works as
   in UDP--+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |       Version Number          |            Length             |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                           Export Time                         |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                       Sequence Number                         |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                    Observation Domain ID                      |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

                  Figure 12: IPFIX Message Header Format

   Version Number:   The IPFIX Version Number MUST be 10, while the
      NetFlow V9 Version Number MUST be 9.

   Length vs. Count:   The Count field in the NetFlow V9 packet header
      counts records in the message (including data and template
      records), while the Length field in the IPFIX Message Header
      counts octets in the message.  Note that this implies that NetFlow
      V9 collectors must rely on datagram boundaries or some other
      external delimeter; or otherwise must completely consume a message
      before finding its end.

   System Uptime:   System uptime in milliseconds is exported in the
      NetFlow V9 packet header.  This field is not present in the IPFIX
      Message Header, and must be exported using an IPFIX Option if
      required.

   Export Time:   Aside from being called UNIX Secs in the NetFlow V9
      packet header specification, the export time in seconds since 1
      January 1970 at 0000 UTC appears in both NetFlow V9 and IPFIX
      message headers.

   Sequence Number:   The NetFlow V9 Sequence Number counts packets,
      while the IPFIX Sequence Number counts records in Data Sets.  Both
      are scoped to Observation Domain.

   Observation Domain ID:   Similarly, the NetFlow V9 sourceID has
      become the IPFIX Observation Domain ID.

B.1.2.  Set Header Format

   Set headers are identical between NetFlow V9 and IPFIX; that is, each
   Set (FlowSet in NetFlow V9 terminology) is prefixed by a 4-byte set
   header containing the Set ID and the length of the set in octets.




Trammell, et al.        Expires January 15, 2009               [Page 46]

Internet-Draft                 IPFIX Files                     July 2008


   Note that the special Set IDs are different between IPFIX and NetFlow
   V9.  IPFIX Template Sets are identified by Set ID 2, while NetFlow V9
   Template FlowSets are identified by Set ID 0.  Similarly, IPFIX
   Options Template Sets are identified by Set ID 3, while NetFlow V9
   Options Template FlowSets are identified by Set ID 1.

   Both protocols reserve Set IDs 0-255, and use Set IDs 256-65535 for
   Data Sets (or FlowSets, in NetFlow V9 terminology).

B.1.3.  Template Format

   Template FlowSets in NetFlow V9 support a subset of functionality of
   those in IPFIX.  Specifically, NetFlow V9 does not have any support
   for vendor-specific Information Elements as IPFIX does, so there is
   no enterprise bit or facility for associating a private enterprise
   number with an information element.

   Options Template FlowSets in NetFlow V9 are similar to Options
   Template Sets in IPFIX in the same way.

B.1.4.  Information Model

   The NetFlow V9 field type definitions are a compatible subset of, and
   have evolved in concert with, the IPFIX Information Model.  IPFIX
   Information Element numbers in the range 1-127 are defined by the
   IPFIX Information Model [RFC5102] to be compatible with the
   corresponding NetFlow V9 field types.

B.1.5.  Template Management

   NetFlow V9 has no concept of a Transport Session as in IPFIX, as
   NetFlow V9 was designed with a connectionless transport in mind.
   Template IDs are therefore scoped to an Exporting Process lifetime
   (i.e., an Exporting Process instance between restarts).  There is no
   facility in NetFlow V9 as in IPFIX for Template withdrawal or
   Template ID reuse.  Template retransmission at the Exporter works as
   in UDbased IPFIX Exporting Processes.

B.1.6.  Transport

   In practice, though NetFlow V9 is designed to be transport-
   independent, it is transported only over UDP.  There is no facility
   as in IPFIX for full connection-oriented transport without datagram
   boundaries, due to the use of a record count field as opposed to a
   message length field in the packet header.  There is no support in
   NetFlow V9 for transport layer security via TLS or DTLS.





Trammell, et al.        Expires January 15, 2009               [Page 47]

Internet-Draft                 IPFIX Files                     July 2008


B.2.  A Method for Transforming NetFlow V9 messages to IPFIX

   This appendix describes a method for transforming NetFlow V9 Packets
   into IPFIX Messages, which can be used to store NetFlow V9 data in
   IPFIX Files.  A process transforming NetFlow V9 Packets into IPFIX
   Messages must handle the fact that NetFlow V9 Packets and IPFIX
   Messages are framed differently, that sequence numbering works
   differently, and that the NetFlow V9 field type definitions are only
   compatible with the IPFIX Information Model field and/or information
   element numbers below Information Element number 128.

   For each incoming NetFlow V9 packet, the transformation process must:

   1.  Verify that the Version field of the packet header is 9.

   2.  Verify that the Sequence Number field of the packet header is
       valid.

   3.  Scan the packet to:

       1.  verify that it contains no Templates with field numbers
           outside the range 1-127;

       2.  verify that it contains no FlowSets with Set IDs between 2
           and 255 inclusive;

       3.  verify that it contains the number of records in FlowSets,
           Template FlowSets, and Options Template FlowSets declared in
           the Count field of the packet header; and

       4.  count the number of records in FlowSets for calculating the
           IPFIX Sequence number.

   4.  Calculate a Sequence Number for each IPFIX Observation Domain by
       storing the last Sequence Number sent for each Observation Domain
       plus the count of records in FlowSets in the previous step to be
       sent as the Sequence Number for the IPFIX Message within that
       Observation Domain following this one.

   5.  Generate a new IPFIX Message Header with:

       1.  a Version field of 10;

       2.  a Length field with the number of octets in the IPFIX
           Message, generally available by subtracting 4 from the length
           of the NetFlow V9 packet as returned from the transport layer
           (accounting for the difference in message header lengths);




Trammell, et al.        Expires January 15, 2009               [Page 48]

Internet-Draft                 IPFIX Files                     July 2008


       3.  the Sequence Number calculated for this message by the
           Sequence Number calculation step; and

       4.  Export Time and Observation Domain ID taken from the UNIX
           secs and Source ID fields of the NetFlow V9 packet header,
           respectively.

   6.  Copy each FlowSet from the Netflow V9 packet to the IPFIX Message
       after the header.  Replace Set ID 0 with Set ID 2 for Template
       Sets, and Set ID 1 with Set ID 3 for Options Template Sets.

   Note that this process loses system uptime information; if such
   information is required, the transformation process will have to
   export that information using IPFIX Options.  This may require a more
   sophisticated transformation process structure.

B.3.  NetFlow V9 Transformation Example

   The following two figures show a single NetFlow V9 packet with
   templates and the corresponding IPFIX Message, exporting a single
   flow record representing 60,303 octets sent from 192.0.2.2 to
   192.0.2.3.  This would be the 3rd packet exported in Observation
   Domain 33 from the NetFlow V9 exporter, containing records starting
   with the 12th record (packet and record sequence numbers count from
   0).

   The ** symbol in the IPFIX example shows those fiP-based IPFIX Exporting Processes.

B.1.6.  Transport

   In practice, though NetFlow V9 is designed to be transport-
   independent, it is transported only over UDP.  There is no facility
   as in IPFIX for full connection-oriented transport without datagram
   boundaries, due to the use of a record count field as opposed to a
   message length field in the packet header.  There is no support in
   NetFlow V9 for transport layer security via TLS or DTLS.





Trammell, et al.        Expires January 15, 2009               [Page 47]

Internet-Draft                 IPFIX Files                     July 2008


B.2.  A Method for Transforming NetFlow V9 messages to IPFIX

   This appendix describes a method for transforming NetFlow V9 Packets
   into IPFIX Messages, which can be used to store NetFlow V9 data in
   IPFIX Files.  A process transforming NetFlow V9 Packets into IPFIX
   Messages must handle the fact that NetFlow V9 Packets and IPFIX
   Messages are framed differently, that sequence numbering works
   differently, and that the NetFlow V9 field type definitions are only
   compatible with the IPFIX Information Model field and/or information
   element numbers below Information Element number 128.

   For each incoming NetFlow V9 packet, the transformation process must:

   1.  Verify that the Version field of the packet header is 9.

   2.  Verify that the Sequence Number field of the packet header is
       valid.

   3.  Scan the packet to:

       1.  verify that it contains no Templates with field numbers
           outside the range 1-127;

       2.  verify that it contains no FlowSets with Set IDs between 2
           and 255 inclusive;

       3.  verify that it contains the number of records in FlowSets,
           Template FlowSets, and Options Template FlowSets declared in
           the Count field of the packet header; and

       4.  count the number of records in FlowSets for calculating the
           IPFIX Sequence number.

   4.  Calculate a Sequence Number for each IPFIX Observation Domain by
       storing the last Sequence Number sent for each Observation Domain
       plus the count of records in FlowSets in the previous step to be
       sent as the Sequence Number for the IPFIX Message within that
       Observation Domain following this one.

   5.  Generate a new IPFIX Message Header with:

       1.  a Version field of 10;

       2.  a Length field with the number of octets in the IPFIX
           Message, generally available by subtracting 4 from the length
           of the NetFlow V9 packet as returned from the transport layer
           (accounting for the difference in message header lengths);




Trammell, et al.        Expires January 15, 2009               [Page 48]

Internet-Draft                 IPFIX Files                     July 2008


       3.  the Sequence Number calculated for this message by the
           Sequence Number calculation step; and

       4.  Export Time and Observation Domain ID taken from the UNIX
           secs and Source ID fields of the NetFlow V9 packet header,
           respectively.

   6.  Copy each FlowSet from the Netflow V9 packet to the IPFIX Message
       after the header.  Replace Set ID 0 with Set ID 2 for Template
       Sets, and Set ID 1 with Set ID 3 for Options Template Sets.

   Note that this process loses system uptime information; if such
   information is required, the transformation process will have to
   export that information using IPFIX Options.  This may require a more
   sophisticated transformation process structure.

B.3.  NetFlow V9 Transformation Example

   The following two figures show a single NetFlow V9 packet with
   templates and the corresponding IPFIX Message, exporting a single
   flow record representing 60,303 octets sent from 192.0.2.2 to
   192.0.2.3.  This would be the 3rd packet exported in Observation
   Domain 33 from the NetFlow V9 exporter, containing records starting
   with the 12th record (packet and record sequence numbers count from
   0).

   The ** symbol in the IPFIX example shows those elds that required
   modification from the NetFlow V9 packet by the transformation
   process.






















Trammell, et al.        Expires January 15, 2009               [Page 49]

Internet-Draft                 IPFIX Files                     July 2008


                        1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |           Version = 9          |         Count = 2             |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |               Uptime = 3750405 ms (1:02:30.405)               |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |   Export Time = 1171557627 epoch sec (2007-02-15 16:40:27)    |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                     Sequence Number = 2                       |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                 Observation Domain ID = 33                    |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |           Set ID = 0          |       Set Length = 20         |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |       Template ID = 256       |       Field Count = 3         |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   | IPV4_SRC_ADDR           =   8 |       Field Length = 4        |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   | IPV4_DST_ADDR           =  12 |       Field Length = 4        |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   | IN_BYTES                =   1 |       Field Length = 4        |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |          Set ID = 256         |       Set Length = 16         |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                         IPV4_SRC_ADDR                         |
   |                           192.0.2.2                           |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                         IPV4_DST_ADDR                         |
   |                           192.0.2.3                           |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                           IN_BYTES                            |
   |                             60303                             |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

                   Figure 13: Example NetFlow V9 Packet















Trammell, et al.        Expires January 15, 2009               [Page 50]

Internet-Draft                 IPFIX Files                     July 2008


                       1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   | **       Version = 10         | **      Length = 52           |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |   Export Time = 1171557627 epoch sec (2007-02-15 16:40:27)    |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   | **                   Sequence Number = 11                     |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                   Observation Domain ID = 33                  |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   | **         Set ID = 2         |       Set Length = 20         |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |       Template ID = 256       |       Field Count  = 3        |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |0| sourceIPv4Address      =  8 |       Field Length = 4        |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |0| destinationIPv4Address = 12 |       Field Length = 4        |
   +-+-+-+-+-+-+-+-+-+-+-+fields that required
   modification from the NetFlow V9 packet by the transformation
   process.






















Trammell, et al.        Expires January 15, 2009               [Page 49]

Internet-Draft                 IPFIX Files                     July 2008


                        1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |           Version = 9          |         Count = 2             |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |               Uptime = 3750405 ms (1:02:30.405)               |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |   Export Time = 1171557627 epoch sec (2007-02-15 16:40:27)    |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                     Sequence Number = 2                       |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                 Observation Domain ID = 33                    |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |           Set ID = 0          |       Set Length = 20         |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |       Template ID = 256       |       Field Count = 3         |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   | IPV4_SRC_ADDR           =   8 |       Field Length = 4        |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   | IPV4_DST_ADDR           =  12 |       Field Length = 4        |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   | IN_BYTES                =   1 |       Field Length = 4        |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |          Set ID = 256         |       Set Length = 16         |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                         IPV4_SRC_ADDR                         |
   |                           192.0.2.2                           |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                         IPV4_DST_ADDR                         |
   |                           192.0.2.3                           |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                           IN_BYTES                            |
   |                             60303                             |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

                   Figure 13: Example NetFlow V9 Packet















Trammell, et al.        Expires January 15, 2009               [Page 50]

Internet-Draft                 IPFIX Files                     July 2008


                       1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   | **       Version = 10         | **      Length = 52           |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |   Export Time = 1171557627 epoch sec (2007-02-15 16:40:27)    |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   | **                   Sequence Number = 11                     |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                   Observation Domain ID = 33                  |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   | **         Set ID = 2         |       Set Length = 20         |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |       Template ID = 256       |       Field Count  = 3        |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |0| sourceIPv4Address      =  8 |       Field Length = 4        |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |0| destinationIPv4Address = 12 |       Field Length = 4        |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |0| octetDeltaCount        =  1 |       Field Length = 4        |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |          Set ID = 256         |       Set Length = 16         |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                       sourceIPv4Address                       |
   |                           192.0.2.2                           |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                     destinationIPv4Address                    |
   |                           192.0.2.3                           |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                        octetDeltaCount                        |
   |                             60303                             |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

              Figure 14: Corresponding Example IPFIX Message


Authors' Addresses

   Brian Trammell
   Hitachi Europe
   c/o ETH Zurich
   Gloriastrasse 35
   8092 Zurich
   Switzerland

   Phone: +41 44 632 70 13
   Email: brian.trammell@hitachi-eu.com




Trammell, et al.        Expires January 15, 2009               [Page 51]

Internet-Draft                 IPFIX Files                     July 2008


   Elisa Boschi
   Hitachi Europe
   c/o ETH Zurich
   Gloriastrasse 35
   8092 Zurich
   Switzerland

   Phone: +41 44 632 70 57
   Email: elisa.boschi@hitachi-eu.com


   Lutz Mark
   Fraunhofer IFAM
   Weiner Str. 12
   38259 Bremen
   Germany

   Phone: +49 421 2246206
   Email: lutz.mark@ifam.fraunhofer.de


   Tanja Zseby
   Fraunhofer Institute for Open Communication Systems
   Kaiserin-Augusta-Allee 31
   10589 Berlin
   Germany

   Phone: +49 30 3463 7153
   Email: tanja.zseby@fokus.fraunhofer.de


   Arno Wagner
   Swiss Federal Institute of Technology Zurich
   Gloriastrasse 35
   8092 Zurich
   Switzerland

   Phone: +41 44 632 70 04
   Email: arno@wagner.name












Trammell, et al.        Expires January 15, 2009               [Page 52]

Internet-Draft                 IPFIX Files                     July 2008


Full Copyright Statement

   Copyright (C) The IETF Trust (2008).

   This document is subject to the rights, licenses and restrictions
   contained in BCP 78, and except as set forth therein, the authors
   retain all their rights.

   This document and the information contained herein are provided on an
   "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS
   OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY, THE IETF TRUST AND
   THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS
   OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF
   THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED
   WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.


Intellectual Property

   The IETF takes no position regarding the validity or scope of any
   Intellectual Property Rights or other rights that might be claimed to
   pertain to the implementation or use of the technology described in
   this document or the extent to which any license under such rights
   might or might not be available; nor does it represent that it has
   made any independent effort to identify any such rights.  Information
   on the procedures with respect to rights in RFC documents can be
   found in BCP 78 and BCP 79.

   Copies of IPR disclosures made to the IETF Secretariat and any
   assurances of licenses to be made available, or the result of an
   attempt made to obtain a general license or permission for the use of
 -+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |0| octetDeltaCount        =  1 |       Field Length = 4        |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |          Set ID = 256         |       Set Length = 16         |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                       sourceIPv4Address                       |
   |                           192.0.2.2                           |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                     destinationIPv4Address                    |
   |                           192.0.2.3                           |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                        octetDeltaCount                        |
   |                             60303                             |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

              Figure 14: Corresponding Example IPFIX Message


Authors' Addresses

   Brian Trammell
   Hitachi Europe
   c/o ETH Zurich
   Gloriastrasse 35
   8092 Zurich
   Switzerland

   Phone: +41 44 632 70 13
   Email: brian.trammell@hitachi-eu.com




Trammell, et al.        Expires January 15, 2009               [Page 51]

Internet-Draft                 IPFIX Files                     July 2008


   Elisa Boschi
   Hitachi Europe
   c/o ETH Zurich
   Gloriastrasse 35
   8092 Zurich
   Switzerland

   Phone: +41 44 632 70 57
   Email: elisa.boschi@hitachi-eu.com


   Lutz Mark
   Fraunhofer IFAM
   Weiner Str. 12
   38259 Bremen
   Germany

   Phone: +49 421 2246206
   Email: lutz.mark@ifam.fraunhofer.de


   Tanja Zseby
   Fraunhofer Institute for Open Communication Systems
   Kaiserin-Augusta-Allee 31
   10589 Berlin
   Germany

   Phone: +49 30 3463 7153
   Email: tanja.zseby@fokus.fraunhofer.de


   Arno Wagner
   Swiss Federal Institute of Technology Zurich
   Gloriastrasse 35
   8092 Zurich
   Switzerland

   Phone: +41 44 632 70 04
   Email: arno@wagner.name












Trammell, et al.        Expires January 15, 2009               [Page 52]

Internet-Draft                 IPFIX Files                     July 2008


Full Copyright Statement

   Copyright (C) The IETF Trust (2008).

   This document is subject to the rights, licenses and restrictions
   contained in BCP 78, and except as set forth therein, the authors
   retain all their rights.

   This document and the information contained herein are provided on an
   "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS
   OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY, THE IETF TRUST AND
   THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS
   OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF
   THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED
   WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.


Intellectual Property

   The IETF takes no position regarding the validity or scope of any
   Intellectual Property Rights or other rights that might be claimed to
   pertain to the implementation or use of the technology described in
   this document or the extent to which any license under such rights
   might or might not be available; nor does it represent that it has
   made any independent effort to identify any such rights.  Information
   on the procedures with respect to rights in RFC documents can be
   found in BCP 78 and BCP 79.

   Copies of IPR disclosures made to the IETF Secretariat and any
   assurances of licenses to be made available, or the result of an
   attempt made to obtain a general license or permission for the use of  such proprietary rights by implementers or users of this
   specification can be obtained from the IETF on-line IPR repository at
   http://www.ietf.org/ipr.

   The IETF invites any interested party to bring to its attention any
   copyrights, patents or patent applications, or other proprietary
   rights that may cover technology that may be required to implement
   this standard.  Please address the information to the IETF at
   ietf-ipr@ietf.org.











Trammell, et al.        Expires January 15, 2009               [Page 53]


  








--------------050305070204080904090002--

--===============1415078296==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

_______________________________________________
IPFIX mailing list
IPFIX@ietf.org
https://www.ietf.org/mailman/listinfo/ipfix

--===============1415078296==--



   such proprietary rights by implementers or users of this
   specification can be obtained from the IETF on-line IPR repository at
   http://www.ietf.org/ipr.

   The IETF invites any interested party to bring to its attention any
   copyrights, patents or patent applications, or other proprietary
   rights that may cover technology that may be required to implement
   this standard.  Please address the information to the IETF at
   ietf-ipr@ietf.org.











Trammell, et al.        Expires January 15, 2009               [Page 53]

--------------050305070204080904090002-- --===============1415078296== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ IPFIX mailing list IPFIX@ietf.org https://www.ietf.org/mailman/listinfo/ipfix --===============1415078296==-- From ipfix-bounces@ietf.org Thu Aug 28 08:50:06 2008 Return-Path: X-Original-To: ipfix-archive@optimus.ietf.org Delivered-To: ietfarch-ipfix-archive@core3.amsl.com Received: from [127.0.0.1] (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 4259C3A6CD5; Thu, 28 Aug 2008 08:50:06 -0700 (PDT) X-Original-To: ipfix@core3.amsl.com Delivered-To: ipfix@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 206743A6880 for ; Thu, 28 Aug 2008 08:50:05 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -3.344 X-Spam-Level: X-Spam-Status: No, score=-3.344 tagged_above=-999 required=5 tests=[BAYES_20=-0.74, MIME_QP_LONG_LINE=1.396, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id KMKL69E0SSwQ for ; Thu, 28 Aug 2008 08:49:57 -0700 (PDT) Received: from smtp.ee.ethz.ch (smtp.ee.ethz.ch [129.132.2.219]) by core3.amsl.com (Postfix) with ESMTP id 510BB3A6B48 for ; Thu, 28 Aug 2008 08:49:56 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by smtp.ee.ethz.ch (Postfix) with ESMTP id 731DED93B0; Thu, 28 Aug 2008 17:49:26 +0200 (MEST) X-Virus-Scanned: by amavisd-new on smtp.ee.ethz.ch Received: from smtp.ee.ethz.ch ([127.0.0.1]) by localhost (.ee.ethz.ch [127.0.0.1]) (amavisd-new, port 10024) with LMTP id UihHZPHxT1AG; Thu, 28 Aug 2008 17:49:26 +0200 (MEST) Received: from pb-10072.ethz.ch (pb-10072.ethz.ch [82.130.103.195]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) (Authenticated sender: briant) by smtp.ee.ethz.ch (Postfix) with ESMTP id E317ED9320; Thu, 28 Aug 2008 17:49:25 +0200 (MEST) Message-Id: <9B5C5A4E-2099-43B4-953E-674E38314582@tik.ee.ethz.ch> From: Brian Trammell To: Gerhard Muenz In-Reply-To: <48A5B5ED.5040107@informatik.uni-tuebingen.de> Mime-Version: 1.0 (Apple Message framework v928.1) Date: Thu, 28 Aug 2008 17:49:25 +0200 References: <48A5B5ED.5040107@informatik.uni-tuebingen.de> X-Mailer: Apple Mail (2.928.1) Cc: IETF IPFIX Working Group Subject: Re: [IPFIX] WG last call on draft-ietf-ipfix-file-02 X-BeenThere: ipfix@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: IPFIX WG discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Sender: ipfix-bounces@ietf.org Errors-To: ipfix-bounces@ietf.org Gerhard, Thanks for your detailed review and comments. I'll address these by = type. I. Comments related to the length and level of detail of the document: The IPFIX File draft differs from other IPFIX drafts in that it 1. = contains its own requirements (analogous to 3917 and 5101 being the = same document) and 2. contains more motivating text than other drafts, = due to its somewhat unusual nature within the IETF (standardizing a = file format without binding it to some specific transport). The = authors believe that the inclusion and length of the requirements and = motivation sections (sections 4 and 5) is warranted by these = considerations. II. Typographical, editorial and stylistic comments: Thanks for these, there were quite a few typos/capitalization issues = we'd not caught yet. Stylistic suggestions have been incorporated = where they improve the clarity of the document. III. Terminology confusion: There are a couple of terms invented here in order to describe = concepts 1. for which From ipfix-bounces@ietf.org Thu Aug 28 08:50:06 2008 Return-Path: X-Original-To: ipfix-archive@lists.ietf.org Delivered-To: ietfarch-ipfix-archive@core3.amsl.com Received: from [127.0.0.1] (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 4259C3A6CD5; Thu, 28 Aug 2008 08:50:06 -0700 (PDT) X-Original-To: ipfix@core3.amsl.com Delivered-To: ipfix@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 206743A6880 for ; Thu, 28 Aug 2008 08:50:05 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -3.344 X-Spam-Level: X-Spam-Status: No, score=-3.344 tagged_above=-999 required=5 tests=[BAYES_20=-0.74, MIME_QP_LONG_LINE=1.396, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id KMKL69E0SSwQ for ; Thu, 28 Aug 2008 08:49:57 -0700 (PDT) Received: from smtp.ee.ethz.ch (smtp.ee.ethz.ch [129.132.2.219]) by core3.amsl.com (Postfix) with ESMTP id 510BB3A6B48 for ; Thu, 28 Aug 2008 08:49:56 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by smtp.ee.ethz.ch (Postfix) with ESMTP id 731DED93B0; Thu, 28 Aug 2008 17:49:26 +0200 (MEST) X-Virus-Scanned: by amavisd-new on smtp.ee.ethz.ch Received: from smtp.ee.ethz.ch ([127.0.0.1]) by localhost (.ee.ethz.ch [127.0.0.1]) (amavisd-new, port 10024) with LMTP id UihHZPHxT1AG; Thu, 28 Aug 2008 17:49:26 +0200 (MEST) Received: from pb-10072.ethz.ch (pb-10072.ethz.ch [82.130.103.195]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) (Authenticated sender: briant) by smtp.ee.ethz.ch (Postfix) with ESMTP id E317ED9320; Thu, 28 Aug 2008 17:49:25 +0200 (MEST) Message-Id: <9B5C5A4E-2099-43B4-953E-674E38314582@tik.ee.ethz.ch> From: Brian Trammell To: Gerhard Muenz In-Reply-To: <48A5B5ED.5040107@informatik.uni-tuebingen.de> Mime-Version: 1.0 (Apple Message framework v928.1) Date: Thu, 28 Aug 2008 17:49:25 +0200 References: <48A5B5ED.5040107@informatik.uni-tuebingen.de> X-Mailer: Apple Mail (2.928.1) Cc: IETF IPFIX Working Group Subject: Re: [IPFIX] WG last call on draft-ietf-ipfix-file-02 X-BeenThere: ipfix@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: IPFIX WG discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Sender: ipfix-bounces@ietf.org Errors-To: ipfix-bounces@ietf.org Gerhard, Thanks for your detailed review and comments. I'll address these by = type. I. Comments related to the length and level of detail of the document: The IPFIX File draft differs from other IPFIX drafts in that it 1. = contains its own requirements (analogous to 3917 and 5101 being the = same document) and 2. contains more motivating text than other drafts, = due to its somewhat unusual nature within the IETF (standardizing a = file format without binding it to some specific transport). The = authors believe that the inclusion and length of the requirements and = motivation sections (sections 4 and 5) is warranted by these = considerations. II. Typographical, editorial and stylistic comments: Thanks for these, there were quite a few typos/capitalization issues = we'd not caught yet. Stylistic suggestions have been incorporated = where they improve the clarity of the document. III. Terminology confusion: There are a couple of terms invented here in order to describe = concepts 1. for which ththere is no accepted terminology but 2. which = appear only for purposes of background. One of these is "IPFIX Message = format", which is intended to mean "the encoding and framing of IPFIX = Messages defined by RFC 5101 as used with the information model = defined by RFC 5102", or less formally, "IPFIX off the wire", _not_ = the format of an individual message. Your comments indicate some = confusion here, and I would appeal to the working group to suggest an = alternative if this is a problem. Similarly, an "IPFIX Message stream" is a collection of IPFIX Messages = in some loose order. This is applying the word "stream" as it is used = when talking about storage, where there is no connotation in English = of an unbounded sequence. I'm going to suggest we leave this term, as = the suggested alternatives have their own problems: "Sequence" = connotes strict ordering (not the case; IPFIX Messages can be = reordered within a File containing a Message stream for e.g. template = consistency.) "Series" has a strict and nonapplicable mathematical = meaning. Alternately, a question to the WG: should we define "IPFIX Message = stream" and "IPFIX Message format" as intermediate terms in the = Terminology section of the document? The issues regarding the interchangeability of "IPFIX File" and "IPFIX = Transport Session" given the fact that Sessions may span Files have = been fixed; many thanks for pointing this one out. V9-related = terminology has been fixed, too; I got a bit confused in a couple of = places switching between V9 and IPFIX. IV. Technical issues: Time window resolution finer than seconds: Perhaps. We were trying to = avoid the explosion of timestamp IEs we have elsewhere in the = information model. Keep in mind that the Time Window is meant, in = multi-file environments, to determine whether a file must be scanned = to find information about a specific moment in time. Going below the = second-level resolution, at least in my experience, is not necessary = _for this use case_. Applicability of opaqueOctets: Indeed, one could create an enterprise- = specific IE and not export ET info about it, then it's opaque. But 1. = we still need to address the how to encap non-IPFIX info in an IPFIX = stream and 2. don't necessarily want to tell people go get a PEN to do = it. I'm inclined to keep opaqueOctets. Thanks again! - Brian On Aug 15, 2008, at 6:59 PM, Gerhard Muenz wrote: > > Dear all, > > Here is a review for draft-ietf-ipfix-file-02. > It is a very lengthy document. Many things are repeated. Maybe you can > shorten it a little bit and make it more concise. > Also, there are many unnecessary filler words like "note that". I do = > not > really like that in a standard document. > > More comments inline. > > Gerhard > > >> Abstract >> >> This document describes a file format for the storage of flow data >> based upon the IPFIX Message format. It proposes a set of >> requirements for flat-file, binary flow data file formats, then > > flat, binary flow data file formats? > >> applies the IPFIX message format to these requirements to build a = >> new >> file format. This IPFIX-based file format is designed to = >> facilitate >> interoperability and reusability among a wide variety of flow >> storage, processing, and analysis tools. > >> 1.1. IPFIX Documents Overview >> >> "Specification of the IPFIX Protocol for the Exchange of IP Traffic >> Flow Information" [RFC5101] and its associated documents define the > > Which are the associated documents? Ah, you mention them in the next > paragraph. In this section, the division into paragraphs seems to be > arbitrary. > >> IPFIX Protocol, which provides network engineers and administrators >> with access to IP traffic flow information. >> >> "Architecture for IP Flow Information Export" [I-D.ietf-ipfix-arch] >> defines the architecture for the export of measured IP flow >> information out of an IPFIX Exporting Process to an IPFIX = >> Collecting >> Process, and the ere is no accepted terminology but 2. which = appear only for purposes of background. One of these is "IPFIX Message = format", which is intended to mean "the encoding and framing of IPFIX = Messages defined by RFC 5101 as used with the information model = defined by RFC 5102", or less formally, "IPFIX off the wire", _not_ = the format of an individual message. Your comments indicate some = confusion here, and I would appeal to the working group to suggest an = alternative if this is a problem. Similarly, an "IPFIX Message stream" is a collection of IPFIX Messages = in some loose order. This is applying the word "stream" as it is used = when talking about storage, where there is no connotation in English = of an unbounded sequence. I'm going to suggest we leave this term, as = the suggested alternatives have their own problems: "Sequence" = connotes strict ordering (not the case; IPFIX Messages can be = reordered within a File containing a Message stream for e.g. template = consistency.) "Series" has a strict and nonapplicable mathematical = meaning. Alternately, a question to the WG: should we define "IPFIX Message = stream" and "IPFIX Message format" as intermediate terms in the = Terminology section of the document? The issues regarding the interchangeability of "IPFIX File" and "IPFIX = Transport Session" given the fact that Sessions may span Files have = been fixed; many thanks for pointing this one out. V9-related = terminology has been fixed, too; I got a bit confused in a couple of = places switching between V9 and IPFIX. IV. Technical issues: Time window resolution finer than seconds: Perhaps. We were trying to = avoid the explosion of timestamp IEs we have elsewhere in the = information model. Keep in mind that the Time Window is meant, in = multi-file environments, to determine whether a file must be scanned = to find information about a specific moment in time. Going below the = second-level resolution, at least in my experience, is not necessary = _for this use case_. Applicability of opaqueOctets: Indeed, one could create an enterprise- = specific IE and not export ET info about it, then it's opaque. But 1. = we still need to address the how to encap non-IPFIX info in an IPFIX = stream and 2. don't necessarily want to tell people go get a PEN to do = it. I'm inclined to keep opaqueOctets. Thanks again! - Brian On Aug 15, 2008, at 6:59 PM, Gerhard Muenz wrote: > > Dear all, > > Here is a review for draft-ietf-ipfix-file-02. > It is a very lengthy document. Many things are repeated. Maybe you can > shorten it a little bit and make it more concise. > Also, there are many unnecessary filler words like "note that". I do = > not > really like that in a standard document. > > More comments inline. > > Gerhard > > >> Abstract >> >> This document describes a file format for the storage of flow data >> based upon the IPFIX Message format. It proposes a set of >> requirements for flat-file, binary flow data file formats, then > > flat, binary flow data file formats? > >> applies the IPFIX message format to these requirements to build a = >> new >> file format. This IPFIX-based file format is designed to = >> facilitate >> interoperability and reusability among a wide variety of flow >> storage, processing, and analysis tools. > >> 1.1. IPFIX Documents Overview >> >> "Specification of the IPFIX Protocol for the Exchange of IP Traffic >> Flow Information" [RFC5101] and its associated documents define the > > Which are the associated documents? Ah, you mention them in the next > paragraph. In this section, the division into paragraphs seems to be > arbitrary. > >> IPFIX Protocol, which provides network engineers and administrators >> with access to IP traffic flow information. >> >> "Architecture for IP Flow Information Export" [I-D.ietf-ipfix-arch] >> defines the architecture for the export of measured IP flow >> information out of an IPFIX Exporting Process to an IPFIX = >> Collecting >> Process, and the babasic terminology used to describe the elements of >> this architecture, per the requirements defined in "Requirements = >> for >> IP Flow Information Export" [RFC3917]. [RFC5101] then covers the > > Confusing as you already mentioned RFC 5101 before. > I suggest modifying the introductory paragraph of this section such = > that > it does not refer to RFC 5101 - or do not repeat it here. > >> details of the method for transporting IPFIX Data Records and >> Templates via a congestion-aware transport protocol from an IPFIX >> Exporting Process to an IPFIX Collecting Process. > > UDP is not congestion-aware. > >> "Information Model for IP Flow Information Export" [RFC5102] >> describes the Information Elements used by IPFIX, including details >> on Information Element naming, numbering, and data type encoding. >> Finally, "IPFIX Applicability" [I-D.ietf-ipfix-as] describes the >> various applications of the IPFIX protocol and their use of >> information exported via IPFIX, and relates the IPFIX = >> architecture to >> other measurement architectures and frameworks. >> >> In addition, "Exporting Type Information for IPFIX Information >> Elements" [I-D.ietf-ipfix-exporting-type] specifies a method for >> encoding Information Model properties within an IPFIX Message = >> stream. >> >> This document references [RFC5101] and the architecture document = >> for >> terminology, defines IPFIX File Writer and IPFIX File Reader in = >> terms >> of the IPFIX Exporting Processes and IPFIX Collecting Process >> definitions from [RFC5101], and extends the IPFIX Information Model >> defined in [RFC5102] to provide new Information Elements for IPFIX >> File metadata. It uses the method described in "Exporting Type >> Information for IPFIX Information Elements" >> [I-D.ietf-ipfix-exporting-type] document to support the self- >> description of IPFIX Files containing enterprise-specific = >> Information >> Elements. >> >> >> 2. Terminology >> >> Terms used in this document that are defined in the Terminology >> section of [RFC5101] are to be interpreted as defined there. > > One more sentence explaining that you define additional terms below. > >> IPFIX File: An IPFIX File is a serialized stream of IPFIX = >> Messages > > A stream usually is continuous while a file has a beginning and an = > end. > Consider: series or sequence of IPFIX Messages. > >> stored on a filesystem. Any IPFIX Message stream that would be >> considered valid when transported one or more of the specified >> IPFIX transports (SCTP, TCP, or UDP) as defined in [RFC5101] is >> considered an IPFIX File for purposes of this document; however, > > What do you mean by "for purposes of this document"? > Possibly remove it? > >> this document extends that definition with recommendations on = >> the >> construction of IPFIX Files that meet the requirements = >> identified >> herein. > > Where is "herein"? in the file? in the document? > >> IPFIX File Reader: An IPFIX File Reader is a Process which reads >> IPFIX Files from a filesystem, and is analogous to an IPFIX > > I'm not a native English speaker, yet "analogous to" sounds very vague > as long as you do not say in which respect the analogy holds. > I do not have a good idea how to express what you want to say. As a > starting point, consider something like: an IPFIX File Reader is = > used as > a replacement of an IPFIX Collection Process. > >> Collecting Process. An IPFIX File Reader MUST behave as an = >> IPFIX >> Collecting Process as outlined in [RFC5101], except as = >> modified by >> this document. >> >> IPFIX File Writer: An IPFIX File Writer is a process which writes >> IPFIX Files to a filesystem, and is analogous to an IPFIX > > same as above > >> Exporting Process. An IPFIX File Writer MUST behave as an IPFIX >> Exporting Process as outlined in [RFC5101], except as modified = >> by >> this document. > >> 3. Design Overview >> >> An IPFIX Fsic terminology used to describe the elements of >> this architecture, per the requirements defined in "Requirements = >> for >> IP Flow Information Export" [RFC3917]. [RFC5101] then covers the > > Confusing as you already mentioned RFC 5101 before. > I suggest modifying the introductory paragraph of this section such = > that > it does not refer to RFC 5101 - or do not repeat it here. > >> details of the method for transporting IPFIX Data Records and >> Templates via a congestion-aware transport protocol from an IPFIX >> Exporting Process to an IPFIX Collecting Process. > > UDP is not congestion-aware. > >> "Information Model for IP Flow Information Export" [RFC5102] >> describes the Information Elements used by IPFIX, including details >> on Information Element naming, numbering, and data type encoding. >> Finally, "IPFIX Applicability" [I-D.ietf-ipfix-as] describes the >> various applications of the IPFIX protocol and their use of >> information exported via IPFIX, and relates the IPFIX = >> architecture to >> other measurement architectures and frameworks. >> >> In addition, "Exporting Type Information for IPFIX Information >> Elements" [I-D.ietf-ipfix-exporting-type] specifies a method for >> encoding Information Model properties within an IPFIX Message = >> stream. >> >> This document references [RFC5101] and the architecture document = >> for >> terminology, defines IPFIX File Writer and IPFIX File Reader in = >> terms >> of the IPFIX Exporting Processes and IPFIX Collecting Process >> definitions from [RFC5101], and extends the IPFIX Information Model >> defined in [RFC5102] to provide new Information Elements for IPFIX >> File metadata. It uses the method described in "Exporting Type >> Information for IPFIX Information Elements" >> [I-D.ietf-ipfix-exporting-type] document to support the self- >> description of IPFIX Files containing enterprise-specific = >> Information >> Elements. >> >> >> 2. Terminology >> >> Terms used in this document that are defined in the Terminology >> section of [RFC5101] are to be interpreted as defined there. > > One more sentence explaining that you define additional terms below. > >> IPFIX File: An IPFIX File is a serialized stream of IPFIX = >> Messages > > A stream usually is continuous while a file has a beginning and an = > end. > Consider: series or sequence of IPFIX Messages. > >> stored on a filesystem. Any IPFIX Message stream that would be >> considered valid when transported one or more of the specified >> IPFIX transports (SCTP, TCP, or UDP) as defined in [RFC5101] is >> considered an IPFIX File for purposes of this document; however, > > What do you mean by "for purposes of this document"? > Possibly remove it? > >> this document extends that definition with recommendations on = >> the >> construction of IPFIX Files that meet the requirements = >> identified >> herein. > > Where is "herein"? in the file? in the document? > >> IPFIX File Reader: An IPFIX File Reader is a Process which reads >> IPFIX Files from a filesystem, and is analogous to an IPFIX > > I'm not a native English speaker, yet "analogous to" sounds very vague > as long as you do not say in which respect the analogy holds. > I do not have a good idea how to express what you want to say. As a > starting point, consider something like: an IPFIX File Reader is = > used as > a replacement of an IPFIX Collection Process. > >> Collecting Process. An IPFIX File Reader MUST behave as an = >> IPFIX >> Collecting Process as outlined in [RFC5101], except as = >> modified by >> this document. >> >> IPFIX File Writer: An IPFIX File Writer is a process which writes >> IPFIX Files to a filesystem, and is analogous to an IPFIX > > same as above > >> Exporting Process. An IPFIX File Writer MUST behave as an IPFIX >> Exporting Process as outlined in [RFC5101], except as modified = >> by >> this document. > >> 3. Design Overview >> >> An IPFIX Filile, as defined by this document, is simply a stream >> containing one or more IPFIX Messages serialized to some = >> filesystem. > > What is simple? Again, consider series or sequence instead of stream. > >> Though any set of valid IPFIX Messages can be serialized into an >> IPFIX File, the specification proposes guidelines designed to ease >> storage and retrieval of flow data using the format. > > Which format? > >> IPFIX Files contain only IPFIX Messages; any file metadata such as >> checksums or export session details are stored using Options within >> the IPFIX Message. This design has several advantages, including >> complete compatibility with the IPFIX Protocol on the wire and free >> manipulability of IPFIX Files through concatenation, appending, and >> splitting (on IPFIX Message boundaries). A schematic of a typical >> file is shown below: >> >> >> +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+ >> | IPFIX File | >> | +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+ | >> | | IPFIX Message | | >> | | +-------------------------------+ | | >> | | | Options Template Set | | | >> | | | Options Template Record | | | >> | | | . . . | | | >> | | +-------------------------------+ | | >> | | +-------------------------------+ | | >> | | | Template Set | | | >> | | | Template Record | | | >> | | | . . . | | | >> | | +-------------------------------+ | | >> | +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+ | >> | | IPFIX Message | | >> | | +-------------------------------+ | | >> | | | Data Set | | | >> | | | Data Record | | | >> | | | . . . | | | >> | | +-------------------------------+ | | >> | | +-------------------------------+ | | >> | | | Data Set | | | >> | | | Data Record | | | >> | | | . . . | | | >> | | +-------------------------------+ | | >> | | . . . | | >> | +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+ | >> | . . . | >> +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+ >> >> Figure 1: Typical File Structure > > Looks a little bit as if the IPFIX File encapsulates the Messages with > header and trailer. > > Consider: > > ^ +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+ > : | IPFIX Message | > : | +-------------------------------+ | > : | | Options Template Set | | > : | | Options Template Record | | > : | | . . . | | > : | +-------------------------------+ | > : | +-------------------------------+ | > : | | Template Set | | > : | | Template Record | | > : | | . . . | | > : | +-------------------------------+ | > : +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+ > : | IPFIX Message | > : | +-----------------------e, as defined by this document, is simply a stream >> containing one or more IPFIX Messages serialized to some = >> filesystem. > > What is simple? Again, consider series or sequence instead of stream. > >> Though any set of valid IPFIX Messages can be serialized into an >> IPFIX File, the specification proposes guidelines designed to ease >> storage and retrieval of flow data using the format. > > Which format? > >> IPFIX Files contain only IPFIX Messages; any file metadata such as >> checksums or export session details are stored using Options within >> the IPFIX Message. This design has several advantages, including >> complete compatibility with the IPFIX Protocol on the wire and free >> manipulability of IPFIX Files through concatenation, appending, and >> splitting (on IPFIX Message boundaries). A schematic of a typical >> file is shown below: >> >> >> +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+ >> | IPFIX File | >> | +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+ | >> | | IPFIX Message | | >> | | +-------------------------------+ | | >> | | | Options Template Set | | | >> | | | Options Template Record | | | >> | | | . . . | | | >> | | +-------------------------------+ | | >> | | +-------------------------------+ | | >> | | | Template Set | | | >> | | | Template Record | | | >> | | | . . . | | | >> | | +-------------------------------+ | | >> | +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+ | >> | | IPFIX Message | | >> | | +-------------------------------+ | | >> | | | Data Set | | | >> | | | Data Record | | | >> | | | . . . | | | >> | | +-------------------------------+ | | >> | | +-------------------------------+ | | >> | | | Data Set | | | >> | | | Data Record | | | >> | | | . . . | | | >> | | +-------------------------------+ | | >> | | . . . | | >> | +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+ | >> | . . . | >> +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+ >> >> Figure 1: Typical File Structure > > Looks a little bit as if the IPFIX File encapsulates the Messages with > header and trailer. > > Consider: > > ^ +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+ > : | IPFIX Message | > : | +-------------------------------+ | > : | | Options Template Set | | > : | | Options Template Record | | > : | | . . . | | > : | +-------------------------------+ | > : | +-------------------------------+ | > : | | Template Set | | > : | | Template Record | | > : | | . . . | | > : | +-------------------------------+ | > : +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+ > : | IPFIX Message | > : | +---------------------------------+ | > : | | Data Set | | > : | | Data Record | | > IPFIX File | | . . . | | > : | +-------------------------------+ | > : | +-------------------------------+ | > : | | Data Set | | > : | | Data Record | | > : | | . . . | | > : | +-------------------------------+ | > : | . . . | > : +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+ > : . . . > V > > >> 4. Motivation > >> The variety of applications of flow data, and the variety of >> presently deployed storage approaches, would seem to indicate the > > "would seem to indicate" =3D> indicate > >> need for a standard approach to flow storage with applicability >> across the continuum of time scales over which flow data is stored. >> A storage format based around flat files would best address the >> variety of storage requirements. While much work has been done on >> structured storage via RDBMS, relational database systems are not a >> good basis for format standardization owing to the fact that their >> internal data structures are generally private to a single >> implementation and subject to change for internal reasons. Also, >> there are a wide variety of operations available on flat files, and >> external tools and standards can be leveraged to meet file-based = >> flow >> storage requirements. Further, flow data is often not very >> semantically complicated, and is managed in very high volume; >> therefore, an RDBMS-based flow storage system would not benefit = >> much >> from the advantages of relational database technology. > > I would remove the last sentence. There is no need to argue against = > the > usage of RDBMS. > >> Over the past decade XML markup has emerged as a new "universal" >> representation format for structured data. It is intended to be >> human-readable; indeed, that is one reason for its rapid adoption. >> However XML has limited usefulness for representing network flow >> data. Network flow data has a simple, repetitive, non-hierarchical >> structure that does not benefit much from XML. An XML = >> representation >> of flow data would be an essentially flat list of the attributes = >> and >> their values for each flow record. >> >> The XML approach to data encoding is very heavyweight when compared >> to binary flow encoding. XML's use of start- and end-tags, and >> plain-text encoding of the actual values, leads to significant >> inefficiency in encoding size. Typical network flow datasets can >> contain millions or billions of flows per hour of traffic >> represented. Any increase in storage size per record can have >> dramatic impact on flow data storage and transfer sizes. While = >> data >> compression algorithms can partially remove the redundancy = >> introduced >> by XML encoding, they introduce additional overhead of their own. >> >> A further problem is that XML processing tools require a full XML >> parser. XML parsers are fully general and therefore complex, >> resource-intensive and relatively slow, introducing significant >> processing time overhead for large network-flow datasets. In >> contrast, parsers for typical binary flow data encodings are simply >> structured, since they only need to parse a very small header and >> then have complete knowledge of all following fields for the >> particular flow. These can then be read in a very efficient linear >> fashion. > > I suggest shortening the text of the three paragraphs above. > The argument that parsing binary IPFIX Messages is simpler than = > parsing > XML-encoded flow data is not convincing. > A better argument is that parsers for IPFIX Messages exist anyway and > can be reused. > >> This l------+ | > : | | Data Set | | > : | | Data Record | | > IPFIX File | | . . . | | > : | +-------------------------------+ | > : | +-------------------------------+ | > : | | Data Set | | > : | | Data Record | | > : | | . . . | | > : | +-------------------------------+ | > : | . . . | > : +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+ > : . . . > V > > >> 4. Motivation > >> The variety of applications of flow data, and the variety of >> presently deployed storage approaches, would seem to indicate the > > "would seem to indicate" =3D> indicate > >> need for a standard approach to flow storage with applicability >> across the continuum of time scales over which flow data is stored. >> A storage format based around flat files would best address the >> variety of storage requirements. While much work has been done on >> structured storage via RDBMS, relational database systems are not a >> good basis for format standardization owing to the fact that their >> internal data structures are generally private to a single >> implementation and subject to change for internal reasons. Also, >> there are a wide variety of operations available on flat files, and >> external tools and standards can be leveraged to meet file-based = >> flow >> storage requirements. Further, flow data is often not very >> semantically complicated, and is managed in very high volume; >> therefore, an RDBMS-based flow storage system would not benefit = >> much >> from the advantages of relational database technology. > > I would remove the last sentence. There is no need to argue against = > the > usage of RDBMS. > >> Over the past decade XML markup has emerged as a new "universal" >> representation format for structured data. It is intended to be >> human-readable; indeed, that is one reason for its rapid adoption. >> However XML has limited usefulness for representing network flow >> data. Network flow data has a simple, repetitive, non-hierarchical >> structure that does not benefit much from XML. An XML = >> representation >> of flow data would be an essentially flat list of the attributes = >> and >> their values for each flow record. >> >> The XML approach to data encoding is very heavyweight when compared >> to binary flow encoding. XML's use of start- and end-tags, and >> plain-text encoding of the actual values, leads to significant >> inefficiency in encoding size. Typical network flow datasets can >> contain millions or billions of flows per hour of traffic >> represented. Any increase in storage size per record can have >> dramatic impact on flow data storage and transfer sizes. While = >> data >> compression algorithms can partially remove the redundancy = >> introduced >> by XML encoding, they introduce additional overhead of their own. >> >> A further problem is that XML processing tools require a full XML >> parser. XML parsers are fully general and therefore complex, >> resource-intensive and relatively slow, introducing significant >> processing time overhead for large network-flow datasets. In >> contrast, parsers for typical binary flow data encodings are simply >> structured, since they only need to parse a very small header and >> then have complete knowledge of all following fields for the >> particular flow. These can then be read in a very efficient linear >> fashion. > > I suggest shortening the text of the three paragraphs above. > The argument that parsing binary IPFIX Messages is simpler than = > parsing > XML-encoded flow data is not convincing. > A better argument is that parsers for IPFIX Messages exist anyway and > can be reused. > >> This leaeads us to propose the IPFIX Message format as the basis = >> for a >> new flow data file format. The IPFIX working group, in defining = >> the >> IPFIX protocol, has already defined an information model and data >> formatting rules for representation of flow data. Especially at >> shorter time scales, when a file is a unit of data interchange, the >> filesystem may be viewed as simply another IPFIX Message transport >> between processes. This format is especially well suited to >> representing flow data, as it was designed specifically for flow = >> data >> export; it is easily extensible unlike ad-hoc serialization, and >> compact unlike XML. In addition, IPFIX is an IETF standard for the >> export and collection of flow data; using a common format for = >> storage >> and analysis at the collection side allows implementors to use >> substantially the same information model and data formatting >> implementation for transport as well as storage. > >> 5.2. Self Description > >> The IPFIX Message format is partially self-describing; that is, = >> IPFIX >> Templates containing only IANA-assigned Information Elements can be >> completely interpreted according to the IPFIX Information Model >> without additional external data. > > I do not think that the IPFIX Message format is self-describing = > because > a single IPFIX Message usually is not self-describing. You need all > messages sequence of a Transport Session to get all the required > Templates etc. > >> However, Templates containing private information elements lack >> detailed type and semantic information; a Collecting Process >> receiving data described by a template containing private = >> Information >> Elements it does not understand can only treat the data contained >> within those Information Elements as octet arrays. To be fully = >> self- >> describing, enterprise-specific Information Elements must be > > Unless this was a MUST, I would replace it by "can". > >> additionally described via IPFIX Options according to the = >> Information >> Element Type Options Template defined in "Exporting Type = >> Information >> for IPFIX Information Elements" [I-D.ietf-ipfix-exporting-type]. > >> 5.4. Indexing and Searching > >> The IPFIX Message format has no direct support for indexing. > > Do you want so search within an IPFIX Message or within an IPFIX = > File (=3D > sequence of IPFIX Messages)? > >> 5.5. Data Integrity > >> Note that more advanced error correction is almost certainly best > > omit "almost certainly" > >> handled at a layer below that addressed by this document. Error >> correction is a topic well addressed by the storage industry in >> general (e.g. by RAID and other technologies), and by specifying a >> flow storage format based upon files, we can leverage these = >> features >> to meet this requirement. > > >> 6.2. Storage of NetFlow V9-collected Flow Data >> >> Although the IPFIX protocol is based on the Cisco Netflow Services, >> Version 9 (NetFlow V9) protocol [RFC3954], the two have diverged > > I would not say that IPFIX is based on NetFlow V9. It's rather the > successor. > >> since work began on IPFIX. However, since the NetFlow V9 = >> information >> model is a compatible subset of the IPFIX information model, it is >> possible to use IPFIX files to store collected NetFlow V9 flow = >> data. >> This approach may be particularly useful in multi-vendor, multi- >> protocol collection infrastructures using both NetFlow V9 and IPFIX >> to export flow data. > > >> 7. Detailed Description >> >> An IPFIX File, as introduced in Section 3 and elaborated below, = >> is at >> its core simply an IPFIX Message stream serialized to some >> filesystem. Any valid serialized IPFIX Message stream MUST be >> accepted by a File Reader as a valid IPFIX file. In this way, the > > IPFIX File > >> filesystem is simply treated as another IPFIX transport alongside >> SCTP, TCP, and UDP. In contrast to normal IPFIX operation, the ds us to propose the IPFIX Message format as the basis = >> for a >> new flow data file format. The IPFIX working group, in defining = >> the >> IPFIX protocol, has already defined an information model and data >> formatting rules for representation of flow data. Especially at >> shorter time scales, when a file is a unit of data interchange, the >> filesystem may be viewed as simply another IPFIX Message transport >> between processes. This format is especially well suited to >> representing flow data, as it was designed specifically for flow = >> data >> export; it is easily extensible unlike ad-hoc serialization, and >> compact unlike XML. In addition, IPFIX is an IETF standard for the >> export and collection of flow data; using a common format for = >> storage >> and analysis at the collection side allows implementors to use >> substantially the same information model and data formatting >> implementation for transport as well as storage. > >> 5.2. Self Description > >> The IPFIX Message format is partially self-describing; that is, = >> IPFIX >> Templates containing only IANA-assigned Information Elements can be >> completely interpreted according to the IPFIX Information Model >> without additional external data. > > I do not think that the IPFIX Message format is self-describing = > because > a single IPFIX Message usually is not self-describing. You need all > messages sequence of a Transport Session to get all the required > Templates etc. > >> However, Templates containing private information elements lack >> detailed type and semantic information; a Collecting Process >> receiving data described by a template containing private = >> Information >> Elements it does not understand can only treat the data contained >> within those Information Elements as octet arrays. To be fully = >> self- >> describing, enterprise-specific Information Elements must be > > Unless this was a MUST, I would replace it by "can". > >> additionally described via IPFIX Options according to the = >> Information >> Element Type Options Template defined in "Exporting Type = >> Information >> for IPFIX Information Elements" [I-D.ietf-ipfix-exporting-type]. > >> 5.4. Indexing and Searching > >> The IPFIX Message format has no direct support for indexing. > > Do you want so search within an IPFIX Message or within an IPFIX = > File (=3D > sequence of IPFIX Messages)? > >> 5.5. Data Integrity > >> Note that more advanced error correction is almost certainly best > > omit "almost certainly" > >> handled at a layer below that addressed by this document. Error >> correction is a topic well addressed by the storage industry in >> general (e.g. by RAID and other technologies), and by specifying a >> flow storage format based upon files, we can leverage these = >> features >> to meet this requirement. > > >> 6.2. Storage of NetFlow V9-collected Flow Data >> >> Although the IPFIX protocol is based on the Cisco Netflow Services, >> Version 9 (NetFlow V9) protocol [RFC3954], the two have diverged > > I would not say that IPFIX is based on NetFlow V9. It's rather the > successor. > >> since work began on IPFIX. However, since the NetFlow V9 = >> information >> model is a compatible subset of the IPFIX information model, it is >> possible to use IPFIX files to store collected NetFlow V9 flow = >> data. >> This approach may be particularly useful in multi-vendor, multi- >> protocol collection infrastructures using both NetFlow V9 and IPFIX >> to export flow data. > > >> 7. Detailed Description >> >> An IPFIX File, as introduced in Section 3 and elaborated below, = >> is at >> its core simply an IPFIX Message stream serialized to some >> filesystem. Any valid serialized IPFIX Message stream MUST be >> accepted by a File Reader as a valid IPFIX file. In this way, the > > IPFIX File > >> filesystem is simply treated as another IPFIX transport alongside >> SCTP, TCP, and UDP. In contrast to normal IPFIX operation, the = = >> time >> between a File Writer writing an IPFIX Message stream to a File = >> and a >> File Reader reading it can be extremely variable. In other words, >> this notional file transport has unusually high latency, as the = >> File >> Reader and File Writer do not necessarily run at the same time. >> >> This section specifies the detailed actions of File Readers and = >> File >> Writers in handling IPFIX Files, and further specifies actions of >> File Writers in specific use cases. Unless otherwise specified >> herein, where appropriate IPFIX File Writers MUST behave as IPFIX > > remove "where"? > >> Exporting Processes, and IPFIX File Readers MUST behave as IPFIX >> Collecting Processes. > > >> 7.2. File Writer Specification >> >> While any valid serialized IPFIX Message stream is a valid IPFIX >> File, the following recommendations will improve representation >> simplicity and read performance in the general case, where = >> possible. >> >> File Writers SHOULD emit each Template Set or Options Template = >> Set to >> appear in the file before any Data Set described by the Templates >> within that Set, to ensure the File Reader can decode every Data = >> Set >> without waiting to process subsequent Templates or Options = >> Templates. > > Is the File Reader allowed to wait for a Template anyway? > In 7.1, you wrote that Templates can be redefined without withdrawal = > as > it is the case in UDP sessions. In this case, decoding errors may = > occur > if the Collector decodes previous Data Sets with the new Template. > >> File Writers SHOULD emit Data Records described by Options = >> Templates >> to appear in the file before any Data Records which depend on the >> scopes defined by those options. >> >> File Writers SHOULD use Template Withdrawals to withdraw = >> Templates if >> template IDs need to be reused. In this case, the new Templates >> reusing those IDs SHOULD appear directly in the file after the >> Template Withdrawals making the IDs available for reuse. Template >> Withdrawals SHOULD NOT be used unless necessary to reuse template > > Template > >> IDs. > > I do not understand the restrictions "SHOULD appear directly after" = > and > "SHOULD NOT be used unless necessary". If there is no good reason, I > would remove these restrictions. > >> Each IPFIX File is generally synonymous with a single Transport >> Session. File Writers SHOULD store the Templates and Options >> required to decode the data within the File in the File itself, and >> File Readers SHOULD NOT use Templates or Options defined in one = >> file >> to decode or interpret Data Sets in another. > > MUST the File Writer repeat Templates and Options in each File if the > Transport Session is split into several files? > >> >> File Writers SHOULD write IPFIX Messages within an IPFIX File in >> ascending Export Time order. >> >> File Writers MAY write Data Records to an IPFIX File in any order. >> However, File Writers that write flow records to an IPFIX File in >> flowStartTime or flowEndTime order SHOULD be consistent in this >> ordering within each File. > > The last sentence does not make sense to me since it is always = > true: If > the IPFIX File Writer writes record ordered by time, then, of course, > the order is consistent. > >> 7.3.1. Collocating a File Writer with a Collecting Process >> >> When collocating a File Writer with an IPFIX Collecting Process for >> archival storage of collected data in IPFIX Files as described in >> Section 6.1, the following recommendations may improve the = >> usefulness >> of the stored data. >> >> The simplest way for a to store the data collected in a single > > "for a" what? > >> Transport Session is to simply write the incoming IPFIX Messages to >> an IPFIX File as they are collected. However, the resulting files >> will lack information about the IPFIX Transport Session used to >> export them, such as the network addresses of the Exporting and >> Collecting Processes a >> time >> between a File Writer writing an IPFIX Message stream to a File = >> and a >> File Reader reading it can be extremely variable. In other words, >> this notional file transport has unusually high latency, as the = >> File >> Reader and File Writer do not necessarily run at the same time. >> >> This section specifies the detailed actions of File Readers and = >> File >> Writers in handling IPFIX Files, and further specifies actions of >> File Writers in specific use cases. Unless otherwise specified >> herein, where appropriate IPFIX File Writers MUST behave as IPFIX > > remove "where"? > >> Exporting Processes, and IPFIX File Readers MUST behave as IPFIX >> Collecting Processes. > > >> 7.2. File Writer Specification >> >> While any valid serialized IPFIX Message stream is a valid IPFIX >> File, the following recommendations will improve representation >> simplicity and read performance in the general case, where = >> possible. >> >> File Writers SHOULD emit each Template Set or Options Template = >> Set to >> appear in the file before any Data Set described by the Templates >> within that Set, to ensure the File Reader can decode every Data = >> Set >> without waiting to process subsequent Templates or Options = >> Templates. > > Is the File Reader allowed to wait for a Template anyway? > In 7.1, you wrote that Templates can be redefined without withdrawal = > as > it is the case in UDP sessions. In this case, decoding errors may = > occur > if the Collector decodes previous Data Sets with the new Template. > >> File Writers SHOULD emit Data Records described by Options = >> Templates >> to appear in the file before any Data Records which depend on the >> scopes defined by those options. >> >> File Writers SHOULD use Template Withdrawals to withdraw = >> Templates if >> template IDs need to be reused. In this case, the new Templates >> reusing those IDs SHOULD appear directly in the file after the >> Template Withdrawals making the IDs available for reuse. Template >> Withdrawals SHOULD NOT be used unless necessary to reuse template > > Template > >> IDs. > > I do not understand the restrictions "SHOULD appear directly after" = > and > "SHOULD NOT be used unless necessary". If there is no good reason, I > would remove these restrictions. > >> Each IPFIX File is generally synonymous with a single Transport >> Session. File Writers SHOULD store the Templates and Options >> required to decode the data within the File in the File itself, and >> File Readers SHOULD NOT use Templates or Options defined in one = >> file >> to decode or interpret Data Sets in another. > > MUST the File Writer repeat Templates and Options in each File if the > Transport Session is split into several files? > >> >> File Writers SHOULD write IPFIX Messages within an IPFIX File in >> ascending Export Time order. >> >> File Writers MAY write Data Records to an IPFIX File in any order. >> However, File Writers that write flow records to an IPFIX File in >> flowStartTime or flowEndTime order SHOULD be consistent in this >> ordering within each File. > > The last sentence does not make sense to me since it is always = > true: If > the IPFIX File Writer writes record ordered by time, then, of course, > the order is consistent. > >> 7.3.1. Collocating a File Writer with a Collecting Process >> >> When collocating a File Writer with an IPFIX Collecting Process for >> archival storage of collected data in IPFIX Files as described in >> Section 6.1, the following recommendations may improve the = >> usefulness >> of the stored data. >> >> The simplest way for a to store the data collected in a single > > "for a" what? > >> Transport Session is to simply write the incoming IPFIX Messages to >> an IPFIX File as they are collected. However, the resulting files >> will lack information about the IPFIX Transport Session used to >> export them, such as the network addresses of the Exporting and >> Collecting Processes andnd the protocols used to transport them. In >> this case, if information about the Transport Session is required, >> the File Writer SHOULD store a single IPFIX Transport Session in an >> IPFIX File and SHOULD record information about the Transport = >> Session >> using the Export Session Details Options Template described in >> Section 8.1.3. > >> 7.3.2. Collocating a File Writer with a Metering Process >> >> Note that File Writers may also be collocated directly with IPFIX > > Remove "note that" > >> Metering Processes, for writing measured information directly to = >> disk >> without intermediate IPFIX Exporting or Collecting Processes. This >> arrangement may be particularly useful when providing data to an >> analysis environment with an IPFIX File based workflow, or when >> testing Metering Processes during development. >> >> When collocating a File Writer with a Metering Process, note that > > Remove "note that" > >> Information Elements associated with Exporting or Collecting >> Processes are meaningless, and SHOULD NOT appear in the Export >> Session Details Options Template described in Section 8.1.3 or the >> Message Details Options Template described in Section 8.1.4. >> >> When collocating a File Writer with an Exporting Process, the = >> Export > > Collocating File Writer and Exporting Process? Do you mean Metering = > Process? > >> Time of each Message SHOULD be the time at which the first Data >> Record in the Message was received from the Exporting Process. > > Hm, I would rather say that the Export Time should be the time of > writing the Message into the File. This is typically the time of the > LAST Data Record in the Message. > When exported by an Exporting Process, the Export Time is never before > the time of the last Data Record either. > >> 7.3.3. Using IPFIX Files for Archival Storage >> >> While in the general case File Writers should store one Transport >> Session per IPFIX File, some applications storing large collections >> of data over long periods of time may benefit from the ability to >> treat a collection of IPFIX Files as a single Transport Session. A >> File Writer MAY be configurable to write data from a single = >> Transport >> Session into multiple IPFIX Files; however, File Writers supporting >> such a configuration option MUST provide a configuration option to >> support one-file-per-session behavior for interoperability = >> purposes. > > I do not think that a mandatory one-file-per-session option makes = > sense. > Sometimes, it is not possible to enforce one-file-per-session, for > example if the disk is full. What do you do then? > >> File Writers compressing or encrypting archival data and File = >> Readers >> reading compressed or encrypted archival data SHOULD follow the >> recommendations in Section 9. >> >> 7.3.4. Using IPFIX Files as Documents >> >> When IPFIX Files are used as documents, to store a set of flows >> relevant to query, investigation, or other common context, or for = >> the >> publication of flow data sets relevant to network research, each = >> File >> MUST be readable as a single Transport Session, self-contained and >> making no reference to metadata stored in separate Files, in = >> order to > > Maybe you should explain what you mean by metadata. > >> ensure interoperability. >> >> When writing Files to be used as documents, File Writers may emit = >> the >> special Data Records described by Options Templates before any = >> other >> Data Records in the File, in the following order, to ease the >> inspection and use of documents by File Readers: >> >> o Time Window records described by the File Time Window Options >> Template as defined in Section 8.1.2 below; followed by >> >> o commonPropertiesId definitions as described in "Reducing >> Redundancy in IPFIX and PSAMP Reports" >> [I-D.ietf-ipfix-reducing-redundancy]; followed by >> >> o Information Element Type Records as described in "Exporting Type >> Informati the protocols used to transport them. In >> this case, if information about the Transport Session is required, >> the File Writer SHOULD store a single IPFIX Transport Session in an >> IPFIX File and SHOULD record information about the Transport = >> Session >> using the Export Session Details Options Template described in >> Section 8.1.3. > >> 7.3.2. Collocating a File Writer with a Metering Process >> >> Note that File Writers may also be collocated directly with IPFIX > > Remove "note that" > >> Metering Processes, for writing measured information directly to = >> disk >> without intermediate IPFIX Exporting or Collecting Processes. This >> arrangement may be particularly useful when providing data to an >> analysis environment with an IPFIX File based workflow, or when >> testing Metering Processes during development. >> >> When collocating a File Writer with a Metering Process, note that > > Remove "note that" > >> Information Elements associated with Exporting or Collecting >> Processes are meaningless, and SHOULD NOT appear in the Export >> Session Details Options Template described in Section 8.1.3 or the >> Message Details Options Template described in Section 8.1.4. >> >> When collocating a File Writer with an Exporting Process, the = >> Export > > Collocating File Writer and Exporting Process? Do you mean Metering = > Process? > >> Time of each Message SHOULD be the time at which the first Data >> Record in the Message was received from the Exporting Process. > > Hm, I would rather say that the Export Time should be the time of > writing the Message into the File. This is typically the time of the > LAST Data Record in the Message. > When exported by an Exporting Process, the Export Time is never before > the time of the last Data Record either. > >> 7.3.3. Using IPFIX Files for Archival Storage >> >> While in the general case File Writers should store one Transport >> Session per IPFIX File, some applications storing large collections >> of data over long periods of time may benefit from the ability to >> treat a collection of IPFIX Files as a single Transport Session. A >> File Writer MAY be configurable to write data from a single = >> Transport >> Session into multiple IPFIX Files; however, File Writers supporting >> such a configuration option MUST provide a configuration option to >> support one-file-per-session behavior for interoperability = >> purposes. > > I do not think that a mandatory one-file-per-session option makes = > sense. > Sometimes, it is not possible to enforce one-file-per-session, for > example if the disk is full. What do you do then? > >> File Writers compressing or encrypting archival data and File = >> Readers >> reading compressed or encrypted archival data SHOULD follow the >> recommendations in Section 9. >> >> 7.3.4. Using IPFIX Files as Documents >> >> When IPFIX Files are used as documents, to store a set of flows >> relevant to query, investigation, or other common context, or for = >> the >> publication of flow data sets relevant to network research, each = >> File >> MUST be readable as a single Transport Session, self-contained and >> making no reference to metadata stored in separate Files, in = >> order to > > Maybe you should explain what you mean by metadata. > >> ensure interoperability. >> >> When writing Files to be used as documents, File Writers may emit = >> the >> special Data Records described by Options Templates before any = >> other >> Data Records in the File, in the following order, to ease the >> inspection and use of documents by File Readers: >> >> o Time Window records described by the File Time Window Options >> Template as defined in Section 8.1.2 below; followed by >> >> o commonPropertiesId definitions as described in "Reducing >> Redundancy in IPFIX and PSAMP Reports" >> [I-D.ietf-ipfix-reducing-redundancy]; followed by >> >> o Information Element Type Records as described in "Exporting Type >> Informationon for IPFIX Information Elements" >> [I-D.ietf-ipfix-exporting-type]; followed by > > Exporting type information should be before reducing redundancy since > common properties may contain enterprise specific IEs. > >> o Export Session details records described by the Export Session >> Details Options Template as defined in Section 8.1.3 below. > > Do we really need to specify the order? > I think it suffices to specify that the Time Window should appear at = > the > beginning of the file. > >> The Export Time of each Message within a File used as a document >> SHOULD be the time at which the Message was written by the File >> Writer. > > Agreed, but this contradicts what is written in 7.3.2. > >> If an IPFIX File used as a document uses the technique described in >> "Reducing Redundancy in IPFIX and PSAMP Reports" >> [I-D.ietf-ipfix-reducing-redundancy] AND all of the non-Options >> Templates in the File contain the commonPropertiesId Information >> Element, a File Reader MAY assume the set of commonPropertiesId >> definitions provides a complete table of contents for the File for >> searching purposes. > > There is a problem: Templates may be redefined in the middle of the > file. How does the File Reader now that all Templates ever defined in > the middle of the File use the same set of common properties? > >> 7.3.5. Using IPFIX Files for Testing >> >> IPFIX Files can be used for testing IPFIX Collecting Processes in = >> two >> ways. First, IPFIX Files can be used to store specific flow data = >> for >> regression and stress testing of Collectors; there are no special >> considerations for IPFIX Files used in this way. >> >> Second, IPFIX Files are useful for storing reference messages which >> do not comply to the IPFIX Protocol in order to test the error >> handling and recovery behavior of Collectors. Of course, IPFIX = >> Files >> intended to be used in this application necessarily MAY violate any >> of of the specifications in this document or in [RFC5101], and such > > duplicated "of" > >> files MUST NOT be transmitted to Collecting Processes or given as >> input File Readers not under test. >> >> Note that an extremely simple IPFIX Exporting Process may be = >> crafted >> for testing purposes by simply reading an IPFIX File and = >> transmitting >> it directly to a Collecting Process. Similarly, an extremely = >> simple >> Collecting Process may be crafted for testing purposes by simply >> accepting connections and/or IPFIX Messages from Exporting = >> Processes >> and writing the session's message stream to an IPFIX File. > > What is the reason for testing such a simple Collecting Process? It = > will > write the invalid messages into a file. > >> 8.1. Recommended Options Templates for IPFIX Files >> >> The following Options Templates allow IPFIX Message streams to meet >> the requirements outlined above without extension to the message >> format or protocol. They are defined in terms of existing >> Information Elements defined in [RFC5102], the Information Elements >> defined in "Exporting Type Information for IPFIX Information >> Elements" [I-D.ietf-ipfix-exporting-type], as well as Information >> Elements defined in Section 8.2. IPFIX File Readers and Writers >> SHOULD support these options templates as defined below. > > Options Templates > >> In addition, IPFIX File Readers and Writers SHOULD support the >> Options Templates defined in "Exporting Type Information for IPFIX >> Information Elements" [I-D.ietf-ipfix-exporting-type] in order to >> support self-description of enterprise-specific Information = >> Elements. > > I understand that a File Writer should add exporting type = > information if > necessary and available. So, a writer should support this feature. > But what about the File Reader? I do not think that supporting the > reading of exporting type information options requires special > capabilities that need to be supported. > >> 8.1.1. Message Checksum Options Template >> for IPFIX Information Elements" >> [I-D.ietf-ipfix-exporting-type]; followed by > > Exporting type information should be before reducing redundancy since > common properties may contain enterprise specific IEs. > >> o Export Session details records described by the Export Session >> Details Options Template as defined in Section 8.1.3 below. > > Do we really need to specify the order? > I think it suffices to specify that the Time Window should appear at = > the > beginning of the file. > >> The Export Time of each Message within a File used as a document >> SHOULD be the time at which the Message was written by the File >> Writer. > > Agreed, but this contradicts what is written in 7.3.2. > >> If an IPFIX File used as a document uses the technique described in >> "Reducing Redundancy in IPFIX and PSAMP Reports" >> [I-D.ietf-ipfix-reducing-redundancy] AND all of the non-Options >> Templates in the File contain the commonPropertiesId Information >> Element, a File Reader MAY assume the set of commonPropertiesId >> definitions provides a complete table of contents for the File for >> searching purposes. > > There is a problem: Templates may be redefined in the middle of the > file. How does the File Reader now that all Templates ever defined in > the middle of the File use the same set of common properties? > >> 7.3.5. Using IPFIX Files for Testing >> >> IPFIX Files can be used for testing IPFIX Collecting Processes in = >> two >> ways. First, IPFIX Files can be used to store specific flow data = >> for >> regression and stress testing of Collectors; there are no special >> considerations for IPFIX Files used in this way. >> >> Second, IPFIX Files are useful for storing reference messages which >> do not comply to the IPFIX Protocol in order to test the error >> handling and recovery behavior of Collectors. Of course, IPFIX = >> Files >> intended to be used in this application necessarily MAY violate any >> of of the specifications in this document or in [RFC5101], and such > > duplicated "of" > >> files MUST NOT be transmitted to Collecting Processes or given as >> input File Readers not under test. >> >> Note that an extremely simple IPFIX Exporting Process may be = >> crafted >> for testing purposes by simply reading an IPFIX File and = >> transmitting >> it directly to a Collecting Process. Similarly, an extremely = >> simple >> Collecting Process may be crafted for testing purposes by simply >> accepting connections and/or IPFIX Messages from Exporting = >> Processes >> and writing the session's message stream to an IPFIX File. > > What is the reason for testing such a simple Collecting Process? It = > will > write the invalid messages into a file. > >> 8.1. Recommended Options Templates for IPFIX Files >> >> The following Options Templates allow IPFIX Message streams to meet >> the requirements outlined above without extension to the message >> format or protocol. They are defined in terms of existing >> Information Elements defined in [RFC5102], the Information Elements >> defined in "Exporting Type Information for IPFIX Information >> Elements" [I-D.ietf-ipfix-exporting-type], as well as Information >> Elements defined in Section 8.2. IPFIX File Readers and Writers >> SHOULD support these options templates as defined below. > > Options Templates > >> In addition, IPFIX File Readers and Writers SHOULD support the >> Options Templates defined in "Exporting Type Information for IPFIX >> Information Elements" [I-D.ietf-ipfix-exporting-type] in order to >> support self-description of enterprise-specific Information = >> Elements. > > I understand that a File Writer should add exporting type = > information if > necessary and available. So, a writer should support this feature. > But what about the File Reader? I do not think that supporting the > reading of exporting type information options requires special > capabilities that need to be supported. > >> 8.1.1. Message Checksum Options Template >> >>>> The Message Checksum Options Template specifies the structure of a >> Data Record for attaching an MD5 message checksum to an IPFIX >> Message. An MD5 message checksum as described MAY be used if long- >> term data integrity is important to the application. The described > > why only "long-term" data integrity? > >> Data Record MUST appear only once per IPFIX Message, but MAY appear >> anywhere within the Message. > >> 8.1.2. File Time Window Options Template >> >> The File Time Window Options Template specifies the structure of a >> Data Record for attaching a time window to an IPFIX File; this Data >> Record is referred to as a time window record. A time window = >> record >> defines the earliest flow start time and the latest flow end time = >> of >> the flow records within a File. One and only one time window = >> record >> MAY appear within an IPFIX File if the time window information is >> available; a File Writer MUST NOT write more than one time window >> record to an IPFIX File. A File Writer that writes a time window >> record to a File MUST NOT write any Flow with a start time before = >> the >> beginning of the window or an end time after the end of the = >> window to >> that File. > > Shouldn't the time window record be located at the beginning of the > file? Yet this requires that we are able to prepend data to the file. > Anyway, a SHOULD would be ok, in my opinion. > >> The template SHOULD contain the following Information Elements: >> >> +--------------------- = >> +---------------------------------------------+ >> | IE | = >> Description | >> +--------------------- = >> +---------------------------------------------+ >> | sessionScope | A marker denoting this Option applies = >> to | >> | [scope] | the whole IPFIX Transport Session = >> (i.e., | >> | | IPFIX File); content is ignored. = >> This | >> | | Information Element MUST be defined as = >> a | >> | | Scope = >> Field. | >> | minFlowStartSeconds | The start time of the earliest flow in = >> the | >> | | Transport Session (i.e., File) in = >> epoch | >> | | = >> seconds. | >> | maxFlowEndSeconds | The end time of the latest flow in = >> the | >> | | Transport Session (i.e., File) in = >> epoch | >> | | = >> seconds. | >> +--------------------- = >> +---------------------------------------------+ > > sessionScope is ambiguous if the session is split into multiple = > files. I > assume that in this case, the time window record describes the = > limits of > a single file? > > Is there a reason for having seconds and not milliseconds as = > alternative? > > What can we do if the flow start or end timestamp is missing in the = > records? > > In the case of PSAMP, we have observationTimeSeconds etc. and it makes > sense to use this timestamp instead. > >> 8.1.3. Export Session Details Options Template >> >> The Export Session Details Options Template specifies the structure >> of a Data Record for recording the details of an IPFIX Transport >> Session in an IPFIX File. It is intended for use in storing a = >> single >> complete IPFIX Transport Session in a single IPFIX File. The > > Why not use it when a session is stored in multiple files as well? > >> described Data Record SHOULD appear only once in a given IPFIX = >> File. >> > >> 8.2. Recommended Information Elements for IPFIX Files >> >> The following Information Elements are used by the options = >> templates >> in Section 8.1 to allow IPFIX Message streams to meet the >> requirements outlined above without extension of the message format >> or protocol. IPFIX File Readers and Writers SHOULD support these >> Informat The Message Checksum Options Template specifies the structure of a >> Data Record for attaching an MD5 message checksum to an IPFIX >> Message. An MD5 message checksum as described MAY be used if long- >> term data integrity is important to the application. The described > > why only "long-term" data integrity? > >> Data Record MUST appear only once per IPFIX Message, but MAY appear >> anywhere within the Message. > >> 8.1.2. File Time Window Options Template >> >> The File Time Window Options Template specifies the structure of a >> Data Record for attaching a time window to an IPFIX File; this Data >> Record is referred to as a time window record. A time window = >> record >> defines the earliest flow start time and the latest flow end time = >> of >> the flow records within a File. One and only one time window = >> record >> MAY appear within an IPFIX File if the time window information is >> available; a File Writer MUST NOT write more than one time window >> record to an IPFIX File. A File Writer that writes a time window >> record to a File MUST NOT write any Flow with a start time before = >> the >> beginning of the window or an end time after the end of the = >> window to >> that File. > > Shouldn't the time window record be located at the beginning of the > file? Yet this requires that we are able to prepend data to the file. > Anyway, a SHOULD would be ok, in my opinion. > >> The template SHOULD contain the following Information Elements: >> >> +--------------------- = >> +---------------------------------------------+ >> | IE | = >> Description | >> +--------------------- = >> +---------------------------------------------+ >> | sessionScope | A marker denoting this Option applies = >> to | >> | [scope] | the whole IPFIX Transport Session = >> (i.e., | >> | | IPFIX File); content is ignored. = >> This | >> | | Information Element MUST be defined as = >> a | >> | | Scope = >> Field. | >> | minFlowStartSeconds | The start time of the earliest flow in = >> the | >> | | Transport Session (i.e., File) in = >> epoch | >> | | = >> seconds. | >> | maxFlowEndSeconds | The end time of the latest flow in = >> the | >> | | Transport Session (i.e., File) in = >> epoch | >> | | = >> seconds. | >> +--------------------- = >> +---------------------------------------------+ > > sessionScope is ambiguous if the session is split into multiple = > files. I > assume that in this case, the time window record describes the = > limits of > a single file? > > Is there a reason for having seconds and not milliseconds as = > alternative? > > What can we do if the flow start or end timestamp is missing in the = > records? > > In the case of PSAMP, we have observationTimeSeconds etc. and it makes > sense to use this timestamp instead. > >> 8.1.3. Export Session Details Options Template >> >> The Export Session Details Options Template specifies the structure >> of a Data Record for recording the details of an IPFIX Transport >> Session in an IPFIX File. It is intended for use in storing a = >> single >> complete IPFIX Transport Session in a single IPFIX File. The > > Why not use it when a session is stored in multiple files as well? > >> described Data Record SHOULD appear only once in a given IPFIX = >> File. >> > >> 8.2. Recommended Information Elements for IPFIX Files >> >> The following Information Elements are used by the options = >> templates >> in Section 8.1 to allow IPFIX Message streams to meet the >> requirements outlined above without extension of the message format >> or protocol. IPFIX File Readers and Writers SHOULD support these >> Informatioion Elements as defined below. >> >> In addition, IPFIX File Readers and Writers SHOULD support the >> Information Elements defined in "Exporting Type Information for = >> IPFIX >> Information Elements" [I-D.ietf-ipfix-exporting-type] in order to >> support full self-description of Information Elements. > > I would remove this paragraph. You have already stated above that the > writer should support exporting type information options. This implies > the support of the corresponding IEs. > For other purposes, the exporting type information IEs are not = > required. > >> 8.2.3. maxExportSeconds >> >> Description: The absolute Export Time of the latest IPFIX Message >> within the scope containing this Information Element. This >> Information Element SHOULD be bound to its containing IPFIX >> Transport Session (i.e., File) via an options record and the >> sessionScope Information Element, as defined below, and SHOULD >> appear only once in a given IPFIX File. > > session !=3D file in the case of multiple files per session > >> 10.1. Encapsulation of Non-IPFIX Data in IPFIX Files >> >> At times it may be useful to export or store non-IPFIX data = >> inline in >> an IPFIX File or Message stream. To do this cleanly, this data = >> must >> be encapsulated into IPFIX Messages so that an IPFIX File Reader or >> Collecting Process can handle it without any need to interpret it. >> At the same time, this data must not be changed during transmission >> or storage. The opaqueOctets Information Element as defined in >> Section 8.2.9 is provided for this encapsulation. > > I wonder if the standardization of opaqueOctets makes sense at all. > If a vendor wants to add such information, he can specify an > enterprise-specific IE for this purpose with optional type = > information. > With opaqueOctets, we never know which kind of data is contained in = > the > field. And we lose one important advantage of IE which is the implicit > semantics of any IE. > >> Processing the encapsulated non-IPFIX data is left to a separate >> processing mechanisms that can identify encapsulated non-IPFIX data >> in an IPFIX message stream, but need not have any other IPFIX >> handling capability, except the ability to skip over all IPFIX >> messages that do not encapsulate non-IPFIX data. >> >> The Message Checksum Options Template, described in Section 8.1.1 = >> may >> be used as a uniform mechanism to identify errors within = >> encapsulated >> data. >> >> Note that this mechanism can only encapsulate data objects up to >> 65,515 octets in length. If the space available in one IPFIX = >> Message >> is not enough for the amount of data to be encapsulated, then the >> data must be broken into smaller segments that are encapsulated = >> into >> consecutive IPFIX Messages. Any additional structuring or = >> semantics >> of the raw data is outside the scope of IPFIX and must be = >> implemented >> within the encapsulated binary data itself. Furthermore, the raw >> encapsulated data cannot be assumed by an IPFIX File Reader to have >> any specific format. >> >> 10.2. Encapsulation of IPFIX Files within Other File Formats >> >> Consequently, it may also be useful to reverse the encapsulation, >> that is, to export or store IPFIX data inline within a non-IPFIX = >> file >> or data stream. This makes sense when the other file format is not >> compatible with the encapsulation described above in Section 10.1. >> Generally speaking, the encapsulation here will be specific to the >> format of the containing file. For example, IPFIX files may be >> embedded in XML elements using hex or Base64 encoding, or in raw >> binary files using start and end delimiters or some form of run- >> length encoding. As there are as many potential encapsulations = >> here >> as there are potential file formats, the specifics of each are = >> out of >> scope for this specification. > > I would remove the whole section. It is completely out ofn Elements as defined below. >> >> In addition, IPFIX File Readers and Writers SHOULD support the >> Information Elements defined in "Exporting Type Information for = >> IPFIX >> Information Elements" [I-D.ietf-ipfix-exporting-type] in order to >> support full self-description of Information Elements. > > I would remove this paragraph. You have already stated above that the > writer should support exporting type information options. This implies > the support of the corresponding IEs. > For other purposes, the exporting type information IEs are not = > required. > >> 8.2.3. maxExportSeconds >> >> Description: The absolute Export Time of the latest IPFIX Message >> within the scope containing this Information Element. This >> Information Element SHOULD be bound to its containing IPFIX >> Transport Session (i.e., File) via an options record and the >> sessionScope Information Element, as defined below, and SHOULD >> appear only once in a given IPFIX File. > > session !=3D file in the case of multiple files per session > >> 10.1. Encapsulation of Non-IPFIX Data in IPFIX Files >> >> At times it may be useful to export or store non-IPFIX data = >> inline in >> an IPFIX File or Message stream. To do this cleanly, this data = >> must >> be encapsulated into IPFIX Messages so that an IPFIX File Reader or >> Collecting Process can handle it without any need to interpret it. >> At the same time, this data must not be changed during transmission >> or storage. The opaqueOctets Information Element as defined in >> Section 8.2.9 is provided for this encapsulation. > > I wonder if the standardization of opaqueOctets makes sense at all. > If a vendor wants to add such information, he can specify an > enterprise-specific IE for this purpose with optional type = > information. > With opaqueOctets, we never know which kind of data is contained in = > the > field. And we lose one important advantage of IE which is the implicit > semantics of any IE. > >> Processing the encapsulated non-IPFIX data is left to a separate >> processing mechanisms that can identify encapsulated non-IPFIX data >> in an IPFIX message stream, but need not have any other IPFIX >> handling capability, except the ability to skip over all IPFIX >> messages that do not encapsulate non-IPFIX data. >> >> The Message Checksum Options Template, described in Section 8.1.1 = >> may >> be used as a uniform mechanism to identify errors within = >> encapsulated >> data. >> >> Note that this mechanism can only encapsulate data objects up to >> 65,515 octets in length. If the space available in one IPFIX = >> Message >> is not enough for the amount of data to be encapsulated, then the >> data must be broken into smaller segments that are encapsulated = >> into >> consecutive IPFIX Messages. Any additional structuring or = >> semantics >> of the raw data is outside the scope of IPFIX and must be = >> implemented >> within the encapsulated binary data itself. Furthermore, the raw >> encapsulated data cannot be assumed by an IPFIX File Reader to have >> any specific format. >> >> 10.2. Encapsulation of IPFIX Files within Other File Formats >> >> Consequently, it may also be useful to reverse the encapsulation, >> that is, to export or store IPFIX data inline within a non-IPFIX = >> file >> or data stream. This makes sense when the other file format is not >> compatible with the encapsulation described above in Section 10.1. >> Generally speaking, the encapsulation here will be specific to the >> format of the containing file. For example, IPFIX files may be >> embedded in XML elements using hex or Base64 encoding, or in raw >> binary files using start and end delimiters or some form of run- >> length encoding. As there are as many potential encapsulations = >> here >> as there are potential file formats, the specifics of each are = >> out of >> scope for this specification. > > I would remove the whole section. It is completely out of s scope. > >> B.1.3. Template Format >> >> Template FlowSets in NetFlow V9 support a subset of functionality = >> of >> those in IPFIX. Specifically, NetFlow V9 does not have any support >> for vendor-specific Information Elements as IPFIX does, so there is >> no enterprise bit or facility for associating a private enterprise >> number with an information element. > > NetFlow V9 does not support variable length fields. > >> Options Template FlowSets in NetFlow V9 are similar to Options >> Template Sets in IPFIX in the same way. > >> B.1.4. Information Model >> >> The NetFlow V9 field type definitions are a compatible subset of, = >> and >> have evolved in concert with, the IPFIX Information Model. IPFIX >> Information Element numbers in the range 1-127 are defined by the > > RFC 5102 talks about IE IDs, not numbers. > >> IPFIX Information Model [RFC5102] to be compatible with the >> corresponding NetFlow V9 field types. > >> B.2. A Method for Transforming NetFlow V9 messages to IPFIX >> >> This appendix describes a method for transforming NetFlow V9 = >> Packets >> into IPFIX Messages, which can be used to store NetFlow V9 data in >> IPFIX Files. A process transforming NetFlow V9 Packets into IPFIX >> Messages must handle the fact that NetFlow V9 Packets and IPFIX >> Messages are framed differently, that sequence numbering works >> differently, and that the NetFlow V9 field type definitions are = >> only >> compatible with the IPFIX Information Model field and/or = >> information >> element numbers below Information Element number 128. > > what is an Information Model field? > > Consider: ...are only compatible with the IPFIX Information Element = > IDs > below 128. > >> For each incoming NetFlow V9 packet, the transformation process = >> must: >> >> 1. Verify that the Version field of the packet header is 9. >> >> 2. Verify that the Sequence Number field of the packet header is >> valid. >> >> 3. Scan the packet to: >> >> 1. verify that it contains no Templates with field numbers > > field types > >> outside the range 1-127; >> >> 2. verify that it contains no FlowSets with Set IDs between 2 >> and 255 inclusive; >> >> 3. verify that it contains the number of records in FlowSets, >> Template FlowSets, and Options Template FlowSets declared = >> in >> the Count field of the packet header; and >> >> 4. count the number of records in FlowSets for calculating the > > only records of Data FlowSets must be counted!!! > >> IPFIX Sequence number. > > Number > >> 4. Calculate a Sequence Number for each IPFIX Observation Domain = >> by >> storing the last Sequence Number sent for each Observation = >> Domain >> plus the count of records in FlowSets in the previous step to = >> be > > Data FlowSets! > >> sent as the Sequence Number for the IPFIX Message within that >> Observation Domain following this one. > > "following this one" is confusing. > I would move step 4 behind step 6 and write that the Sequence Number = > is > increased by the number of records in Data FlowSets contained in the > written Message. > >> 5. Generate a new IPFIX Message Header with: >> >> 1. a Version field of 10; >> >> 2. a Length field with the number of octets in the IPFIX >> Message, generally available by subtracting 4 from the = >> length >> of the NetFlow V9 packet as returned from the transport = >> layer >> (accounting for the difference in message header lengths); >> >> 3. the Sequence Number calculated for this message by the >> Sequence Number calculation step; and >> >> 4. Export Time and Observation Domain ID taken from the UNIX >> secs and Source ID fields of the NetFlow V9 packet header, >> respectively. >> >> 6. Copy each FlowSet from the Netflow V9 packet to the IPFIX = >> Message >> after the header. Replace Set ID 0 with Set ID 2 for Template cope. > >> B.1.3. Template Format >> >> Template FlowSets in NetFlow V9 support a subset of functionality = >> of >> those in IPFIX. Specifically, NetFlow V9 does not have any support >> for vendor-specific Information Elements as IPFIX does, so there is >> no enterprise bit or facility for associating a private enterprise >> number with an information element. > > NetFlow V9 does not support variable length fields. > >> Options Template FlowSets in NetFlow V9 are similar to Options >> Template Sets in IPFIX in the same way. > >> B.1.4. Information Model >> >> The NetFlow V9 field type definitions are a compatible subset of, = >> and >> have evolved in concert with, the IPFIX Information Model. IPFIX >> Information Element numbers in the range 1-127 are defined by the > > RFC 5102 talks about IE IDs, not numbers. > >> IPFIX Information Model [RFC5102] to be compatible with the >> corresponding NetFlow V9 field types. > >> B.2. A Method for Transforming NetFlow V9 messages to IPFIX >> >> This appendix describes a method for transforming NetFlow V9 = >> Packets >> into IPFIX Messages, which can be used to store NetFlow V9 data in >> IPFIX Files. A process transforming NetFlow V9 Packets into IPFIX >> Messages must handle the fact that NetFlow V9 Packets and IPFIX >> Messages are framed differently, that sequence numbering works >> differently, and that the NetFlow V9 field type definitions are = >> only >> compatible with the IPFIX Information Model field and/or = >> information >> element numbers below Information Element number 128. > > what is an Information Model field? > > Consider: ...are only compatible with the IPFIX Information Element = > IDs > below 128. > >> For each incoming NetFlow V9 packet, the transformation process = >> must: >> >> 1. Verify that the Version field of the packet header is 9. >> >> 2. Verify that the Sequence Number field of the packet header is >> valid. >> >> 3. Scan the packet to: >> >> 1. verify that it contains no Templates with field numbers > > field types > >> outside the range 1-127; >> >> 2. verify that it contains no FlowSets with Set IDs between 2 >> and 255 inclusive; >> >> 3. verify that it contains the number of records in FlowSets, >> Template FlowSets, and Options Template FlowSets declared = >> in >> the Count field of the packet header; and >> >> 4. count the number of records in FlowSets for calculating the > > only records of Data FlowSets must be counted!!! > >> IPFIX Sequence number. > > Number > >> 4. Calculate a Sequence Number for each IPFIX Observation Domain = >> by >> storing the last Sequence Number sent for each Observation = >> Domain >> plus the count of records in FlowSets in the previous step to = >> be > > Data FlowSets! > >> sent as the Sequence Number for the IPFIX Message within that >> Observation Domain following this one. > > "following this one" is confusing. > I would move step 4 behind step 6 and write that the Sequence Number = > is > increased by the number of records in Data FlowSets contained in the > written Message. > >> 5. Generate a new IPFIX Message Header with: >> >> 1. a Version field of 10; >> >> 2. a Length field with the number of octets in the IPFIX >> Message, generally available by subtracting 4 from the = >> length >> of the NetFlow V9 packet as returned from the transport = >> layer >> (accounting for the difference in message header lengths); >> >> 3. the Sequence Number calculated for this message by the >> Sequence Number calculation step; and >> >> 4. Export Time and Observation Domain ID taken from the UNIX >> secs and Source ID fields of the NetFlow V9 packet header, >> respectively. >> >> 6. Copy each FlowSet from the Netflow V9 packet to the IPFIX = >> Message >> after the header. Replace Set ID 0 with Set ID 2 for Template >>>> Sets, and Set ID 1 with Set ID 3 for Options Template Sets. >> >> Note that this process loses system uptime information; if such >> information is required, the transformation process will have to >> export that information using IPFIX Options. This may require a = >> more >> sophisticated transformation process structure. >> > > > > -- = > Dipl.-Ing. Gerhard M=FCnz > Computer Networks and Internet > Wilhelm Schickard Institute for Computer Science > University of Tuebingen > Sand 13 (Room B309), D-72076 Tuebingen, Germany > Phone: +49 7071 29-70534 Fax: +49 7071 29-5220 > E-mail: muenz@informatik.uni-tuebingen.de > WWW: http://net.informatik.uni-tuebingen.de/~muenz > > _______________________________________________ > IPFIX mailing list > IPFIX@ietf.org > https://www.ietf.org/mailman/listinfo/ipfix _______________________________________________ IPFIX mailing list IPFIX@ietf.org https://www.ietf.org/mailman/listinfo/ipfix Sets, and Set ID 1 with Set ID 3 for Options Template Sets. >> >> Note that this process loses system uptime information; if such >> information is required, the transformation process will have to >> export that information using IPFIX Options. This may require a = >> more >> sophisticated transformation process structure. >> > > > > -- = > Dipl.-Ing. Gerhard M=FCnz > Computer Networks and Internet > Wilhelm Schickard Institute for Computer Science > University of Tuebingen > Sand 13 (Room B309), D-72076 Tuebingen, Germany > Phone: +49 7071 29-70534 Fax: +49 7071 29-5220 > E-mail: muenz@informatik.uni-tuebingen.de > WWW: http://net.informatik.uni-tuebingen.de/~muenz > > _______________________________________________ > IPFIX mailing list > IPFIX@ietf.org > https://www.ietf.org/mailman/listinfo/ipfix _______________________________________________ IPFIX mailing list IPFIX@ietf.org https://www.ietf.org/mailman/listinfo/ipfix From ipfix-bounces@ietf.org Fri Aug 29 02:27:41 2008 Return-Path: X-Original-To: ipfix-archive@optimus.ietf.org Delivered-To: ietfarch-ipfix-archive@core3.amsl.com Received: from [127.0.0.1] (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 944883A67C1; Fri, 29 Aug 2008 02:27:41 -0700 (PDT) X-Original-To: ipfix@core3.amsl.com Delivered-To: ipfix@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 6D1573A6910 for ; Fri, 29 Aug 2008 02:27:40 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.639 X-Spam-Level: X-Spam-Status: No, score=-2.639 tagged_above=-999 required=5 tests=[AWL=-2.990, BAYES_50=0.001, HELO_EQ_DE=0.35] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id tLBxoapofxTg for ; Fri, 29 Aug 2008 02:27:39 -0700 (PDT) Received: from smtp.cs.uni-tuebingen.de (u-173-c156.cs.uni-tuebingen.de [134.2.173.156]) by core3.amsl.com (Postfix) with ESMTP id A680C3A68FF for ; Fri, 29 Aug 2008 02:27:38 -0700 (PDT) Received: from u-172-c138.cs.uni-tuebingen.de ([134.2.172.138]) by smtp.cs.uni-tuebingen.de with esmtpsa (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.69) (envelope-from ) id 1KZ0GV-0000lp-GQ for ipfix@ietf.org; Fri, 29 Aug 2008 11:27:35 +0200 Message-ID: <48B7C105.8070708@informatik.uni-tuebingen.de> Date: Fri, 29 Aug 2008 11:27:33 +0200 From: Gerhard Muenz User-Agent: Thunderbird 2.0.0.16 (Windows/20080708) MIME-Version: 1.0 To: "ipfix@ietf.org" References: In-Reply-To: Subject: Re: [IPFIX] WG last call on draft-ietf-ipfix-mib-04 X-BeenThere: ipfix@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: IPFIX WG discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============1091118548==" Sender: ipfix-bounces@ietf.org Errors-To: ipfix-bounces@ietf.org This is a cryptographically signed message in MIME format. --===============1091118548== Content-Type: multipart/signed; protocol="application/x-pkcs7-signature"; micalg=sha1; boundary="------------ms020408060505010004040409" This is a cryptographically signed message in MIME format. --------------ms020408060505010004040409 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Dear all, Please find comments on draft-ietf-ipfix-mib-04 below. Currently, we have ipfixTransportSessionTable which does not provide any information about the role of the given device within the transport session. We do not know if the device is exporter or collector. On the other hand, you define "directional" tables for the Templates: ipfixExportedTemplateTable ipfixExportedTemplateDefinitionTable ipfixCollectedTemplateTable ipfixCollectedTemplateDefinitionTable Is there any specific reason? There are two alternatives which require fewer tables: 1) One could add an entry about the role of the device to the ipfixTransportSessionTable, for example in form of a field with values "exporter" and "collector". As the Templates are scoped to a given Transport Session, it is clear which Templates are exported and which Templates are collected. So we would have only three tables instead of fi= ve: ipfixTransportSessionTable ipfixTemplateTable ipfixTemplateDefinitionTable 2) Or, one could split the ipfixTransportSessionTable into: - ipfixOutgoingTransportSessionTable for Transport Sessions where the given device is exportFrom ipfix-bounces@ietf.org Fri Aug 29 02:27:41 2008 Return-Path: X-Original-To: ipfix-archive@lists.ietf.org Delivered-To: ietfarch-ipfix-archive@core3.amsl.com Received: from [127.0.0.1] (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 944883A67C1; Fri, 29 Aug 2008 02:27:41 -0700 (PDT) X-Original-To: ipfix@core3.amsl.com Delivered-To: ipfix@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 6D1573A6910 for ; Fri, 29 Aug 2008 02:27:40 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.639 X-Spam-Level: X-Spam-Status: No, score=-2.639 tagged_above=-999 required=5 tests=[AWL=-2.990, BAYES_50=0.001, HELO_EQ_DE=0.35] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id tLBxoapofxTg for ; Fri, 29 Aug 2008 02:27:39 -0700 (PDT) Received: from smtp.cs.uni-tuebingen.de (u-173-c156.cs.uni-tuebingen.de [134.2.173.156]) by core3.amsl.com (Postfix) with ESMTP id A680C3A68FF for ; Fri, 29 Aug 2008 02:27:38 -0700 (PDT) Received: from u-172-c138.cs.uni-tuebingen.de ([134.2.172.138]) by smtp.cs.uni-tuebingen.de with esmtpsa (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.69) (envelope-from ) id 1KZ0GV-0000lp-GQ for ipfix@ietf.org; Fri, 29 Aug 2008 11:27:35 +0200 Message-ID: <48B7C105.8070708@informatik.uni-tuebingen.de> Date: Fri, 29 Aug 2008 11:27:33 +0200 From: Gerhard Muenz User-Agent: Thunderbird 2.0.0.16 (Windows/20080708) MIME-Version: 1.0 To: "ipfix@ietf.org" References: In-Reply-To: Subject: Re: [IPFIX] WG last call on draft-ietf-ipfix-mib-04 X-BeenThere: ipfix@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: IPFIX WG discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============1091118548==" Sender: ipfix-bounces@ietf.org Errors-To: ipfix-bounces@ietf.org This is a cryptographically signed message in MIME format. --===============1091118548== Content-Type: multipart/signed; protocol="application/x-pkcs7-signature"; micalg=sha1; boundary="------------ms020408060505010004040409" This is a cryptographically signed message in MIME format. --------------ms020408060505010004040409 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Dear all, Please find comments on draft-ietf-ipfix-mib-04 below. Currently, we have ipfixTransportSessionTable which does not provide any information about the role of the given device within the transport session. We do not know if the device is exporter or collector. On the other hand, you define "directional" tables for the Templates: ipfixExportedTemplateTable ipfixExportedTemplateDefinitionTable ipfixCollectedTemplateTable ipfixCollectedTemplateDefinitionTable Is there any specific reason? There are two alternatives which require fewer tables: 1) One could add an entry about the role of the device to the ipfixTransportSessionTable, for example in form of a field with values "exporter" and "collector". As the Templates are scoped to a given Transport Session, it is clear which Templates are exported and which Templates are collected. So we would have only three tables instead of fi= ve: ipfixTransportSessionTable ipfixTemplateTable ipfixTemplateDefinitionTable 2) Or, one could split the ipfixTransportSessionTable into: - ipfixOutgoingTransportSessionTable for Transport Sessions where the given device is exporterer. - ipfixIncomingTransportSessionTable for Transport Sessions where the given device is collector. Again, we only need two tables for the Templates: ipfixTemplateTable ipfixTemplateDefinitionTable Reducing the number of tables is certainly not a design goal for MIBs. However, I think that it is more logical to have the information of the device role (exporter and collector) together with the data about the Transport Session. > 5.3. The Template Tables >=20 > There are two Template tables, the Exported Template > (ipfixExportedTemplateTable) table and the Collected Template table > (ipfixCollectedTemplateTable). Those Template tables list all > Templates (including Option Templates) that are sent (by an Exporter= ) > or received (by a Collector). The (Option) Templates are unique per= > Transport Session and Observation Domain, thus the table is indexed > by the Transport Session Index (ipfixTransportSessionIndex) and the > Observation Domain Id (ipfixExportedObservationDomainId or > ipfixCollectedObservationDomainId). It contains the Set Id and an I propose to list the indices in a clearer way one below the other: The (Option) Templates are unique per Transport Session and Observation Domain, thus the table is indexed by the following indices: - Transport Session Index (ipfixTransportSessionIndex) - Observation Domain Id (ipfixExportedObservationDomainId or ipfixCollectedObservationDomainId). With two indices, explaining them in the text may still be ok. But you have much longer lists of indices for the other Tables. > Access Time denoting the time when the (Option) Template was last > sent or received. >=20 > To resume the above example the Exporter may want to export the a remove "the" or "a" > Template and an Option Template for each Transport Session defined > above. This leads to the following Template Table defining Template= > and Option Template: >=20 > ipfixExportedTemplateTable (3) > | > +- ipfixExportedTemplateEntry (1) > | > +- index (5) > | +- index (3) > | + index (257) It is difficult to deduce the meaning of the indices. Maybe add in brackets which index refers to what: +- index (5) (ipfixTransportSessionIndex) | +- index (3) (ipfixExportedObservationDomainId) | + index (257) (ipfixExportedTemplateId) > | | +- ipfixExportedObservationDomainId (1) =3D 3 > | | +- ipfixExportedTemplateId (2) =3D 257 > | | +- ipfixExportedTemplateSetId (3) =3D 2 > | | +- ipfixExportedTemplateAccessTime (4) > | | =3D 2008-7-1,12:49:11.2,+2:0= > | | > | + index (264) > | +- ipfixExportedObservationDomainId (1) =3D 3 > | +- ipfixExportedTemplateId (2) =3D 264 > | +- ipfixExportedTemplateSetId (3) =3D 3 > | +- ipfixExportedTemplateAccessTime (4) > . =3D 2008-7-1,12:47:04.8,+2:0= > . > . > . > +- index (11) > +- index (3) > + index (273) > | +- ipfixExportedObservationDomainId (1) =3D 3 > | +- ipfixExportedTemplateId (2) =3D 273 > | +- ipfixExportedTemplateSetId (3) =3D 2 > | +- ipfixExportedTemplateAccessTime (4) > | =3D 2008-7-1,12:49:11.2,+2:0= > | > + index (289) > +- ipfixExportedObservationDomainId (1) =3D 3 > +- ipfixExportedTemplateId (2) =3D 289 > +- ipfixExportedTemplateSetId (3) =3D 3 > +- ipfixExportedTemplateAccessTime (4) > =3D 2008-7-1,12:47:04.8,+2:0= >=20 > We assume that the Collector with index 5 in the Transport Session > table of the Exporter has stored the connection to the Exporter with= > the Transport Session index of 17 then its Template table would look= > as fo. - ipfixIncomingTransportSessionTable for Transport Sessions where the given device is collector. Again, we only need two tables for the Templates: ipfixTemplateTable ipfixTemplateDefinitionTable Reducing the number of tables is certainly not a design goal for MIBs. However, I think that it is more logical to have the information of the device role (exporter and collector) together with the data about the Transport Session. > 5.3. The Template Tables >=20 > There are two Template tables, the Exported Template > (ipfixExportedTemplateTable) table and the Collected Template table > (ipfixCollectedTemplateTable). Those Template tables list all > Templates (including Option Templates) that are sent (by an Exporter= ) > or received (by a Collector). The (Option) Templates are unique per= > Transport Session and Observation Domain, thus the table is indexed > by the Transport Session Index (ipfixTransportSessionIndex) and the > Observation Domain Id (ipfixExportedObservationDomainId or > ipfixCollectedObservationDomainId). It contains the Set Id and an I propose to list the indices in a clearer way one below the other: The (Option) Templates are unique per Transport Session and Observation Domain, thus the table is indexed by the following indices: - Transport Session Index (ipfixTransportSessionIndex) - Observation Domain Id (ipfixExportedObservationDomainId or ipfixCollectedObservationDomainId). With two indices, explaining them in the text may still be ok. But you have much longer lists of indices for the other Tables. > Access Time denoting the time when the (Option) Template was last > sent or received. >=20 > To resume the above example the Exporter may want to export the a remove "the" or "a" > Template and an Option Template for each Transport Session defined > above. This leads to the following Template Table defining Template= > and Option Template: >=20 > ipfixExportedTemplateTable (3) > | > +- ipfixExportedTemplateEntry (1) > | > +- index (5) > | +- index (3) > | + index (257) It is difficult to deduce the meaning of the indices. Maybe add in brackets which index refers to what: +- index (5) (ipfixTransportSessionIndex) | +- index (3) (ipfixExportedObservationDomainId) | + index (257) (ipfixExportedTemplateId) > | | +- ipfixExportedObservationDomainId (1) =3D 3 > | | +- ipfixExportedTemplateId (2) =3D 257 > | | +- ipfixExportedTemplateSetId (3) =3D 2 > | | +- ipfixExportedTemplateAccessTime (4) > | | =3D 2008-7-1,12:49:11.2,+2:0= > | | > | + index (264) > | +- ipfixExportedObservationDomainId (1) =3D 3 > | +- ipfixExportedTemplateId (2) =3D 264 > | +- ipfixExportedTemplateSetId (3) =3D 3 > | +- ipfixExportedTemplateAccessTime (4) > . =3D 2008-7-1,12:47:04.8,+2:0= > . > . > . > +- index (11) > +- index (3) > + index (273) > | +- ipfixExportedObservationDomainId (1) =3D 3 > | +- ipfixExportedTemplateId (2) =3D 273 > | +- ipfixExportedTemplateSetId (3) =3D 2 > | +- ipfixExportedTemplateAccessTime (4) > | =3D 2008-7-1,12:49:11.2,+2:0= > | > + index (289) > +- ipfixExportedObservationDomainId (1) =3D 3 > +- ipfixExportedTemplateId (2) =3D 289 > +- ipfixExportedTemplateSetId (3) =3D 3 > +- ipfixExportedTemplateAccessTime (4) > =3D 2008-7-1,12:47:04.8,+2:0= >=20 > We assume that the Collector with index 5 in the Transport Session > table of the Exporter has stored the connection to the Exporter with= > the Transport Session index of 17 then its Template table would look= > as follllows: The sentence is confusing. The Transport Session tables does not contain any entry called Collector, only ipfixTransportSessionDestinationAddress and ipfixTransportSessionDestinationPort. Maybe: We assume that the Transport Session that is stored with index 5 in the Transport Session table of the Exporter is stored with index 17 in the Transport Session table of the (corresponding) Collector. Then, the Template table would look as follows: > 5.4. The Template Definition Tables >=20 > As with the Template tables there are two Template Definition tables= , > the Exported Template Definition table > (ipfixExportedTemplateDefinitionTable) and the Collected Template > Definition table (ipfixCollectedTemplateDefinitionTable). Those > tables list all the Information Elements contained in a Template or > Option Template. Therefore it has the same indexes as the > corresponding Template table plus the Template Id. Its own index > denotes the order of the Information Element inside the Template if > necessary. Besides the Information Element Id and the length of the= > encoded value the table contains flags for each Information Element.= > The flags indicate if the Information Element is used for scoping or= > as a Flow key. What about enterprise-specific IEs? Don't we need an enterprise number entry? > 5.5. The Export Table >=20 > On Exporters, the Export table (ipfixExportTable) can be used to > support features like failover, load-balancing, duplicate export to > several Collectors etc. The table has 5 indexes that link an entry > with the Metering Process table (ipfixMeteringProcessCacheId, see > below), the Exported Template table (ipfixExportedObservationDomainI= d > and ipfixExportedTemplateId) and the Transport Session table > (ipfixTransportSessionIndex). Those entries with the same Is there a reason for not ordering the indexes as they appear in the MIB?= > ipfixExportIndex, the same ipfixMeteringProcessCacheId and the same > ipfixExportedObservationDomainId define a Transport Session group. > This also The member type for each group member describes its remove "This also" > functionality. Why is a Transport Session group restricted to the output of a single cac= he? > If the Exporter does not use Transport Session grouping then each > ipfixExportIndex contains a single ipfixMeteringProcessCacheId and ipfixTransportSessionIndex? > thus a singe Transport Session and this session MUST have the member= single > type primary(1). > To continue the example we assume that the Exporter uses the 2 two > connections shown in the examples above as the primary export for a > session protected by a secondary backup connection. The Exporter > then has the following entries in the ipfixExportTable: >=20 > ipfixExportTable (3) > | > +- ipfixExportEntry (1) > | > +- index (7) > | +- index (9) > | +- index (3) > | +- index (257) > | | +- index (5) > | | +- ipfixExportIndex (1) =3D 7 > | | +- ipfixExportMemberType (2) =3D 1 (primary) > | | > | +- index (273) > | +- index (11) > | +- ipfixExportIndex (1) =3D 7 > | +- ipfixExportMemberType (2) =3D 2 (secondary) > | > +- index (8) > +- index (9) > +- index (3) > +- index (264) > | +- index (5) > | +- ipfixExportIndex (1) =3D 8 > | +- ipfixExportMemberType (2) =3D 2 (secondary) > +- index (289) > +- index (11) > +- ipfixExportIndex (1) =3D 7 8? > +- ipfixExportMemberType (2) =3D 1 (primary) Can you explain this example? There are two Metering Processes? And the output is sent to different primary collectors? And in case of a failure, each collector serves as backup for the other? >ows: The sentence is confusing. The Transport Session tables does not contain any entry called Collector, only ipfixTransportSessionDestinationAddress and ipfixTransportSessionDestinationPort. Maybe: We assume that the Transport Session that is stored with index 5 in the Transport Session table of the Exporter is stored with index 17 in the Transport Session table of the (corresponding) Collector. Then, the Template table would look as follows: > 5.4. The Template Definition Tables >=20 > As with the Template tables there are two Template Definition tables= , > the Exported Template Definition table > (ipfixExportedTemplateDefinitionTable) and the Collected Template > Definition table (ipfixCollectedTemplateDefinitionTable). Those > tables list all the Information Elements contained in a Template or > Option Template. Therefore it has the same indexes as the > corresponding Template table plus the Template Id. Its own index > denotes the order of the Information Element inside the Template if > necessary. Besides the Information Element Id and the length of the= > encoded value the table contains flags for each Information Element.= > The flags indicate if the Information Element is used for scoping or= > as a Flow key. What about enterprise-specific IEs? Don't we need an enterprise number entry? > 5.5. The Export Table >=20 > On Exporters, the Export table (ipfixExportTable) can be used to > support features like failover, load-balancing, duplicate export to > several Collectors etc. The table has 5 indexes that link an entry > with the Metering Process table (ipfixMeteringProcessCacheId, see > below), the Exported Template table (ipfixExportedObservationDomainI= d > and ipfixExportedTemplateId) and the Transport Session table > (ipfixTransportSessionIndex). Those entries with the same Is there a reason for not ordering the indexes as they appear in the MIB?= > ipfixExportIndex, the same ipfixMeteringProcessCacheId and the same > ipfixExportedObservationDomainId define a Transport Session group. > This also The member type for each group member describes its remove "This also" > functionality. Why is a Transport Session group restricted to the output of a single cac= he? > If the Exporter does not use Transport Session grouping then each > ipfixExportIndex contains a single ipfixMeteringProcessCacheId and ipfixTransportSessionIndex? > thus a singe Transport Session and this session MUST have the member= single > type primary(1). > To continue the example we assume that the Exporter uses the 2 two > connections shown in the examples above as the primary export for a > session protected by a secondary backup connection. The Exporter > then has the following entries in the ipfixExportTable: >=20 > ipfixExportTable (3) > | > +- ipfixExportEntry (1) > | > +- index (7) > | +- index (9) > | +- index (3) > | +- index (257) > | | +- index (5) > | | +- ipfixExportIndex (1) =3D 7 > | | +- ipfixExportMemberType (2) =3D 1 (primary) > | | > | +- index (273) > | +- index (11) > | +- ipfixExportIndex (1) =3D 7 > | +- ipfixExportMemberType (2) =3D 2 (secondary) > | > +- index (8) > +- index (9) > +- index (3) > +- index (264) > | +- index (5) > | +- ipfixExportIndex (1) =3D 8 > | +- ipfixExportMemberType (2) =3D 2 (secondary) > +- index (289) > +- index (11) > +- ipfixExportIndex (1) =3D 7 8? > +- ipfixExportMemberType (2) =3D 1 (primary) Can you explain this example? There are two Metering Processes? And the output is sent to different primary collectors? And in case of a failure, each collector serves as backup for the other? > 5 5.6. The Metering Process Table >=20 > The Metering Process as defined in [RFC5101] consists of a set of > function. Maintaining the Flow Records is one of them. This > function is responsible for passing the Flow Records to the Exportin= g > Process but also for detecting Flow expiration. The Flow Records "but also" =3D> "and also" > that are maintained by the Metering Process can be grouped by the > Observation Points they are observed. The instance that maintains > such a group of Flow Records is a kind of cache. For this reason th= e > Metering Process table (ipfixMeteringProcessTable) is grouped by > cache IDs (ipfixMeteringProcessCacheId). Each cache can be > maintained by a separate instance of the Metering Process which is > represented by the Metering Process ID (ipfixMeteringProcessId). To= > specify the Observation Point(s) where the Flow Records are gathered= > the ipfixObservationPointGroupReference may contain the an "the" or "an"? > ipfixObservationPointGroupId from the Observation Point table > (ipfixObservationPointTable) described in the next section. If an > Observation Point cannot be given the What do you mean by "cannot be given"? > ipfixObservationPointGroupReference MUST be zero(0). The timeouts > (ipfixMeteringProcessCacheActiveTimeout and > ipfixMeteringProcessCacheInactiveTimeout) specify when Flow Records > are passed to the Exporting Process. >=20 > ipfixMeteringProcessTable(8) > | > +- ipfixMeteringProcessEntry(1) > | > +- index(9) > +- ipfixMeteringProcessCacheId(1) =3D 9 > +- ipfixMeteringProcessId(2) =3D 287 > +- ipfixObservationPointGroupReference(3) =3D 17 > +- ipfixMeteringProcessCacheActiveTimeout(4) =3D 100 > +- ipfixMeteringProcessCacheInactiveTimeout(5) =3D 100 Regards, Gerhard --=20 Dipl.-Ing. Gerhard M=FCnz Computer Networks and Internet Wilhelm Schickard Institute for Computer Science University of Tuebingen Sand 13 (Room B309), D-72076 Tuebingen, Germany Phone: +49 7071 29-70534 Fax: +49 7071 29-5220 E-mail: muenz@informatik.uni-tuebingen.de WWW: http://net.informatik.uni-tuebingen.de/~muenz --------------ms020408060505010004040409 Content-Type: application/x-pkcs7-signature; name="smime.p7s" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="smime.p7s" Content-Description: S/MIME Cryptographic Signature MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAQAAoIIJdTCC AxUwggJ+oAMCAQICED9aGsYWkMr+s4zmyODhB+IwDQYJKoZIhvcNAQEFBQAwYjELMAkGA1UE BhMCWkExJTAjBgNVBAoTHFRoYXd0ZSBDb25zdWx0aW5nIChQdHkpIEx0ZC4xLDAqBgNVBAMT I1RoYXd0ZSBQZXJzb25hbCBGcmVlbWFpbCBJc3N1aW5nIENBMB4XDTA4MDIxMzE1MTUxN1oX DTA5MDIxMjE1MTUxN1owbDEOMAwGA1UEBBMFTXVlbnoxEDAOBgNVBCoTB0dlcmhhcmQxFjAU BgNVBAMTDUdlcmhhcmQgTXVlbnoxMDAuBgkqhkiG9w0BCQEWIW11ZW56QGluZm9ybWF0aWsu dW5pLXR1ZWJpbmdlbi5kZTCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAMZex/Sq sAxkzTVvKP/YAgkaeXA+ngH59Aa0bbRPsKOWzAndGqty5EKcEzrnKqEJ27qHFvoF/pHp88U2 7SJI/xbqkgWeV2jRaldipZQYlnjYLQcmb4cewIFuGRRSVrm3BquzX38aYazuE4+DVH2Z3a8z n0FcdMXhA1NR2Ma1rh4G7SIeZ+hC7czbvNRPraBliGdQhs8J/6yP/iL8aNYAl9c7CL4ofRj8 Y9orMOV/4vtWTq76/VQUVdbhUMiv0D8aHqI1ZvGskhRRvmITgQRVbbn8N8WTpZ0UCgMDjxPP 9i5IhLfp6oBtsKl4OZ0RXvSLZrbJTkBX3vnEutcyxDvyNgMCAwEAAaM+MDwwLAYDVR0RBCUw I4EhbXVlbnpAaW5mb3JtYXRpay51bmktdHVlYmluZ2VuLmRlMAwGA1UdEwEB/wQCMAAwDQYJ KoZIhvcNAQEFBQADgYEAX5SiD6epJePwBjJumOsTF6wzeuZRDLYlN+fOpXwd2C0Yx6i8iIZ9 l/J/nGaE1YpJPfX5oJDE+tOk1vYh2E9ThLOj9kJ3buZmgOCdVu90qtCWhfhli7RCYcJ+G9M3 FCnqbrzI/waPPXGB8/DY1HKgPj5G+oKPUK+GD2aE1Q3PYGowggMVMIICfqADAgECAhA/WhrG FpDK/rOM5sjg4QfiMA0GCSqGSIb3DQEBBQUAMGIxCzAJBgNVBAYTAlpBMSUwIwYDVQQKExxU aGF3dGUgQ29uc3VsdGluZyAoUHR5KSBMdGQuMSwwKgYDVQQDEyNUaGF3dGUgUGVyc29uYWwg RnJlZW1haWwgSXNzdWluZyBDQTAeFw0wODAyMTMxNTE1MTdaFw0wOTAyMTIxNTE1MTdaMGwx DjAMBgNVBAQTBU11ZW56MRAwDgYDVQQqEwdHZXJoYXJkMRYwFAYDVQQDEw1HZXJoYXJkIE11 ZW56MTAwLgYJKoZIhvcNAQkBFiFtdWVuekBpbmZvcm1hdGlrLnVuaS10dWViaW5nZW4uZGUw ggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQDGXsf0qrAMZM01byj/2AIJGnlwPp4B +fQGtG.6. The Metering Process Table >=20 > The Metering Process as defined in [RFC5101] consists of a set of > function. Maintaining the Flow Records is one of them. This > function is responsible for passing the Flow Records to the Exportin= g > Process but also for detecting Flow expiration. The Flow Records "but also" =3D> "and also" > that are maintained by the Metering Process can be grouped by the > Observation Points they are observed. The instance that maintains > such a group of Flow Records is a kind of cache. For this reason th= e > Metering Process table (ipfixMeteringProcessTable) is grouped by > cache IDs (ipfixMeteringProcessCacheId). Each cache can be > maintained by a separate instance of the Metering Process which is > represented by the Metering Process ID (ipfixMeteringProcessId). To= > specify the Observation Point(s) where the Flow Records are gathered= > the ipfixObservationPointGroupReference may contain the an "the" or "an"? > ipfixObservationPointGroupId from the Observation Point table > (ipfixObservationPointTable) described in the next section. If an > Observation Point cannot be given the What do you mean by "cannot be given"? > ipfixObservationPointGroupReference MUST be zero(0). The timeouts > (ipfixMeteringProcessCacheActiveTimeout and > ipfixMeteringProcessCacheInactiveTimeout) specify when Flow Records > are passed to the Exporting Process. >=20 > ipfixMeteringProcessTable(8) > | > +- ipfixMeteringProcessEntry(1) > | > +- index(9) > +- ipfixMeteringProcessCacheId(1) =3D 9 > +- ipfixMeteringProcessId(2) =3D 287 > +- ipfixObservationPointGroupReference(3) =3D 17 > +- ipfixMeteringProcessCacheActiveTimeout(4) =3D 100 > +- ipfixMeteringProcessCacheInactiveTimeout(5) =3D 100 Regards, Gerhard --=20 Dipl.-Ing. Gerhard M=FCnz Computer Networks and Internet Wilhelm Schickard Institute for Computer Science University of Tuebingen Sand 13 (Room B309), D-72076 Tuebingen, Germany Phone: +49 7071 29-70534 Fax: +49 7071 29-5220 E-mail: muenz@informatik.uni-tuebingen.de WWW: http://net.informatik.uni-tuebingen.de/~muenz --------------ms020408060505010004040409 Content-Type: application/x-pkcs7-signature; name="smime.p7s" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="smime.p7s" Content-Description: S/MIME Cryptographic Signature MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAQAAoIIJdTCC AxUwggJ+oAMCAQICED9aGsYWkMr+s4zmyODhB+IwDQYJKoZIhvcNAQEFBQAwYjELMAkGA1UE BhMCWkExJTAjBgNVBAoTHFRoYXd0ZSBDb25zdWx0aW5nIChQdHkpIEx0ZC4xLDAqBgNVBAMT I1RoYXd0ZSBQZXJzb25hbCBGcmVlbWFpbCBJc3N1aW5nIENBMB4XDTA4MDIxMzE1MTUxN1oX DTA5MDIxMjE1MTUxN1owbDEOMAwGA1UEBBMFTXVlbnoxEDAOBgNVBCoTB0dlcmhhcmQxFjAU BgNVBAMTDUdlcmhhcmQgTXVlbnoxMDAuBgkqhkiG9w0BCQEWIW11ZW56QGluZm9ybWF0aWsu dW5pLXR1ZWJpbmdlbi5kZTCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAMZex/Sq sAxkzTVvKP/YAgkaeXA+ngH59Aa0bbRPsKOWzAndGqty5EKcEzrnKqEJ27qHFvoF/pHp88U2 7SJI/xbqkgWeV2jRaldipZQYlnjYLQcmb4cewIFuGRRSVrm3BquzX38aYazuE4+DVH2Z3a8z n0FcdMXhA1NR2Ma1rh4G7SIeZ+hC7czbvNRPraBliGdQhs8J/6yP/iL8aNYAl9c7CL4ofRj8 Y9orMOV/4vtWTq76/VQUVdbhUMiv0D8aHqI1ZvGskhRRvmITgQRVbbn8N8WTpZ0UCgMDjxPP 9i5IhLfp6oBtsKl4OZ0RXvSLZrbJTkBX3vnEutcyxDvyNgMCAwEAAaM+MDwwLAYDVR0RBCUw I4EhbXVlbnpAaW5mb3JtYXRpay51bmktdHVlYmluZ2VuLmRlMAwGA1UdEwEB/wQCMAAwDQYJ KoZIhvcNAQEFBQADgYEAX5SiD6epJePwBjJumOsTF6wzeuZRDLYlN+fOpXwd2C0Yx6i8iIZ9 l/J/nGaE1YpJPfX5oJDE+tOk1vYh2E9ThLOj9kJ3buZmgOCdVu90qtCWhfhli7RCYcJ+G9M3 FCnqbrzI/waPPXGB8/DY1HKgPj5G+oKPUK+GD2aE1Q3PYGowggMVMIICfqADAgECAhA/WhrG FpDK/rOM5sjg4QfiMA0GCSqGSIb3DQEBBQUAMGIxCzAJBgNVBAYTAlpBMSUwIwYDVQQKExxU aGF3dGUgQ29uc3VsdGluZyAoUHR5KSBMdGQuMSwwKgYDVQQDEyNUaGF3dGUgUGVyc29uYWwg RnJlZW1haWwgSXNzdWluZyBDQTAeFw0wODAyMTMxNTE1MTdaFw0wOTAyMTIxNTE1MTdaMGwx DjAMBgNVBAQTBU11ZW56MRAwDgYDVQQqEwdHZXJoYXJkMRYwFAYDVQQDEw1HZXJoYXJkIE11 ZW56MTAwLgYJKoZIhvcNAQkBFiFtdWVuekBpbmZvcm1hdGlrLnVuaS10dWViaW5nZW4uZGUw ggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQDGXsf0qrAMZM01byj/2AIJGnlwPp4B +fQGtG2020T7CjlswJ3RqrcuRCnBM65yqhCdu6hxb6Bf6R6fPFNu0iSP8W6pIFnldo0WpXYqWU GJZ42C0HJm+HHsCBbhkUUla5twars19/GmGs7hOPg1R9md2vM59BXHTF4QNTUdjGta4eBu0i HmfoQu3M27zUT62gZYhnUIbPCf+sj/4i/GjWAJfXOwi+KH0Y/GPaKzDlf+L7Vk6u+v1UFFXW 4VDIr9A/Gh6iNWbxrJIUUb5iE4EEVW25/DfFk6WdFAoDA48Tz/YuSIS36eqAbbCpeDmdEV70 i2a2yU5AV975xLrXMsQ78jYDAgMBAAGjPjA8MCwGA1UdEQQlMCOBIW11ZW56QGluZm9ybWF0 aWsudW5pLXR1ZWJpbmdlbi5kZTAMBgNVHRMBAf8EAjAAMA0GCSqGSIb3DQEBBQUAA4GBAF+U og+nqSXj8AYybpjrExesM3rmUQy2JTfnzqV8HdgtGMeovIiGfZfyf5xmhNWKST31+aCQxPrT pNb2IdhPU4Szo/ZCd27mZoDgnVbvdKrQloX4ZYu0QmHCfhvTNxQp6m68yP8Gjz1xgfPw2NRy oD4+RvqCj1Cvhg9mhNUNz2BqMIIDPzCCAqigAwIBAgIBDTANBgkqhkiG9w0BAQUFADCB0TEL MAkGA1UEBhMCWkExFTATBgNVBAgTDFdlc3Rlcm4gQ2FwZTESMBAGA1UEBxMJQ2FwZSBUb3du MRowGAYDVQQKExFUaGF3dGUgQ29uc3VsdGluZzEoMCYGA1UECxMfQ2VydGlmaWNhdGlvbiBT ZXJ2aWNlcyBEaXZpc2lvbjEkMCIGA1UEAxMbVGhhd3RlIFBlcnNvbmFsIEZyZWVtYWlsIENB MSswKQYJKoZIhvcNAQkBFhxwZXJzb25hbC1mcmVlbWFpbEB0aGF3dGUuY29tMB4XDTAzMDcx NzAwMDAwMFoXDTEzMDcxNjIzNTk1OVowYjELMAkGA1UEBhMCWkExJTAjBgNVBAoTHFRoYXd0 ZSBDb25zdWx0aW5nIChQdHkpIEx0ZC4xLDAqBgNVBAMTI1RoYXd0ZSBQZXJzb25hbCBGcmVl bWFpbCBJc3N1aW5nIENBMIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQDEpjxVc1X7TrnK mVoeaMB1BHCd3+n/ox7svc31W/Iadr1/DDph8r9RzgHU5VAKMNcCY1osiRVwjt3J8CuFWqo/ cVbLrzwLB+fxH5E2JCoTzyvV84J3PQO+K/67GD4Hv0CAAmTXp6a7n2XRxSpUhQ9IBH+nttE8 YQRAHmQZcmC3+wIDAQABo4GUMIGRMBIGA1UdEwEB/wQIMAYBAf8CAQAwQwYDVR0fBDwwOjA4 oDagNIYyaHR0cDovL2NybC50aGF3dGUuY29tL1RoYXd0ZVBlcnNvbmFsRnJlZW1haWxDQS5j cmwwCwYDVR0PBAQDAgEGMCkGA1UdEQQiMCCkHjAcMRowGAYDVQQDExFQcml2YXRlTGFiZWwy LTEzODANBgkqhkiG9w0BAQUFAAOBgQBIjNFQg+oLLswNo2asZw9/r6y+whehQ5aUnX9MIbj4 Nh+qLZ82L8D0HFAgk3A8/a3hYWLD2ToZfoSxmRsAxRoLgnSeJVCUYsfbJ3FXJY3dqZw5jowg T2Vfldr394fWxghOrvbqNOUQGls1TXfjViF4gtwhGTXeJLHTHUb/XV9lTzGCA2QwggNgAgEB MHYwYjELMAkGA1UEBhMCWkExJTAjBgNVBAoTHFRoYXd0ZSBDb25zdWx0aW5nIChQdHkpIEx0 ZC4xLDAqBgNVBAMTI1RoYXd0ZSBQZXJzb25hbCBGcmVlbWFpbCBJc3N1aW5nIENBAhA/WhrG FpDK/rOM5sjg4QfiMAkGBSsOAwIaBQCgggHDMBgGCSqGSIb3DQEJAzELBgkqhkiG9w0BBwEw HAYJKoZIhvcNAQkFMQ8XDTA4MDgyOTA5MjczM1owIwYJKoZIhvcNAQkEMRYEFMDKyg2SxIW1 NAisoKPsBj3uQDZlMFIGCSqGSIb3DQEJDzFFMEMwCgYIKoZIhvcNAwcwDgYIKoZIhvcNAwIC AgCAMA0GCCqGSIb3DQMCAgFAMAcGBSsOAwIHMA0GCCqGSIb3DQMCAgEoMIGFBgkrBgEEAYI3 EAQxeDB2MGIxCzAJBgNVBAYTAlpBMSUwIwYDVQQKExxUaGF3dGUgQ29uc3VsdGluZyAoUHR5 KSBMdGQuMSwwKgYDVQQDEyNUaGF3dGUgUGVyc29uYWwgRnJlZW1haWwgSXNzdWluZyBDQQIQ P1oaxhaQyv6zjObI4OEH4jCBhwYLKoZIhvcNAQkQAgsxeKB2MGIxCzAJBgNVBAYTAlpBMSUw IwYDVQQKExxUaGF3dGUgQ29uc3VsdGluZyAoUHR5KSBMdGQuMSwwKgYDVQQDEyNUaGF3dGUg UGVyc29uYWwgRnJlZW1haWwgSXNzdWluZyBDQQIQP1oaxhaQyv6zjObI4OEH4jANBgkqhkiG 9w0BAQEFAASCAQC3yaOLCCXlsjfnSaze0dFuYUGpNU2RMblhRuggRUJ3YhZXLk8UmTOWS0OE 5XDYgeO/01Q2MzL0Ag2Sm8ORoFndjxs8I12dq4J275d1q72mmLYkgZQymOt0vnCEcYLn17qI UKhwyc4RUGr14L1+h5GGQK3QpNwm4tKkdfXMOKcrhujVUt7G0SWwEOqqKxU7f3/z9HIxg78f iNzd+YeuruUZ9p7bMGu0WValSP3T0hzk4ZHU0jCfrdIw8RNhIUaiEH7wtUKnOTloAF5nfB+P UuiY9f3lKY6hTmuQIVyoXXUuvLo9+OzOCdK2tuuza1Nu6W4YoouFP7pk+pOzavird1vgAAAA AAAA --------------ms020408060505010004040409-- --===============1091118548== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ IPFIX mailing list IPFIX@ietf.org https://www.ietf.org/mailman/listinfo/ipfix --===============1091118548==-- T7CjlswJ3RqrcuRCnBM65yqhCdu6hxb6Bf6R6fPFNu0iSP8W6pIFnldo0WpXYqWU GJZ42C0HJm+HHsCBbhkUUla5twars19/GmGs7hOPg1R9md2vM59BXHTF4QNTUdjGta4eBu0i HmfoQu3M27zUT62gZYhnUIbPCf+sj/4i/GjWAJfXOwi+KH0Y/GPaKzDlf+L7Vk6u+v1UFFXW 4VDIr9A/Gh6iNWbxrJIUUb5iE4EEVW25/DfFk6WdFAoDA48Tz/YuSIS36eqAbbCpeDmdEV70 i2a2yU5AV975xLrXMsQ78jYDAgMBAAGjPjA8MCwGA1UdEQQlMCOBIW11ZW56QGluZm9ybWF0 aWsudW5pLXR1ZWJpbmdlbi5kZTAMBgNVHRMBAf8EAjAAMA0GCSqGSIb3DQEBBQUAA4GBAF+U og+nqSXj8AYybpjrExesM3rmUQy2JTfnzqV8HdgtGMeovIiGfZfyf5xmhNWKST31+aCQxPrT pNb2IdhPU4Szo/ZCd27mZoDgnVbvdKrQloX4ZYu0QmHCfhvTNxQp6m68yP8Gjz1xgfPw2NRy oD4+RvqCj1Cvhg9mhNUNz2BqMIIDPzCCAqigAwIBAgIBDTANBgkqhkiG9w0BAQUFADCB0TEL MAkGA1UEBhMCWkExFTATBgNVBAgTDFdlc3Rlcm4gQ2FwZTESMBAGA1UEBxMJQ2FwZSBUb3du MRowGAYDVQQKExFUaGF3dGUgQ29uc3VsdGluZzEoMCYGA1UECxMfQ2VydGlmaWNhdGlvbiBT ZXJ2aWNlcyBEaXZpc2lvbjEkMCIGA1UEAxMbVGhhd3RlIFBlcnNvbmFsIEZyZWVtYWlsIENB MSswKQYJKoZIhvcNAQkBFhxwZXJzb25hbC1mcmVlbWFpbEB0aGF3dGUuY29tMB4XDTAzMDcx NzAwMDAwMFoXDTEzMDcxNjIzNTk1OVowYjELMAkGA1UEBhMCWkExJTAjBgNVBAoTHFRoYXd0 ZSBDb25zdWx0aW5nIChQdHkpIEx0ZC4xLDAqBgNVBAMTI1RoYXd0ZSBQZXJzb25hbCBGcmVl bWFpbCBJc3N1aW5nIENBMIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQDEpjxVc1X7TrnK mVoeaMB1BHCd3+n/ox7svc31W/Iadr1/DDph8r9RzgHU5VAKMNcCY1osiRVwjt3J8CuFWqo/ cVbLrzwLB+fxH5E2JCoTzyvV84J3PQO+K/67GD4Hv0CAAmTXp6a7n2XRxSpUhQ9IBH+nttE8 YQRAHmQZcmC3+wIDAQABo4GUMIGRMBIGA1UdEwEB/wQIMAYBAf8CAQAwQwYDVR0fBDwwOjA4 oDagNIYyaHR0cDovL2NybC50aGF3dGUuY29tL1RoYXd0ZVBlcnNvbmFsRnJlZW1haWxDQS5j cmwwCwYDVR0PBAQDAgEGMCkGA1UdEQQiMCCkHjAcMRowGAYDVQQDExFQcml2YXRlTGFiZWwy LTEzODANBgkqhkiG9w0BAQUFAAOBgQBIjNFQg+oLLswNo2asZw9/r6y+whehQ5aUnX9MIbj4 Nh+qLZ82L8D0HFAgk3A8/a3hYWLD2ToZfoSxmRsAxRoLgnSeJVCUYsfbJ3FXJY3dqZw5jowg T2Vfldr394fWxghOrvbqNOUQGls1TXfjViF4gtwhGTXeJLHTHUb/XV9lTzGCA2QwggNgAgEB MHYwYjELMAkGA1UEBhMCWkExJTAjBgNVBAoTHFRoYXd0ZSBDb25zdWx0aW5nIChQdHkpIEx0 ZC4xLDAqBgNVBAMTI1RoYXd0ZSBQZXJzb25hbCBGcmVlbWFpbCBJc3N1aW5nIENBAhA/WhrG FpDK/rOM5sjg4QfiMAkGBSsOAwIaBQCgggHDMBgGCSqGSIb3DQEJAzELBgkqhkiG9w0BBwEw HAYJKoZIhvcNAQkFMQ8XDTA4MDgyOTA5MjczM1owIwYJKoZIhvcNAQkEMRYEFMDKyg2SxIW1 NAisoKPsBj3uQDZlMFIGCSqGSIb3DQEJDzFFMEMwCgYIKoZIhvcNAwcwDgYIKoZIhvcNAwIC AgCAMA0GCCqGSIb3DQMCAgFAMAcGBSsOAwIHMA0GCCqGSIb3DQMCAgEoMIGFBgkrBgEEAYI3 EAQxeDB2MGIxCzAJBgNVBAYTAlpBMSUwIwYDVQQKExxUaGF3dGUgQ29uc3VsdGluZyAoUHR5 KSBMdGQuMSwwKgYDVQQDEyNUaGF3dGUgUGVyc29uYWwgRnJlZW1haWwgSXNzdWluZyBDQQIQ P1oaxhaQyv6zjObI4OEH4jCBhwYLKoZIhvcNAQkQAgsxeKB2MGIxCzAJBgNVBAYTAlpBMSUw IwYDVQQKExxUaGF3dGUgQ29uc3VsdGluZyAoUHR5KSBMdGQuMSwwKgYDVQQDEyNUaGF3dGUg UGVyc29uYWwgRnJlZW1haWwgSXNzdWluZyBDQQIQP1oaxhaQyv6zjObI4OEH4jANBgkqhkiG 9w0BAQEFAASCAQC3yaOLCCXlsjfnSaze0dFuYUGpNU2RMblhRuggRUJ3YhZXLk8UmTOWS0OE 5XDYgeO/01Q2MzL0Ag2Sm8ORoFndjxs8I12dq4J275d1q72mmLYkgZQymOt0vnCEcYLn17qI UKhwyc4RUGr14L1+h5GGQK3QpNwm4tKkdfXMOKcrhujVUt7G0SWwEOqqKxU7f3/z9HIxg78f iNzd+YeuruUZ9p7bMGu0WValSP3T0hzk4ZHU0jCfrdIw8RNhIUaiEH7wtUKnOTloAF5nfB+P UuiY9f3lKY6hTmuQIVyoXXUuvLo9+OzOCdK2tuuza1Nu6W4YoouFP7pk+pOzavird1vgAAAA AAAA --------------ms020408060505010004040409-- --===============1091118548== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ IPFIX mailing list IPFIX@ietf.org https://www.ietf.org/mailman/listinfo/ipfix --===============1091118548==-- From ipfix-bounces@ietf.org Fri Aug 29 07:07:22 2008 Return-Path: X-Original-To: ipfix-archive@lists.ietf.org Delivered-To: ietfarch-ipfix-archive@core3.amsl.com Received: from [127.0.0.1] (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 075243A6A11; Fri, 29 Aug 2008 07:07:22 -0700 (PDT) X-Original-To: ipfix@core3.amsl.com Delivered-To: ipfix@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id B5F013A6819 for ; Fri, 29 Aug 2008 07:07:20 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -3.192 X-Spam-Level: X-Spam-Status: No, score=-3.192 tagged_above=-999 required=5 tests=[AWL=-0.943, BAYES_00=-2.599, HELO_EQ_DE=0.35] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 2na0JEyAESb7 for ; Fri, 29 Aug 2008 07:07:19 -0700 (PDT) Received: from smtp.cs.uni-tuebingen.de (u-173-c156.cs.uni-tuebingen.de [134.2.173.156]) by core3.amsl.com (Postfix) with ESMTP id 7A20C3A6A11 for ; Fri, 29 Aug 2008 07:07:19 -0700 (PDT) Received: from u-172-c138.cs.uni-tuebingen.de ([134.2.172.138]) by smtp.cs.uni-tuebingen.de with esmtpsa (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.69) (envelope-from ) id 1KZ4dG-0000t2-VW; Fri, 29 Aug 2008 16:07:23 +0200 Message-ID: <48B80299.1080303@informatik.uni-tuebingen.de> Date: Fri, 29 Aug 2008 16:07:21 +0200 From: Gerhard Muenz User-Agent: Thunderbird 2.0.0.16 (Windows/20080708) MIME-Version: 1.0 To: Brian Trammell References: <48A5B5ED.5040107@informatik.uni-tuebingen.de> <9B5C5A4E-2099-43B4-953E-674E38314582@tik.ee.ethz.ch> In-Reply-To: <9B5C5A4E-2099-43B4-953E-674E38314582@tik.ee.ethz.ch> Cc: IETF IPFIX Working Group Subject: Re: [IPFIX] WG last call on draft-ietf-ipfix-file-02 X-BeenThere: ipfix@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: IPFIX WG discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============0019352277==" Sender: ipfix-bounces@ietf.org Errors-To: ipfix-bounces@ietf.org This is a cryptographically signed message in MIME format. --===============0019352277== Content-Type: multipart/signed; protocol="application/x-pkcs7-signature"; micalg=sha1; boundary="------------ms080401080104000804030509" This is a cryptographically signed message in MIME format. --------------ms080401080104000804030509 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Brian, Brian Trammell wrote: > Thanks for your detailed review and comments. I'll address these by type. > > I. Comments related to the length and level of detail of the document: > > The IPFIX File draft differs from other IPFIX drafts in that it 1. > contains its own requirements (analogous to 3917 and 5101 being the same > document) and 2. contains more motivating text than other drafts, due to > its somewhat unusual nature within the IETF (standardizing a file format > without binding it to some specific transport). The authors believe that > the inclusion and length of the requirements and motivation sections > (sections 4 and 5) is warranted by these considerations. I partially agree: motivation and requirements should be included. The question is if we need that many pages. The general concept of the file format is straight forward and easy to understand. The important aspects can be explained on a couple of pages. So, an implementer who is already familiar with IPFIX shFrom ipfix-bounces@ietf.org Fri Aug 29 07:07:22 2008 Return-Path: X-Original-To: ipfix-archive@optimus.ietf.org Delivered-To: ietfarch-ipfix-archive@core3.amsl.com Received: from [127.0.0.1] (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 075243A6A11; Fri, 29 Aug 2008 07:07:22 -0700 (PDT) X-Original-To: ipfix@core3.amsl.com Delivered-To: ipfix@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id B5F013A6819 for ; Fri, 29 Aug 2008 07:07:20 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -3.192 X-Spam-Level: X-Spam-Status: No, score=-3.192 tagged_above=-999 required=5 tests=[AWL=-0.943, BAYES_00=-2.599, HELO_EQ_DE=0.35] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 2na0JEyAESb7 for ; Fri, 29 Aug 2008 07:07:19 -0700 (PDT) Received: from smtp.cs.uni-tuebingen.de (u-173-c156.cs.uni-tuebingen.de [134.2.173.156]) by core3.amsl.com (Postfix) with ESMTP id 7A20C3A6A11 for ; Fri, 29 Aug 2008 07:07:19 -0700 (PDT) Received: from u-172-c138.cs.uni-tuebingen.de ([134.2.172.138]) by smtp.cs.uni-tuebingen.de with esmtpsa (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.69) (envelope-from ) id 1KZ4dG-0000t2-VW; Fri, 29 Aug 2008 16:07:23 +0200 Message-ID: <48B80299.1080303@informatik.uni-tuebingen.de> Date: Fri, 29 Aug 2008 16:07:21 +0200 From: Gerhard Muenz User-Agent: Thunderbird 2.0.0.16 (Windows/20080708) MIME-Version: 1.0 To: Brian Trammell References: <48A5B5ED.5040107@informatik.uni-tuebingen.de> <9B5C5A4E-2099-43B4-953E-674E38314582@tik.ee.ethz.ch> In-Reply-To: <9B5C5A4E-2099-43B4-953E-674E38314582@tik.ee.ethz.ch> Cc: IETF IPFIX Working Group Subject: Re: [IPFIX] WG last call on draft-ietf-ipfix-file-02 X-BeenThere: ipfix@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: IPFIX WG discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============0019352277==" Sender: ipfix-bounces@ietf.org Errors-To: ipfix-bounces@ietf.org This is a cryptographically signed message in MIME format. --===============0019352277== Content-Type: multipart/signed; protocol="application/x-pkcs7-signature"; micalg=sha1; boundary="------------ms080401080104000804030509" This is a cryptographically signed message in MIME format. --------------ms080401080104000804030509 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Brian, Brian Trammell wrote: > Thanks for your detailed review and comments. I'll address these by type. > > I. Comments related to the length and level of detail of the document: > > The IPFIX File draft differs from other IPFIX drafts in that it 1. > contains its own requirements (analogous to 3917 and 5101 being the same > document) and 2. contains more motivating text than other drafts, due to > its somewhat unusual nature within the IETF (standardizing a file format > without binding it to some specific transport). The authors believe that > the inclusion and length of the requirements and motivation sections > (sections 4 and 5) is warranted by these considerations. I partially agree: motivation and requirements should be included. The question is if we need that many pages. The general concept of the file format is straight forward and easy to understand. The important aspects can be explained on a couple of pages. So, an implementer who is already familiar with IPFIX ould not be urged to read through 50 pages. What he needs to read are sections 3, 7, 8. Sections 4 to 6 provide background information which are not necessary to implement the standard (but rather useful to sell the standard to customers). Sections 9 and 10 cover special cases. Apart from reducing verbosity, you could mention in the introduction which sections contain the important information. > II. Typographical, editorial and stylistic comments: > > Thanks for these, there were quite a few typos/capitalization issues > we'd not caught yet. Stylistic suggestions have been incorporated where > they improve the clarity of the document. > > III. Terminology confusion: > > There are a couple of terms invented here in order to describe concepts > 1. for which there is no accepted terminology but 2. which appear only > for purposes of background. One of these is "IPFIX Message format", > which is intended to mean "the encoding and framing of IPFIX Messages > defined by RFC 5101 as used with the information model defined by RFC > 5102", or less formally, "IPFIX off the wire", _not_ the format of an > individual message. Your comments indicate some confusion here, and I > would appeal to the working group to suggest an alternative if this is a > problem. RFC 5101 section 3 defines the IPFIX Message format, which is the format of a single IPFIX Message. Do you propose to use "IPFIX Message format" in a different way? > Similarly, an "IPFIX Message stream" is a collection of IPFIX Messages > in some loose order. This is applying the word "stream" as it is used > when talking about storage, where there is no connotation in English of > an unbounded sequence. I'm going to suggest we leave this term, as the > suggested alternatives have their own problems: "Sequence" connotes > strict ordering (not the case; IPFIX Messages can be reordered within a > File containing a Message stream for e.g. template consistency.) > "Series" has a strict and nonapplicable mathematical meaning. Ok. > Alternately, a question to the WG: should we define "IPFIX Message > stream" and "IPFIX Message format" as intermediate terms in the > Terminology section of the document? I do not think that this is necessary. > The issues regarding the interchangeability of "IPFIX File" and "IPFIX > Transport Session" given the fact that Sessions may span Files have been > fixed; many thanks for pointing this one out. V9-related terminology has > been fixed, too; I got a bit confused in a couple of places switching > between V9 and IPFIX. > > IV. Technical issues: > > Time window resolution finer than seconds: Perhaps. We were trying to > avoid the explosion of timestamp IEs we have elsewhere in the > information model. Keep in mind that the Time Window is meant, in > multi-file environments, to determine whether a file must be scanned to > find information about a specific moment in time. Going below the > second-level resolution, at least in my experience, is not necessary > _for this use case_. > > Applicability of opaqueOctets: Indeed, one could create an > enterprise-specific IE and not export ET info about it, then it's > opaque. But 1. we still need to address the how to encap non-IPFIX info > in an IPFIX stream and 2. don't necessarily want to tell people go get a > PEN to do it. I'm inclined to keep opaqueOctets. I still do not see the need nor the utility of opaqueOctets. Who is "people"? Usually vendors, so they do have their PEN. On the other hand, if you read an IPFIX File including opaqueOctets fields, you do not know how to interprete it. In fact, opaqueOctets does not fulfill one of the important requirements of IPFIX File: self description. > Thanks again! Welcome. Gerhard --------------ms080401080104000804030509 Content-Type: application/x-pkcs7-signature; name="smime.p7s" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="smime.p7s" Content-Description: S/MIME Cryptographic Signature MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAQAAoIIJdTCC AxUwggJ+oAMCAQICED9aGsYWkMr+s4zmyODhB+IwDQYJKshould not be urged to read through 50 pages. What he needs to read are sections 3, 7, 8. Sections 4 to 6 provide background information which are not necessary to implement the standard (but rather useful to sell the standard to customers). Sections 9 and 10 cover special cases. Apart from reducing verbosity, you could mention in the introduction which sections contain the important information. > II. Typographical, editorial and stylistic comments: > > Thanks for these, there were quite a few typos/capitalization issues > we'd not caught yet. Stylistic suggestions have been incorporated where > they improve the clarity of the document. > > III. Terminology confusion: > > There are a couple of terms invented here in order to describe concepts > 1. for which there is no accepted terminology but 2. which appear only > for purposes of background. One of these is "IPFIX Message format", > which is intended to mean "the encoding and framing of IPFIX Messages > defined by RFC 5101 as used with the information model defined by RFC > 5102", or less formally, "IPFIX off the wire", _not_ the format of an > individual message. Your comments indicate some confusion here, and I > would appeal to the working group to suggest an alternative if this is a > problem. RFC 5101 section 3 defines the IPFIX Message format, which is the format of a single IPFIX Message. Do you propose to use "IPFIX Message format" in a different way? > Similarly, an "IPFIX Message stream" is a collection of IPFIX Messages > in some loose order. This is applying the word "stream" as it is used > when talking about storage, where there is no connotation in English of > an unbounded sequence. I'm going to suggest we leave this term, as the > suggested alternatives have their own problems: "Sequence" connotes > strict ordering (not the case; IPFIX Messages can be reordered within a > File containing a Message stream for e.g. template consistency.) > "Series" has a strict and nonapplicable mathematical meaning. Ok. > Alternately, a question to the WG: should we define "IPFIX Message > stream" and "IPFIX Message format" as intermediate terms in the > Terminology section of the document? I do not think that this is necessary. > The issues regarding the interchangeability of "IPFIX File" and "IPFIX > Transport Session" given the fact that Sessions may span Files have been > fixed; many thanks for pointing this one out. V9-related terminology has > been fixed, too; I got a bit confused in a couple of places switching > between V9 and IPFIX. > > IV. Technical issues: > > Time window resolution finer than seconds: Perhaps. We were trying to > avoid the explosion of timestamp IEs we have elsewhere in the > information model. Keep in mind that the Time Window is meant, in > multi-file environments, to determine whether a file must be scanned to > find information about a specific moment in time. Going below the > second-level resolution, at least in my experience, is not necessary > _for this use case_. > > Applicability of opaqueOctets: Indeed, one could create an > enterprise-specific IE and not export ET info about it, then it's > opaque. But 1. we still need to address the how to encap non-IPFIX info > in an IPFIX stream and 2. don't necessarily want to tell people go get a > PEN to do it. I'm inclined to keep opaqueOctets. I still do not see the need nor the utility of opaqueOctets. Who is "people"? Usually vendors, so they do have their PEN. On the other hand, if you read an IPFIX File including opaqueOctets fields, you do not know how to interprete it. In fact, opaqueOctets does not fulfill one of the important requirements of IPFIX File: self description. > Thanks again! Welcome. Gerhard --------------ms080401080104000804030509 Content-Type: application/x-pkcs7-signature; name="smime.p7s" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="smime.p7s" Content-Description: S/MIME Cryptographic Signature MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAQAAoIIJdTCC AxUwggJ+oAMCAQICED9aGsYWkMr+s4zmyODhB+IwDQYoZIhvcNAQEFBQAwYjELMAkGA1UE BhMCWkExJTAjBgNVBAoTHFRoYXd0ZSBDb25zdWx0aW5nIChQdHkpIEx0ZC4xLDAqBgNVBAMT I1RoYXd0ZSBQZXJzb25hbCBGcmVlbWFpbCBJc3N1aW5nIENBMB4XDTA4MDIxMzE1MTUxN1oX DTA5MDIxMjE1MTUxN1owbDEOMAwGA1UEBBMFTXVlbnoxEDAOBgNVBCoTB0dlcmhhcmQxFjAU BgNVBAMTDUdlcmhhcmQgTXVlbnoxMDAuBgkqhkiG9w0BCQEWIW11ZW56QGluZm9ybWF0aWsu dW5pLXR1ZWJpbmdlbi5kZTCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAMZex/Sq sAxkzTVvKP/YAgkaeXA+ngH59Aa0bbRPsKOWzAndGqty5EKcEzrnKqEJ27qHFvoF/pHp88U2 7SJI/xbqkgWeV2jRaldipZQYlnjYLQcmb4cewIFuGRRSVrm3BquzX38aYazuE4+DVH2Z3a8z n0FcdMXhA1NR2Ma1rh4G7SIeZ+hC7czbvNRPraBliGdQhs8J/6yP/iL8aNYAl9c7CL4ofRj8 Y9orMOV/4vtWTq76/VQUVdbhUMiv0D8aHqI1ZvGskhRRvmITgQRVbbn8N8WTpZ0UCgMDjxPP 9i5IhLfp6oBtsKl4OZ0RXvSLZrbJTkBX3vnEutcyxDvyNgMCAwEAAaM+MDwwLAYDVR0RBCUw I4EhbXVlbnpAaW5mb3JtYXRpay51bmktdHVlYmluZ2VuLmRlMAwGA1UdEwEB/wQCMAAwDQYJ KoZIhvcNAQEFBQADgYEAX5SiD6epJePwBjJumOsTF6wzeuZRDLYlN+fOpXwd2C0Yx6i8iIZ9 l/J/nGaE1YpJPfX5oJDE+tOk1vYh2E9ThLOj9kJ3buZmgOCdVu90qtCWhfhli7RCYcJ+G9M3 FCnqbrzI/waPPXGB8/DY1HKgPj5G+oKPUK+GD2aE1Q3PYGowggMVMIICfqADAgECAhA/WhrG FpDK/rOM5sjg4QfiMA0GCSqGSIb3DQEBBQUAMGIxCzAJBgNVBAYTAlpBMSUwIwYDVQQKExxU aGF3dGUgQ29uc3VsdGluZyAoUHR5KSBMdGQuMSwwKgYDVQQDEyNUaGF3dGUgUGVyc29uYWwg RnJlZW1haWwgSXNzdWluZyBDQTAeFw0wODAyMTMxNTE1MTdaFw0wOTAyMTIxNTE1MTdaMGwx DjAMBgNVBAQTBU11ZW56MRAwDgYDVQQqEwdHZXJoYXJkMRYwFAYDVQQDEw1HZXJoYXJkIE11 ZW56MTAwLgYJKoZIhvcNAQkBFiFtdWVuekBpbmZvcm1hdGlrLnVuaS10dWViaW5nZW4uZGUw ggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQDGXsf0qrAMZM01byj/2AIJGnlwPp4B +fQGtG20T7CjlswJ3RqrcuRCnBM65yqhCdu6hxb6Bf6R6fPFNu0iSP8W6pIFnldo0WpXYqWU GJZ42C0HJm+HHsCBbhkUUla5twars19/GmGs7hOPg1R9md2vM59BXHTF4QNTUdjGta4eBu0i HmfoQu3M27zUT62gZYhnUIbPCf+sj/4i/GjWAJfXOwi+KH0Y/GPaKzDlf+L7Vk6u+v1UFFXW 4VDIr9A/Gh6iNWbxrJIUUb5iE4EEVW25/DfFk6WdFAoDA48Tz/YuSIS36eqAbbCpeDmdEV70 i2a2yU5AV975xLrXMsQ78jYDAgMBAAGjPjA8MCwGA1UdEQQlMCOBIW11ZW56QGluZm9ybWF0 aWsudW5pLXR1ZWJpbmdlbi5kZTAMBgNVHRMBAf8EAjAAMA0GCSqGSIb3DQEBBQUAA4GBAF+U og+nqSXj8AYybpjrExesM3rmUQy2JTfnzqV8HdgtGMeovIiGfZfyf5xmhNWKST31+aCQxPrT pNb2IdhPU4Szo/ZCd27mZoDgnVbvdKrQloX4ZYu0QmHCfhvTNxQp6m68yP8Gjz1xgfPw2NRy oD4+RvqCj1Cvhg9mhNUNz2BqMIIDPzCCAqigAwIBAgIBDTANBgkqhkiG9w0BAQUFADCB0TEL MAkGA1UEBhMCWkExFTATBgNVBAgTDFdlc3Rlcm4gQ2FwZTESMBAGA1UEBxMJQ2FwZSBUb3du MRowGAYDVQQKExFUaGF3dGUgQ29uc3VsdGluZzEoMCYGA1UECxMfQ2VydGlmaWNhdGlvbiBT ZXJ2aWNlcyBEaXZpc2lvbjEkMCIGA1UEAxMbVGhhd3RlIFBlcnNvbmFsIEZyZWVtYWlsIENB MSswKQYJKoZIhvcNAQkBFhxwZXJzb25hbC1mcmVlbWFpbEB0aGF3dGUuY29tMB4XDTAzMDcx NzAwMDAwMFoXDTEzMDcxNjIzNTk1OVowYjELMAkGA1UEBhMCWkExJTAjBgNVBAoTHFRoYXd0 ZSBDb25zdWx0aW5nIChQdHkpIEx0ZC4xLDAqBgNVBAMTI1RoYXd0ZSBQZXJzb25hbCBGcmVl bWFpbCBJc3N1aW5nIENBMIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQDEpjxVc1X7TrnK mVoeaMB1BHCd3+n/ox7svc31W/Iadr1/DDph8r9RzgHU5VAKMNcCY1osiRVwjt3J8CuFWqo/ cVbLrzwLB+fxH5E2JCoTzyvV84J3PQO+K/67GD4Hv0CAAmTXp6a7n2XRxSpUhQ9IBH+nttE8 YQRAHmQZcmC3+wIDAQABo4GUMIGRMBIGA1UdEwEB/wQIMAYBAf8CAQAwQwYDVR0fBDwwOjA4 oDagNIYyaHR0cDovL2NybC50aGF3dGUuY29tL1RoYXd0ZVBlcnNvbmFsRnJlZW1haWxDQS5j cmwwCwYDVR0PBAQDAgEGMCkGA1UdEQQiMCCkHjAcMRowGAYDVQQDExFQcml2YXRlTGFiZWwy LTEzODANBgkqhkiG9w0BAQUFAAOBgQBIjNFQg+oLLswNo2asZw9/r6y+whehQ5aUnX9MIbj4 Nh+qLZ82L8D0HFAgk3A8/a3hYWLD2ToZfoSxmRsAxRoLgnSeJVCUYsfbJ3FXJY3dqZw5jowg T2Vfldr394fWxghOrvbqNOUQGls1TXfjViF4gtwhGTXeJLHTHUb/XV9lTzGCA2QwggNgAgEB MHYwYjELMAkGA1UEBhMCWkExJTAjBgNVBAoTHFRoYXd0ZSBDb25zdWx0aW5nIChQdHkpIEx0 ZC4xLDAqBgNVBAMTI1RoYXd0ZSBQZXJzb25hbCBGcmVlbWFpbCBJc3N1aW5nIENBAhA/WhrG FpDK/rOM5sjg4QfiMAkGBSsOAwIaBQCgggHDMBgGCSqGSIb3DQEJAzELBgkqhkiG9w0BBwEw HAYJKoZIhvcNAQkFMQ8XDTA4MDgyOTE0MDcyMVowIwYJKoZIhvcNAQkEMRYEFKXHa7hDL+3n rvHbKWWefmv/QIjOMFIGCSqGSIb3DQEJDzFFMEMwCgYIKoZIhvcNAwcwDgYIKoZIhvcNAwIC AgCAMA0GCCqGSIb3DQMCAgFAMAcGBSsOAwIHMA0GCCqGSIb3DQMCAgEoMIGFBgkrBgEEAYI3 EAQxeDB2MGIxCzAJBgNVBAYTAlpBMSUwIwYDVQQKExxUaGF3dGUgQ29uc3VsdGluZyAoUHR5 KSBMdGQuMSwwKgYDVQQDEyNUaGF3dGUgUGVyc29uYWwgRnJlZW1haWwgSXNzdWluZyBDQQIQ P1oaxhaQyv6zjObI4OEH4jCBhwYLKoZIhvcNAQkQAgsxeKB2MGIxCzAJBgNVBAYTAlpBMSUw IwYDVQQKExxUaGF3dGUgQ29uc3VsdGluZyAoUHR5KSBMdGQuMSwwKgYDVQQDEyNUaGF3dGUg UGVyc29uYWwgRnJlZW1haWwgSXNzdWluZyBDQQIQP1oaxhaQyv6zjObI4OEH4jANBgkqhkiG 9w0BAQEFAASCAQBlS8oEPDr6qvTkMWbwFVeROzY8NrWYWNG80RZ7sYep7uqbstm0OxYzElAJ K+hzNW9+MiK0GZDAoD3GZGlkO+cEUoUOzVp9KX2LuJxtvdHKpICtLGUEk9dxShUfsXCkul9u 5ivG/eCpesMVHal3CMVVNQElKJsU23xQkNPD2kVRkRQCNijOileeH4uZghle+x+YZ04V2YYT pqfiveaz0xdMVsCeCeFtdUc0pKoX9Wkjde7TqmZEe//s4CVzI2jCU5+6f0jg2WgNgBmG8FPu yVsq09W7NQeRFGbOLh3kox9Z4NOgLFBhdP7367Tg0HuICdHelx9zjGrCkqfrGS4of8GGAAAA AAAA --------------ms080401080104000804030509-- --===============0019352277== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ IPFIX mailing list IPFIX@ietf.org https://www.ietf.org/mailman/listinfo/ipfix --===============0019352277==-- JKoZIhvcNAQEFBQAwYjELMAkGA1UE BhMCWkExJTAjBgNVBAoTHFRoYXd0ZSBDb25zdWx0aW5nIChQdHkpIEx0ZC4xLDAqBgNVBAMT I1RoYXd0ZSBQZXJzb25hbCBGcmVlbWFpbCBJc3N1aW5nIENBMB4XDTA4MDIxMzE1MTUxN1oX DTA5MDIxMjE1MTUxN1owbDEOMAwGA1UEBBMFTXVlbnoxEDAOBgNVBCoTB0dlcmhhcmQxFjAU BgNVBAMTDUdlcmhhcmQgTXVlbnoxMDAuBgkqhkiG9w0BCQEWIW11ZW56QGluZm9ybWF0aWsu dW5pLXR1ZWJpbmdlbi5kZTCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAMZex/Sq sAxkzTVvKP/YAgkaeXA+ngH59Aa0bbRPsKOWzAndGqty5EKcEzrnKqEJ27qHFvoF/pHp88U2 7SJI/xbqkgWeV2jRaldipZQYlnjYLQcmb4cewIFuGRRSVrm3BquzX38aYazuE4+DVH2Z3a8z n0FcdMXhA1NR2Ma1rh4G7SIeZ+hC7czbvNRPraBliGdQhs8J/6yP/iL8aNYAl9c7CL4ofRj8 Y9orMOV/4vtWTq76/VQUVdbhUMiv0D8aHqI1ZvGskhRRvmITgQRVbbn8N8WTpZ0UCgMDjxPP 9i5IhLfp6oBtsKl4OZ0RXvSLZrbJTkBX3vnEutcyxDvyNgMCAwEAAaM+MDwwLAYDVR0RBCUw I4EhbXVlbnpAaW5mb3JtYXRpay51bmktdHVlYmluZ2VuLmRlMAwGA1UdEwEB/wQCMAAwDQYJ KoZIhvcNAQEFBQADgYEAX5SiD6epJePwBjJumOsTF6wzeuZRDLYlN+fOpXwd2C0Yx6i8iIZ9 l/J/nGaE1YpJPfX5oJDE+tOk1vYh2E9ThLOj9kJ3buZmgOCdVu90qtCWhfhli7RCYcJ+G9M3 FCnqbrzI/waPPXGB8/DY1HKgPj5G+oKPUK+GD2aE1Q3PYGowggMVMIICfqADAgECAhA/WhrG FpDK/rOM5sjg4QfiMA0GCSqGSIb3DQEBBQUAMGIxCzAJBgNVBAYTAlpBMSUwIwYDVQQKExxU aGF3dGUgQ29uc3VsdGluZyAoUHR5KSBMdGQuMSwwKgYDVQQDEyNUaGF3dGUgUGVyc29uYWwg RnJlZW1haWwgSXNzdWluZyBDQTAeFw0wODAyMTMxNTE1MTdaFw0wOTAyMTIxNTE1MTdaMGwx DjAMBgNVBAQTBU11ZW56MRAwDgYDVQQqEwdHZXJoYXJkMRYwFAYDVQQDEw1HZXJoYXJkIE11 ZW56MTAwLgYJKoZIhvcNAQkBFiFtdWVuekBpbmZvcm1hdGlrLnVuaS10dWViaW5nZW4uZGUw ggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQDGXsf0qrAMZM01byj/2AIJGnlwPp4B +fQGtG20T7CjlswJ3RqrcuRCnBM65yqhCdu6hxb6Bf6R6fPFNu0iSP8W6pIFnldo0WpXYqWU GJZ42C0HJm+HHsCBbhkUUla5twars19/GmGs7hOPg1R9md2vM59BXHTF4QNTUdjGta4eBu0i HmfoQu3M27zUT62gZYhnUIbPCf+sj/4i/GjWAJfXOwi+KH0Y/GPaKzDlf+L7Vk6u+v1UFFXW 4VDIr9A/Gh6iNWbxrJIUUb5iE4EEVW25/DfFk6WdFAoDA48Tz/YuSIS36eqAbbCpeDmdEV70 i2a2yU5AV975xLrXMsQ78jYDAgMBAAGjPjA8MCwGA1UdEQQlMCOBIW11ZW56QGluZm9ybWF0 aWsudW5pLXR1ZWJpbmdlbi5kZTAMBgNVHRMBAf8EAjAAMA0GCSqGSIb3DQEBBQUAA4GBAF+U og+nqSXj8AYybpjrExesM3rmUQy2JTfnzqV8HdgtGMeovIiGfZfyf5xmhNWKST31+aCQxPrT pNb2IdhPU4Szo/ZCd27mZoDgnVbvdKrQloX4ZYu0QmHCfhvTNxQp6m68yP8Gjz1xgfPw2NRy oD4+RvqCj1Cvhg9mhNUNz2BqMIIDPzCCAqigAwIBAgIBDTANBgkqhkiG9w0BAQUFADCB0TEL MAkGA1UEBhMCWkExFTATBgNVBAgTDFdlc3Rlcm4gQ2FwZTESMBAGA1UEBxMJQ2FwZSBUb3du MRowGAYDVQQKExFUaGF3dGUgQ29uc3VsdGluZzEoMCYGA1UECxMfQ2VydGlmaWNhdGlvbiBT ZXJ2aWNlcyBEaXZpc2lvbjEkMCIGA1UEAxMbVGhhd3RlIFBlcnNvbmFsIEZyZWVtYWlsIENB MSswKQYJKoZIhvcNAQkBFhxwZXJzb25hbC1mcmVlbWFpbEB0aGF3dGUuY29tMB4XDTAzMDcx NzAwMDAwMFoXDTEzMDcxNjIzNTk1OVowYjELMAkGA1UEBhMCWkExJTAjBgNVBAoTHFRoYXd0 ZSBDb25zdWx0aW5nIChQdHkpIEx0ZC4xLDAqBgNVBAMTI1RoYXd0ZSBQZXJzb25hbCBGcmVl bWFpbCBJc3N1aW5nIENBMIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQDEpjxVc1X7TrnK mVoeaMB1BHCd3+n/ox7svc31W/Iadr1/DDph8r9RzgHU5VAKMNcCY1osiRVwjt3J8CuFWqo/ cVbLrzwLB+fxH5E2JCoTzyvV84J3PQO+K/67GD4Hv0CAAmTXp6a7n2XRxSpUhQ9IBH+nttE8 YQRAHmQZcmC3+wIDAQABo4GUMIGRMBIGA1UdEwEB/wQIMAYBAf8CAQAwQwYDVR0fBDwwOjA4 oDagNIYyaHR0cDovL2NybC50aGF3dGUuY29tL1RoYXd0ZVBlcnNvbmFsRnJlZW1haWxDQS5j cmwwCwYDVR0PBAQDAgEGMCkGA1UdEQQiMCCkHjAcMRowGAYDVQQDExFQcml2YXRlTGFiZWwy LTEzODANBgkqhkiG9w0BAQUFAAOBgQBIjNFQg+oLLswNo2asZw9/r6y+whehQ5aUnX9MIbj4 Nh+qLZ82L8D0HFAgk3A8/a3hYWLD2ToZfoSxmRsAxRoLgnSeJVCUYsfbJ3FXJY3dqZw5jowg T2Vfldr394fWxghOrvbqNOUQGls1TXfjViF4gtwhGTXeJLHTHUb/XV9lTzGCA2QwggNgAgEB MHYwYjELMAkGA1UEBhMCWkExJTAjBgNVBAoTHFRoYXd0ZSBDb25zdWx0aW5nIChQdHkpIEx0 ZC4xLDAqBgNVBAMTI1RoYXd0ZSBQZXJzb25hbCBGcmVlbWFpbCBJc3N1aW5nIENBAhA/WhrG FpDK/rOM5sjg4QfiMAkGBSsOAwIaBQCgggHDMBgGCSqGSIb3DQEJAzELBgkqhkiG9w0BBwEw HAYJKoZIhvcNAQkFMQ8XDTA4MDgyOTE0MDcyMVowIwYJKoZIhvcNAQkEMRYEFKXHa7hDL+3n rvHbKWWefmv/QIjOMFIGCSqGSIb3DQEJDzFFMEMwCgYIKoZIhvcNAwcwDgYIKoZIhvcNAwIC AgCAMA0GCCqGSIb3DQMCAgFAMAcGBSsOAwIHMA0GCCqGSIb3DQMCAgEoMIGFBgkrBgEEAYI3 EAQxeDB2MGIxCzAJBgNVBAYTAlpBMSUwIwYDVQQKExxUaGF3dGUgQ29uc3VsdGluZyAoUHR5 KSBMdGQuMSwwKgYDVQQDEyNUaGF3dGUgUGVyc29uYWwgRnJlZW1haWwgSXNzdWluZyBDQQIQ P1oaxhaQyv6zjObI4OEH4jCBhwYLKoZIhvcNAQkQAgsxeKB2MGIxCzAJBgNVBAYTAlpBMSUw IwYDVQQKExxUaGF3dGUgQ29uc3VsdGluZyAoUHR5KSBMdGQuMSwwKgYDVQQDEyNUaGF3dGUg UGVyc29uYWwgRnJlZW1haWwgSXNzdWluZyBDQQIQP1oaxhaQyv6zjObI4OEH4jANBgkqhkiG 9w0BAQEFAASCAQBlS8oEPDr6qvTkMWbwFVeROzY8NrWYWNG80RZ7sYep7uqbstm0OxYzElAJ K+hzNW9+MiK0GZDAoD3GZGlkO+cEUoUOzVp9KX2LuJxtvdHKpICtLGUEk9dxShUfsXCkul9u 5ivG/eCpesMVHal3CMVVNQElKJsU23xQkNPD2kVRkRQCNijOileeH4uZghle+x+YZ04V2YYT pqfiveaz0xdMVsCeCeFtdUc0pKoX9Wkjde7TqmZEe//s4CVzI2jCU5+6f0jg2WgNgBmG8FPu yVsq09W7NQeRFGbOLh3kox9Z4NOgLFBhdP7367Tg0HuICdHelx9zjGrCkqfrGS4of8GGAAAA AAAA --------------ms080401080104000804030509-- --===============0019352277== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ IPFIX mailing list IPFIX@ietf.org https://www.ietf.org/mailman/listinfo/ipfix --===============0019352277==-- From ipfix-bounces@ietf.org Fri Aug 29 08:01:34 2008 Return-Path: X-Original-To: ipfix-archive@lists.ietf.org Delivered-To: ietfarch-ipfix-archive@core3.amsl.com Received: from [127.0.0.1] (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 3143D3A6917; Fri, 29 Aug 2008 08:01:34 -0700 (PDT) X-Original-To: ipfix@core3.amsl.com Delivered-To: ipfix@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 9A9D03A69E1 for ; Fri, 29 Aug 2008 08:01:33 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -4.971 X-Spam-Level: X-Spam-Status: No, score=-4.971 tagged_above=-999 required=5 tests=[AWL=1.628, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id NDONM8BahVk7 for ; Fri, 29 Aug 2008 08:01:32 -0700 (PDT) Received: from smtp.ee.ethz.ch (smtp.ee.ethz.ch [129.132.2.219]) by core3.amsl.com (Postfix) with ESMTP id 5A4C13A687C for ; Fri, 29 Aug 2008 08:01:28 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by smtp.ee.ethz.ch (Postfix) with ESMTP id 648BBD9338; Fri, 29 Aug 2008 17:01:30 +0200 (MEST) X-Virus-Scanned: by amavisd-new on smtp.ee.ethz.ch Received: from smtp.ee.ethz.ch ([127.0.0.1]) by localhost (.ee.ethz.ch [127.0.0.1]) (amavisd-new, port 10024) with LMTP id DXpQz8Qy47Up; Fri, 29 Aug 2008 17:01:30 +0200 (MEST) Received: from pb-10072.ethz.ch (pb-10072.ethz.ch [82.130.103.195]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) (Authenticated sender: briant) by smtp.ee.ethz.ch (Postfix) with ESMTP id EF2B2D93B9; Fri, 29 Aug 2008 17:01:29 +0200 (MEST) Message-Id: <907A7623-1FB0-4F6D-8D4A-92F4952867EC@tik.ee.ethz.ch> From: Brian Trammell To: Gerhard Muenz In-Reply-To: <48B80299.1080303@informatik.uni-tuebingen.de> Mime-Version: 1.0 (Apple Message framework v928.1) Date: Fri, 29 Aug 2008 17:01:29 +0200 References: <48A5B5ED.5040107@informatik.uni-tuebingen.de> <9B5C5A4E-2099-43B4-953E-674E38314582@tik.ee.ethz.ch> <48B80299.1080303@informatik.uni-tuebingen.de> X-Mailer: Apple Mail (2.928.1) Cc: IETF IPFIX Working Group Subject: Re: [IPFIX] WG last call on draft-ietf-ipfix-file-02 X-BeenThere: ipfix@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: IPFIX WG discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: ipfix-bounces@ietf.org Errors-To: ipfix-bounces@ietf.org On Aug 29, 2008, at 4:07 PM, Gerhard Muenz wrote: > > Brian, > > Brian Trammell wrote: >> Thanks for your detailed review and comments. I'll address these by >> type. >> >> I. Comments related to the length and level of detail of the >> document: >> >> The IPFIX File draft differs from other IPFIX drafts in that it 1. >> contains its own requirements (analogous to 3917 and 5101 being the >> same >> document) and 2. contains more motivating text than other drafts, >> due to >> its somewhat unusual nature within the IETF (standardizing a file >> format >> without binding it to some specific transport). The authors believe >> that >> the inclusion and length of the requirements and motivation sections >> (sections 4 and 5) is warranted by these considerations. > > I partially agree: motivation and requirements should be included. The > question is if we need that many pages. > > The general concept of the file format is straight forward and easy to > understand. The important aspects can be explained on a couple of > pages. > So, an implementer who is already familiar with IPFIX should not be > urged to read through 50 pages. What he needs to read are sections > 3, 7, > 8. Sections 4 to 6 provide background information which are not > necessary to implement the standard (but rather useful to sell the > standard to customers). Sections 9 and 10 cover special cases. Apart > from reducing verbosity, you could mention in the introduction which > sections contain the important information. This is an excellent point; I agree completely. I'll modify the introduction to add exactly this sort of implementor-friendly overview. >> III. Terminology confusion: >> >> There are a couple of terms invented here in order to describe >> concepts >> 1. for which there is no accepted terminology but 2. which appear >> only >> for purposes of background. One of these is "IPFIX Message format", >> which is intended to mean "the encoding and framing of IPFIX Messages >> defined by RFC 5101 as used with the information model defined by RFC >> 5102", or less formally, "IPFIX off the wire", _not_ the format of an >> individual message. Your comments indicate some confusion here, and I >> would appeal to the working group to suggest an alternative if this >> is a >> problem. > > RFC 5101 section 3 defines the IPFIX Message format, which is the > format > of a single IPFIX Message. Indeed, as a section header, but not as a point of terminology. > Do you propose to use "IPFIX Message format" in a different way? No. But we need a way, in the flow of this document, to describe "IPFIX without transport". "IPFIX Message format" appears to have caused some confusion. Does anyone have a clearer suggestion? >> The issues regarding the interchangeability of "IPFIX File" and >> "IPFIX >> Transport Session" given the fact that Sessions may span Files have >> been >> fixed; many thanks for pointing this one out. V9-related >> terminology has >> been fixed, too; I got a bit confused in a couple of places switching >> between V9 and IPFIX. >> >> IV. Technical issues: >> >> Time window resolution finer than seconds: Perhaps. We were trying to >> avoid the explosion of timestamp IEs we have elsewhere in the >> information model. Keep in mind that the Time Window is meant, in >> multi-file environments, to determine whether a file must be >> scanned to >> find information about a specific moment in time. Going below the >> second-level resolution, at least in my experience, is not necessary >> _for this use case_. >> >> Applicability of opaqueOctets: Indeed, one could create an >> enterprise-specific IE and not export ET info about it, then it's >> opaque. But 1. we still need to address the how to encap non-IPFIX >> info >> in an IPFIX stream and 2. don't necessarily want to tell people go >> get a >> PEN to do it. I'm inclined to keep opaqueOctets. > > I still do not see the need nor the utility of opaqueOctets. Think of it as an IPFIX-friendly way to say "skip this embedded data". It has clear semantics (i.e. "Do Not Do Anything With This"), unlike undefined esIEs. Regards, Brian _______________________________________________ IPFIX mailing list IPFIX@ietf.org https://www.ietf.org/mailman/listinfo/ipfix From ipfix-bounces@ietf.org Fri Aug 29 08:01:34 2008 Return-Path: X-Original-To: ipfix-archive@optimus.ietf.org Delivered-To: ietfarch-ipfix-archive@core3.amsl.com Received: from [127.0.0.1] (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 3143D3A6917; Fri, 29 Aug 2008 08:01:34 -0700 (PDT) X-Original-To: ipfix@core3.amsl.com Delivered-To: ipfix@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 9A9D03A69E1 for ; Fri, 29 Aug 2008 08:01:33 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -4.971 X-Spam-Level: X-Spam-Status: No, score=-4.971 tagged_above=-999 required=5 tests=[AWL=1.628, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id NDONM8BahVk7 for ; Fri, 29 Aug 2008 08:01:32 -0700 (PDT) Received: from smtp.ee.ethz.ch (smtp.ee.ethz.ch [129.132.2.219]) by core3.amsl.com (Postfix) with ESMTP id 5A4C13A687C for ; Fri, 29 Aug 2008 08:01:28 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by smtp.ee.ethz.ch (Postfix) with ESMTP id 648BBD9338; Fri, 29 Aug 2008 17:01:30 +0200 (MEST) X-Virus-Scanned: by amavisd-new on smtp.ee.ethz.ch Received: from smtp.ee.ethz.ch ([127.0.0.1]) by localhost (.ee.ethz.ch [127.0.0.1]) (amavisd-new, port 10024) with LMTP id DXpQz8Qy47Up; Fri, 29 Aug 2008 17:01:30 +0200 (MEST) Received: from pb-10072.ethz.ch (pb-10072.ethz.ch [82.130.103.195]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) (Authenticated sender: briant) by smtp.ee.ethz.ch (Postfix) with ESMTP id EF2B2D93B9; Fri, 29 Aug 2008 17:01:29 +0200 (MEST) Message-Id: <907A7623-1FB0-4F6D-8D4A-92F4952867EC@tik.ee.ethz.ch> From: Brian Trammell To: Gerhard Muenz In-Reply-To: <48B80299.1080303@informatik.uni-tuebingen.de> Mime-Version: 1.0 (Apple Message framework v928.1) Date: Fri, 29 Aug 2008 17:01:29 +0200 References: <48A5B5ED.5040107@informatik.uni-tuebingen.de> <9B5C5A4E-2099-43B4-953E-674E38314582@tik.ee.ethz.ch> <48B80299.1080303@informatik.uni-tuebingen.de> X-Mailer: Apple Mail (2.928.1) Cc: IETF IPFIX Working Group Subject: Re: [IPFIX] WG last call on draft-ietf-ipfix-file-02 X-BeenThere: ipfix@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: IPFIX WG discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: ipfix-bounces@ietf.org Errors-To: ipfix-bounces@ietf.org On Aug 29, 2008, at 4:07 PM, Gerhard Muenz wrote: > > Brian, > > Brian Trammell wrote: >> Thanks for your detailed review and comments. I'll address these by >> type. >> >> I. Comments related to the length and level of detail of the >> document: >> >> The IPFIX File draft differs from other IPFIX drafts in that it 1. >> contains its own requirements (analogous to 3917 and 5101 being the >> same >> document) and 2. contains more motivating text than other drafts, >> due to >> its somewhat unusual nature within the IETF (standardizing a file >> format >> without binding it to some specific transport). The authors believe >> that >> the inclusion and length of the requirements and motivation sections >> (sections 4 and 5) is warranted by these considerations. > > I partially agree: motivation and requirements should be included. The > question is if we need that many pages. > > The general concept of the file format is straight forward and easy to > understand. The important aspects can be explained on a couple of > pages. > So, an implementer who is already familiar with IPFIX should not be > urged to read through 50 pages. What he needs to read are sections > 3, 7, > 8. Sections 4 to 6 provide background information which are not > necessary to implement the standard (but rather useful to sell the > standard to customers). Sections 9 and 10 cover special cases. Apart > from reducing verbosity, you could mention in the introduction which > sections contain the important information. This is an excellent point; I agree completely. I'll modify the introduction to add exactly this sort of implementor-friendly overview. >> III. Terminology confusion: >> >> There are a couple of terms invented here in order to describe >> concepts >> 1. for which there is no accepted terminology but 2. which appear >> only >> for purposes of background. One of these is "IPFIX Message format", >> which is intended to mean "the encoding and framing of IPFIX Messages >> defined by RFC 5101 as used with the information model defined by RFC >> 5102", or less formally, "IPFIX off the wire", _not_ the format of an >> individual message. Your comments indicate some confusion here, and I >> would appeal to the working group to suggest an alternative if this >> is a >> problem. > > RFC 5101 section 3 defines the IPFIX Message format, which is the > format > of a single IPFIX Message. Indeed, as a section header, but not as a point of terminology. > Do you propose to use "IPFIX Message format" in a different way? No. But we need a way, in the flow of this document, to describe "IPFIX without transport". "IPFIX Message format" appears to have caused some confusion. Does anyone have a clearer suggestion? >> The issues regarding the interchangeability of "IPFIX File" and >> "IPFIX >> Transport Session" given the fact that Sessions may span Files have >> been >> fixed; many thanks for pointing this one out. V9-related >> terminology has >> been fixed, too; I got a bit confused in a couple of places switching >> between V9 and IPFIX. >> >> IV. Technical issues: >> >> Time window resolution finer than seconds: Perhaps. We were trying to >> avoid the explosion of timestamp IEs we have elsewhere in the >> information model. Keep in mind that the Time Window is meant, in >> multi-file environments, to determine whether a file must be >> scanned to >> find information about a specific moment in time. Going below the >> second-level resolution, at least in my experience, is not necessary >> _for this use case_. >> >> Applicability of opaqueOctets: Indeed, one could create an >> enterprise-specific IE and not export ET info about it, then it's >> opaque. But 1. we still need to address the how to encap non-IPFIX >> info >> in an IPFIX stream and 2. don't necessarily want to tell people go >> get a >> PEN to do it. I'm inclined to keep opaqueOctets. > > I still do not see the need nor the utility of opaqueOctets. Think of it as an IPFIX-friendly way to say "skip this embedded data". It has clear semantics (i.e. "Do Not Do Anything With This"), unlike undefined esIEs. Regards, Brian _______________________________________________ IPFIX mailing list IPFIX@ietf.org https://www.ietf.org/mailman/listinfo/ipfix From ipfix-bounces@ietf.org Fri Aug 29 09:04:29 2008 Return-Path: X-Original-To: ipfix-archive@lists.ietf.org Delivered-To: ietfarch-ipfix-archive@core3.amsl.com Received: from [127.0.0.1] (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 6210C3A6994; Fri, 29 Aug 2008 09:04:29 -0700 (PDT) X-Original-To: ipfix@core3.amsl.com Delivered-To: ipfix@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id D5E203A6A20 for ; Fri, 29 Aug 2008 09:04:27 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.785 X-Spam-Level: X-Spam-Status: No, score=-6.785 tagged_above=-999 required=5 tests=[AWL=1.814, BAYES_00=-2.599, GB_I_LETTER=-2, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id L1A1KZJlRE0G for ; Fri, 29 Aug 2008 09:04:25 -0700 (PDT) Received: from smtp.ee.ethz.ch (smtp.ee.ethz.ch [129.132.2.219]) by core3.amsl.com (Postfix) with ESMTP id 3FA803A677C for ; Fri, 29 Aug 2008 09:04:24 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by smtp.ee.ethz.ch (Postfix) with ESMTP id 4D813D9339 for ; Fri, 29 Aug 2008 18:04:25 +0200 (MEST) X-Virus-Scanned: by amavisd-new on smtp.ee.ethz.ch Received: from smtp.ee.ethz.ch ([127.0.0.1]) by localhost (.ee.ethz.ch [127.0.0.1]) (amavisd-new, port 10024) with LMTP id IopCOvbjY-Xp for ; Fri, 29 Aug 2008 18:04:25 +0200 (MEST) Received: from pb-10072.ethz.ch (pb-10072.ethz.ch [82.130.103.195]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) (Authenticated sender: briant) by smtp.ee.ethz.ch (Postfix) with ESMTP id E1283D9338 for ; Fri, 29 Aug 2008 18:04:24 +0200 (MEST) Message-Id: <3B2C1602-5624-4428-A706-5CFF6C3DDE52@tik.ee.ethz.ch> From: Brian Trammell To: IETF IPFIX Working Group Mime-Version: 1.0 (Apple Message framework v928.1) Date: Fri, 29 Aug 2008 18:04:24 +0200 X-Mailer: Apple Mail (2.928.1) Subject: [IPFIX] SCTP per-stream WGLC review X-BeenThere: ipfix@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: IPFIX WG discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: ipfix-bounces@ietf.org Errors-To: ipfix-bounces@ietf.org All, The following is a review of the SCTP perstream draft for WGLC. Serious issues (repeated inline): It is not clear in the language of the document that it is not intended to update 5101, though its intended status precludes this. Additionally, there are some specifications with respect to Collecting Processes which may break interoperability. The Data Record Reliability Options Template does not appear to be necessary, or at least, its purpose is unclear. As I read it the CP can deduce all the information it provides rather easily. General comments: I have not consistently corrected the grammar in the document where it does not have a significant impact on clarity (e.g. spurious articles), though the document does need a grammar and typographic editing pass between now and final publication. There is one general terminology issue which should be addressed; it appears that throughout the document the phrase "Template ID" is used to mean "number identifying a Template" as well as "Data Records (or Data Sets) described by (or associated with) a Template". I agree this latter construct is really awkward, but "Template ID" is a confusing shorthand for the concept, aFrom ipfix-bounces@ietf.org Fri Aug 29 09:04:29 2008 Return-Path: X-Original-To: ipfix-archive@optimus.ietf.org Delivered-To: ietfarch-ipfix-archive@core3.amsl.com Received: from [127.0.0.1] (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 6210C3A6994; Fri, 29 Aug 2008 09:04:29 -0700 (PDT) X-Original-To: ipfix@core3.amsl.com Delivered-To: ipfix@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id D5E203A6A20 for ; Fri, 29 Aug 2008 09:04:27 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.785 X-Spam-Level: X-Spam-Status: No, score=-6.785 tagged_above=-999 required=5 tests=[AWL=1.814, BAYES_00=-2.599, GB_I_LETTER=-2, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id L1A1KZJlRE0G for ; Fri, 29 Aug 2008 09:04:25 -0700 (PDT) Received: from smtp.ee.ethz.ch (smtp.ee.ethz.ch [129.132.2.219]) by core3.amsl.com (Postfix) with ESMTP id 3FA803A677C for ; Fri, 29 Aug 2008 09:04:24 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by smtp.ee.ethz.ch (Postfix) with ESMTP id 4D813D9339 for ; Fri, 29 Aug 2008 18:04:25 +0200 (MEST) X-Virus-Scanned: by amavisd-new on smtp.ee.ethz.ch Received: from smtp.ee.ethz.ch ([127.0.0.1]) by localhost (.ee.ethz.ch [127.0.0.1]) (amavisd-new, port 10024) with LMTP id IopCOvbjY-Xp for ; Fri, 29 Aug 2008 18:04:25 +0200 (MEST) Received: from pb-10072.ethz.ch (pb-10072.ethz.ch [82.130.103.195]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) (Authenticated sender: briant) by smtp.ee.ethz.ch (Postfix) with ESMTP id E1283D9338 for ; Fri, 29 Aug 2008 18:04:24 +0200 (MEST) Message-Id: <3B2C1602-5624-4428-A706-5CFF6C3DDE52@tik.ee.ethz.ch> From: Brian Trammell To: IETF IPFIX Working Group Mime-Version: 1.0 (Apple Message framework v928.1) Date: Fri, 29 Aug 2008 18:04:24 +0200 X-Mailer: Apple Mail (2.928.1) Subject: [IPFIX] SCTP per-stream WGLC review X-BeenThere: ipfix@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: IPFIX WG discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: ipfix-bounces@ietf.org Errors-To: ipfix-bounces@ietf.org All, The following is a review of the SCTP perstream draft for WGLC. Serious issues (repeated inline): It is not clear in the language of the document that it is not intended to update 5101, though its intended status precludes this. Additionally, there are some specifications with respect to Collecting Processes which may break interoperability. The Data Record Reliability Options Template does not appear to be necessary, or at least, its purpose is unclear. As I read it the CP can deduce all the information it provides rather easily. General comments: I have not consistently corrected the grammar in the document where it does not have a significant impact on clarity (e.g. spurious articles), though the document does need a grammar and typographic editing pass between now and final publication. There is one general terminology issue which should be addressed; it appears that throughout the document the phrase "Template ID" is used to mean "number identifying a Template" as well as "Data Records (or Data Sets) described by (or associated with) a Template". I agree this latter construct is really awkward, but "Template ID" is a confusing shorthand for the concept,nd suggest finding a replacement and using it consistently. > 1. Terminology > > IPFIX-specific terminology used in this document is defined > in section 2 of [RFC5101]. As in [RFC5101], these IPFIX- > specific terms have the first letter of a word capitalized > when used in this document. > > Template Reuse Delay > > The configurable timeout to allow the Collecting Process > to receive and process the last Data Record using this > Template information before which the Template Withdrawal > Message MUST NOT be sent. [RFC5101] specifies a default > value of 5 seconds. The wording here is confusing. I don't know what a Template Reuse Delay is from this text. Is this correct: "The time after sending the last Data Set described by a given Template the Exporting Process MUST wait before sending a Template Withdrawal Message for the Template." It's not a CP delay, or is it? I'm confused as to what configurability at the CP has to do with it. > 2. Introduction > > The IPFIX working group has specified a protocol to export IP > Flow information [RFC5101]. This protocol is designed to > export information about IP traffic Flows and related > measurement data, where a Flow is defined by a set of key > attributes (e.g. source and destination IP address, source > and destination port, etc.). However, thanks to its template > mechanism, the IPFIX protocol can export any type of > information, as long as the relevant Information Element is > specified in the IPFIX Information Model [RFC5102], > registered with IANA, or specified as an enterprise-specific > Information Element. > > The IPFIX protocol [RFC5101] specifies that IP traffic > measurements for Flows are exported using a TLV (type, > length, value) format. The information is exported using a > Template Record that is sent once to export the {type, > length} pairs that define the data format for the Information > Elements in a Flow. The Data Records specify values for each > Flow. > > The IPFIX protocol [RFC5101] is flexible: it foresees the > usage > of the multiple SCTP streams per association; it allows the > transmission of Data Sets, Template Sets, and/or Options > Template Sets on any stream; it offers the full or partial > reliability export of Data Sets; it proposes the ordered or > out- > of-order delivery of Data Sets. However, due to bandwidth > > > Expires January 1, 2009 [Page 5] > > Internet-Draft July > 2008 > > > restrictions and packet losses in the network as well as > resource constraints on the Exporter and Collector (e.g., > limited buffer sizes), it is not always possible to export all > Data Sets in a reliable way. > I'm going to make a few stylistic suggestions here in the introduction, because the introduction, I think, needs to be very clear as to what the document is, and it reads a little awkwardly to me as it stands. > Without delving into the details of the specifications > described > later on in this document, the basic idea is to export the > Template Record and its associated Data Sets into a single > unique SCTP stream, ideally to limit the Template ID to a > single > stream, while imposing in-order transmission. Introductory construct is a little unclear. Suggest replacement of this paragraph with: This document specifies a method for exporting a Template Record and its associated Data Sets in a single SCTP stream, limiting each Template ID to a single stream if possible, and imposing in-order transmission. > The specification in this document offers several advantages > such as: calculation of Data Record losses in case of > parti and suggest finding a replacement and using it consistently. > 1. Terminology > > IPFIX-specific terminology used in this document is defined > in section 2 of [RFC5101]. As in [RFC5101], these IPFIX- > specific terms have the first letter of a word capitalized > when used in this document. > > Template Reuse Delay > > The configurable timeout to allow the Collecting Process > to receive and process the last Data Record using this > Template information before which the Template Withdrawal > Message MUST NOT be sent. [RFC5101] specifies a default > value of 5 seconds. The wording here is confusing. I don't know what a Template Reuse Delay is from this text. Is this correct: "The time after sending the last Data Set described by a given Template the Exporting Process MUST wait before sending a Template Withdrawal Message for the Template." It's not a CP delay, or is it? I'm confused as to what configurability at the CP has to do with it. > 2. Introduction > > The IPFIX working group has specified a protocol to export IP > Flow information [RFC5101]. This protocol is designed to > export information about IP traffic Flows and related > measurement data, where a Flow is defined by a set of key > attributes (e.g. source and destination IP address, source > and destination port, etc.). However, thanks to its template > mechanism, the IPFIX protocol can export any type of > information, as long as the relevant Information Element is > specified in the IPFIX Information Model [RFC5102], > registered with IANA, or specified as an enterprise-specific > Information Element. > > The IPFIX protocol [RFC5101] specifies that IP traffic > measurements for Flows are exported using a TLV (type, > length, value) format. The information is exported using a > Template Record that is sent once to export the {type, > length} pairs that define the data format for the Information > Elements in a Flow. The Data Records specify values for each > Flow. > > The IPFIX protocol [RFC5101] is flexible: it foresees the > usage > of the multiple SCTP streams per association; it allows the > transmission of Data Sets, Template Sets, and/or Options > Template Sets on any stream; it offers the full or partial > reliability export of Data Sets; it proposes the ordered or > out- > of-order delivery of Data Sets. However, due to bandwidth > > > Expires January 1, 2009 [Page 5] > > Internet-Draft July > 2008 > > > restrictions and packet losses in the network as well as > resource constraints on the Exporter and Collector (e.g., > limited buffer sizes), it is not always possible to export all > Data Sets in a reliable way. > I'm going to make a few stylistic suggestions here in the introduction, because the introduction, I think, needs to be very clear as to what the document is, and it reads a little awkwardly to me as it stands. > Without delving into the details of the specifications > described > later on in this document, the basic idea is to export the > Template Record and its associated Data Sets into a single > unique SCTP stream, ideally to limit the Template ID to a > single > stream, while imposing in-order transmission. Introductory construct is a little unclear. Suggest replacement of this paragraph with: This document specifies a method for exporting a Template Record and its associated Data Sets in a single SCTP stream, limiting each Template ID to a single stream if possible, and imposing in-order transmission. > The specification in this document offers several advantages > such as: calculation of Data Record losses in case of > parally- > reliable SCTP export, immediate export of the Template > Withdrawal Message, immediate reuse of template ID within a > stream, reduced likelihood of losing Data Record, and the > Collecting Process's job is easier. Text is not parallel; there are typographic errors here as well. Suggest replacement of this paragraph with: This method offers several advantages over IPFIX export as specified in [RFC5101] such as the ability to calculate Data Record losses for PR-SCTP, immediate export of Template Withdrawal Messages, immediate reuse of Template IDs within an SCTP stream, reduced likelihood of Data Record loss, and reduced demands on the Collecting Process. > 2.1. Relationship with IPFIX and PSAMP > > The specification in this document applies to the IPFIX > protocol specifications [RFC5101]. However, it only applies > to the SCTP transport protocol [RFC4960] option of the IPFIX > protocol specifications, specifically in the case of the > partial reliability extension [RFC3758]. All specifications > from [RFC5101] apply unless specified otherwise in this > document. > > As the Packet Sampling (PSAMP) protocol specifications > [PSAMP-PROTO] are based on the IPFIX protocol specifications, > the specifications in this document are also valid for the > PSAMP protocol. Therefore, the advantages specified by this > document also apply to PSAMP. The document specifies a method, not a set of advantages; suggest "Therefore, the method..." > > 2.2. Applicability > > The specifications are required in cases where we must know > how > many Data Records of a certain type (i.e. from a certain > Template ID) were lost. Furthermore, they apply in cases > where > the Exporter can not afford to export all the Flow Records > reliably, due to the limited resources to buffer the huge > amount > of flow records. Such situations may occur if Data Sets are > generated at a higher rate at the Exporter than can be > transferred to the Collector because of bandwidth > limitations in > > > Expires January 1, 2009 [Page 6] > > Internet-Draft July > 2008 > > > the network or slow reception at the Collector. A typically > example is a router exporting billing records. typical example > > To be more precise, the specification applicability is the > case > where multiple Template IDs are sent within a SCTP Transport > Session and the calculation of the Data Record loss for a > particular one Template ID is required. Indeed, with the > current IPFIX specifications [RFC5101], if an IPFIX Message is > lost (UDP or SCTP partially reliable), it is not possible to > determine to which Template ID of the Transport Session the > lost > Data Records belong to. Not in the general case, no. An EP can of course use only a single template for all export in a given session, or use a separate observation domain for each application. This is not the _only_ way under the current spec to estimate loss, it's just the most SCTP friendly, and the least hackish. > In terms of Collector, there is backwards compatibility: the > Collecting Process does not require any changes to support an > Exporter that complies to the specifications in this document. Suggest: Exporting Processes following this specification will interoperate with existing Collecting Processes that comply with [RFC5101]; no changes are required at the Collecting Process to support this method. > > > 2.3. Limitations > > To be compliant with the specifications in this document, the > Transport Session must support multiple SCTP streams. > Furthermore, if the SCTP Transport Session does not support > enough streams for the incrtially- > reliable SCTP export, immediate export of the Template > Withdrawal Message, immediate reuse of template ID within a > stream, reduced likelihood of losing Data Record, and the > Collecting Process's job is easier. Text is not parallel; there are typographic errors here as well. Suggest replacement of this paragraph with: This method offers several advantages over IPFIX export as specified in [RFC5101] such as the ability to calculate Data Record losses for PR-SCTP, immediate export of Template Withdrawal Messages, immediate reuse of Template IDs within an SCTP stream, reduced likelihood of Data Record loss, and reduced demands on the Collecting Process. > 2.1. Relationship with IPFIX and PSAMP > > The specification in this document applies to the IPFIX > protocol specifications [RFC5101]. However, it only applies > to the SCTP transport protocol [RFC4960] option of the IPFIX > protocol specifications, specifically in the case of the > partial reliability extension [RFC3758]. All specifications > from [RFC5101] apply unless specified otherwise in this > document. > > As the Packet Sampling (PSAMP) protocol specifications > [PSAMP-PROTO] are based on the IPFIX protocol specifications, > the specifications in this document are also valid for the > PSAMP protocol. Therefore, the advantages specified by this > document also apply to PSAMP. The document specifies a method, not a set of advantages; suggest "Therefore, the method..." > > 2.2. Applicability > > The specifications are required in cases where we must know > how > many Data Records of a certain type (i.e. from a certain > Template ID) were lost. Furthermore, they apply in cases > where > the Exporter can not afford to export all the Flow Records > reliably, due to the limited resources to buffer the huge > amount > of flow records. Such situations may occur if Data Sets are > generated at a higher rate at the Exporter than can be > transferred to the Collector because of bandwidth > limitations in > > > Expires January 1, 2009 [Page 6] > > Internet-Draft July > 2008 > > > the network or slow reception at the Collector. A typically > example is a router exporting billing records. typical example > > To be more precise, the specification applicability is the > case > where multiple Template IDs are sent within a SCTP Transport > Session and the calculation of the Data Record loss for a > particular one Template ID is required. Indeed, with the > current IPFIX specifications [RFC5101], if an IPFIX Message is > lost (UDP or SCTP partially reliable), it is not possible to > determine to which Template ID of the Transport Session the > lost > Data Records belong to. Not in the general case, no. An EP can of course use only a single template for all export in a given session, or use a separate observation domain for each application. This is not the _only_ way under the current spec to estimate loss, it's just the most SCTP friendly, and the least hackish. > In terms of Collector, there is backwards compatibility: the > Collecting Process does not require any changes to support an > Exporter that complies to the specifications in this document. Suggest: Exporting Processes following this specification will interoperate with existing Collecting Processes that comply with [RFC5101]; no changes are required at the Collecting Process to support this method. > > > 2.3. Limitations > > To be compliant with the specifications in this document, the > Transport Session must support multiple SCTP streams. > Furthermore, if the SCTP Transport Session does not support > enough streams for the ineasing number of Template ID in the > Transport Session, the addition of streams must be supported > according to [SCTP-RESET]. Alternatively, the new Template ID > and associated Data Records may be added to an existing stream > at the cost of diluting the granularity of Data Records loss. > The other alternatives, which is not practical in operational > networks, is to restart the SCTP association with an increase > number of streams. A few unclear and nonstandard constructs here, and a little terminology confusion. The discussion of alternatives to one-session- per-stream and SCTP reset is not a limitation of the approach, it's a workaround to limitations in certain SCTP stacks, and is a bit detailed for an introduction I think. Suggest replacement: This method requires multiple SCTP streams in the association between the Exporting and Collecting Process, ideally one per Template. The SCTP association should support the addition of streams according to [SCTP-RESET] in order to handle the transmission of additional Templates during the Transport Session. > 3. IPFIX Protocol Specifications Limitations and Improvements I think this section needs some frontmatter explaining its organization. From the point of view of the TOC it makes sense, but reading the document through end to end it's a little disorienting. > > 3.2.2. IPFIX Export per SCTP Stream Advantages > > By exporting each Template Record and the corresponding Data > Records within a single stream and imposing in-order > transmission, the Template will always arrive before the > associated Data Records. Therefore, there is no risk that > the Collecting Process discards Data Records while waiting > for the Template to arrive. > > Furthermore, when reusing a Template ID within a stream, the > Template Withdrawal Message will be guaranteed to arrive > before the new definition of the Template and therefore the > Template Record may be sent directly after the Template > Withdrawal Message. In other words, the Template Reuse Delay > restriction (by default, 5 seconds, as specified in [RFC5101] > is removed for Template ID reuse within the same stream. > > Another advantage with the new specifications in this > document is that the Collecting Process's job is now easier. > Indeed, the Collecting Process doesn't have to store the Data > Records while waiting for the Template Records, as the > transmission order is always guaranteed. This way, extra > reliability of the Data Records is achieved without extra > burden on the Collecting Process. Prefer "reduces load on the CP" to "the CP's job is now easier"; the latter reads quite informally for a specification. It might be nice if there were some way for a full-featured CP (i.e. one that supports buffering of Data Sets with missing Templates) to know whether it had to preallocate buffers for such a thing, but this is a very minor technical point. > > 3.3.2. IPFIX Export per SCTP Stream Advantages > > By exporting each Template Record and the corresponding Data > Records within a single stream, imposing in-order > transmission, and limiting the Template ID to a single > stream, the issue of ordered transmission across multiple > streams is avoided. > > By exporting all corresponding Data Records within the same > ordered stream as the Template Definition, each stream is > independent and self-contained and the interaction between > streams is limited to that of Options Data interactions. This > has several advantageous consequences, including the order > preservation that does not result in the blocking of unrelated > data and the Collector's job simplification (as the Template > Records are guaranteed to be delivered before the associated > Dacreasing number of Template ID in the > Transport Session, the addition of streams must be supported > according to [SCTP-RESET]. Alternatively, the new Template ID > and associated Data Records may be added to an existing stream > at the cost of diluting the granularity of Data Records loss. > The other alternatives, which is not practical in operational > networks, is to restart the SCTP association with an increase > number of streams. A few unclear and nonstandard constructs here, and a little terminology confusion. The discussion of alternatives to one-session- per-stream and SCTP reset is not a limitation of the approach, it's a workaround to limitations in certain SCTP stacks, and is a bit detailed for an introduction I think. Suggest replacement: This method requires multiple SCTP streams in the association between the Exporting and Collecting Process, ideally one per Template. The SCTP association should support the addition of streams according to [SCTP-RESET] in order to handle the transmission of additional Templates during the Transport Session. > 3. IPFIX Protocol Specifications Limitations and Improvements I think this section needs some frontmatter explaining its organization. From the point of view of the TOC it makes sense, but reading the document through end to end it's a little disorienting. > > 3.2.2. IPFIX Export per SCTP Stream Advantages > > By exporting each Template Record and the corresponding Data > Records within a single stream and imposing in-order > transmission, the Template will always arrive before the > associated Data Records. Therefore, there is no risk that > the Collecting Process discards Data Records while waiting > for the Template to arrive. > > Furthermore, when reusing a Template ID within a stream, the > Template Withdrawal Message will be guaranteed to arrive > before the new definition of the Template and therefore the > Template Record may be sent directly after the Template > Withdrawal Message. In other words, the Template Reuse Delay > restriction (by default, 5 seconds, as specified in [RFC5101] > is removed for Template ID reuse within the same stream. > > Another advantage with the new specifications in this > document is that the Collecting Process's job is now easier. > Indeed, the Collecting Process doesn't have to store the Data > Records while waiting for the Template Records, as the > transmission order is always guaranteed. This way, extra > reliability of the Data Records is achieved without extra > burden on the Collecting Process. Prefer "reduces load on the CP" to "the CP's job is now easier"; the latter reads quite informally for a specification. It might be nice if there were some way for a full-featured CP (i.e. one that supports buffering of Data Sets with missing Templates) to know whether it had to preallocate buffers for such a thing, but this is a very minor technical point. > > 3.3.2. IPFIX Export per SCTP Stream Advantages > > By exporting each Template Record and the corresponding Data > Records within a single stream, imposing in-order > transmission, and limiting the Template ID to a single > stream, the issue of ordered transmission across multiple > streams is avoided. > > By exporting all corresponding Data Records within the same > ordered stream as the Template Definition, each stream is > independent and self-contained and the interaction between > streams is limited to that of Options Data interactions. This > has several advantageous consequences, including the order > preservation that does not result in the blocking of unrelated > data and the Collector's job simplification (as the Template > Records are guaranteed to be delivered before the associated > ta Records). see previous comments about the "collector's job" language. > > > 4. Specifications > > 4.1. Template Management > > This section introduces modifications compared to the Template > Management section 8 in [RFC5101]. > > > Expires January 1, 2009 [Page 10] > > Internet-Draft July > 2008 > > > > As specified in [RFC5101], Template Sets and Options Template > Sets MUST be sent reliably. In other words, any IPFIX Message > containing an (Options) Template Set MUST be sent reliably. Verify that all 2119 keywords drawn from 5101 actually do restate the point properly; as I recall there was a bit of trouble over this with the implementation guidelines. > > Any Data Sets associated with a Template Record MUST be sent > on > the same stream on which the Template Record was sent. The use of 2119 MUST language here implies a mandatory modification to 5101. I know this is not the case given the document's intended status, but perhaps there should be some frontmatter in section four scoping the 2119 keywords within this section to implementations of the specification only. > > The Exporter SHOULD send a single Template and associated Data > Sets within a single stream in order to calculate the > potential > Data Record loss for this Template ID. in order to _enable_ calculation (it's the collector doing the loss calculation, right, not the exporter?) > However, the Exporter > MAY group related Templates and their associated Data Sets > within a single stream so loss statistics are calculated for > the > group. This may be suitable in cases where there is > insufficient SCTP streams to send each Template on its own > stream and/or the case where there are slight variations on a > single Template to show that some fields were unavailable at > the > time of monitoring. This last sentence is kind of confusing. What I think you mean is: This is suitable in cases where there are only slight variations among the templates in a group, e.g. the omission of unavailable fields for export efficiency, and may be necessary if the SCTP association does not support enough streams to export each Template in its own stream. Hmm... Actually, I think the specification would be greatly improved by the formal introduction of a "template group" concept, and the rephrasing of the specification to cover the export of one template _group_ per stream. Since the omission of missing fields is not only common practice but recommended by the documents, you're often likely to get a set of related templates. Exporting these as a group is I think what you're really trying to do with this document, since it's application-granularity loss you're trying to estimate, yes? > If a SCTP stream contains a mixture of Data Records defined > by a > Template Record and Options Template Record(s), the Data > Records > defined by the Options Template Record(s) SHOULD be sent > reliably within the same stream so that the Collector does not > consider any loss to be associated with the Options Data. > Indeed, if the Collector does not have the guarantee that the > Data Records defined by the Options Template Record are sent > reliably, the Collector can not determine whether the loss in > that stream belongs to the Data Records defined by the > Template > Record, defined by the Option Template Record, or by both of > them. By sending the Options Data reliably (which is usually > required to interpret the Data Records correctly), any loss > will > be limited to the non-option Data Record and loss can still be > calculated on a per Template basis. > > For each (Options) Template Record, the Exporting Process MUST > send the Data Record Reliability OptiData Records). see previous comments about the "collector's job" language. > > > 4. Specifications > > 4.1. Template Management > > This section introduces modifications compared to the Template > Management section 8 in [RFC5101]. > > > Expires January 1, 2009 [Page 10] > > Internet-Draft July > 2008 > > > > As specified in [RFC5101], Template Sets and Options Template > Sets MUST be sent reliably. In other words, any IPFIX Message > containing an (Options) Template Set MUST be sent reliably. Verify that all 2119 keywords drawn from 5101 actually do restate the point properly; as I recall there was a bit of trouble over this with the implementation guidelines. > > Any Data Sets associated with a Template Record MUST be sent > on > the same stream on which the Template Record was sent. The use of 2119 MUST language here implies a mandatory modification to 5101. I know this is not the case given the document's intended status, but perhaps there should be some frontmatter in section four scoping the 2119 keywords within this section to implementations of the specification only. > > The Exporter SHOULD send a single Template and associated Data > Sets within a single stream in order to calculate the > potential > Data Record loss for this Template ID. in order to _enable_ calculation (it's the collector doing the loss calculation, right, not the exporter?) > However, the Exporter > MAY group related Templates and their associated Data Sets > within a single stream so loss statistics are calculated for > the > group. This may be suitable in cases where there is > insufficient SCTP streams to send each Template on its own > stream and/or the case where there are slight variations on a > single Template to show that some fields were unavailable at > the > time of monitoring. This last sentence is kind of confusing. What I think you mean is: This is suitable in cases where there are only slight variations among the templates in a group, e.g. the omission of unavailable fields for export efficiency, and may be necessary if the SCTP association does not support enough streams to export each Template in its own stream. Hmm... Actually, I think the specification would be greatly improved by the formal introduction of a "template group" concept, and the rephrasing of the specification to cover the export of one template _group_ per stream. Since the omission of missing fields is not only common practice but recommended by the documents, you're often likely to get a set of related templates. Exporting these as a group is I think what you're really trying to do with this document, since it's application-granularity loss you're trying to estimate, yes? > If a SCTP stream contains a mixture of Data Records defined > by a > Template Record and Options Template Record(s), the Data > Records > defined by the Options Template Record(s) SHOULD be sent > reliably within the same stream so that the Collector does not > consider any loss to be associated with the Options Data. > Indeed, if the Collector does not have the guarantee that the > Data Records defined by the Options Template Record are sent > reliably, the Collector can not determine whether the loss in > that stream belongs to the Data Records defined by the > Template > Record, defined by the Option Template Record, or by both of > them. By sending the Options Data reliably (which is usually > required to interpret the Data Records correctly), any loss > will > be limited to the non-option Data Record and loss can still be > calculated on a per Template basis. > > For each (Options) Template Record, the Exporting Process MUST > send the Data Record Reliability Option Template using an > Option > Template with the following Information Elements: > > SCOPE: Template ID > NOT-SCOPE: dataRecordsReliability > > The Data Record Reliability Option Template MUST be sent > reliably. > It's a template, so it must be sent reliably. This is redundant. Or do you mean the record described by this template? More seriously, I don't see the point of this Option. Can't the CP just deduce from the EP's actions whether it's sending the data for a given template reliably or not? Also, what additional functionality does the CP really gain by deducing this? Doesn't all the stream loss estimation stuff still work (well, on a per-group basis, anyway)? > > Expires January 1, 2009 [Page 11] > > Internet-Draft July > 2008 > > > The Option Data Record SHOULD be sent before the Data Record > that needs it so that it arrives first and is available for > the > Collector to use. And I presume it should also be sent reliably? > > > 4.2. New Information Element > > dataRecordsReliability > > Description: > The Data Records reliability associated with this > Template ID. The integer value 1 means that the Data > Records are sent reliably, while the integer value 2 > means that the Data Records are not sent reliably. > Abstract Data Type: boolean > Data Type Semantics: identifier > ElementId: xxx > Status: current Inconsistent. Is it an integer, or a boolean? > > > 4.3. SCTP > > This section introduces modifications compared to the "SCTP" > section 10.2 (and subsections) in [RFC5101]. More > specifically > the "Stream" section 10.2.4.3 See above about 2119 language and intended status. This frontmatter clearly reads as if it supercedes or updates 5101, which is not the case. > > PR-SCTP [RFC3758] MUST be implemented by all compliant > implementations. > > All IPFIX Messages MUST be sent in order within a stream. > > Depending on the application requirement, the Exporting > Process > MAY send Data Sets with full or partial reliability. > Unreliable > data transfer MAY be used where the application does not > require > reliable transmission or the use of a retransmission queue is > impractical due to resource restrictions at the Exporter. > > > If the Exporting Process requires to export a new Template but > there are no more free SCTP streams available, it SHOULD > attempt > to increase the number of outbound streams it is able to send > to, per [SCTP-RESET]. Alternatively, the Exporting Process > MAY > add the Template Set and Data Records to an existing stream at > the cost of diluting the granularity of Data Records loss. > > > > > > Expires January 1, 2009 [Page 12] > > Internet-Draft July > 2008 > > > 4.4. Template Withdrawal Message > > This section introduces Template Withdrawal Message-related > modifications compared to the Template Management section 8 in > [RFC5101]. As above. > > Templates that are not used anymore SHOULD be deleted. Before > reusing a Template ID, the Template MUST be deleted. In order > to delete an allocated Template, the Template is withdrawn > through the use of a Template Withdrawal Message. The > Template > Withdrawal Message MUST be sent on the same stream as the > Template Record. Inconsistent terminology. Template deletion == withdrawal? > > As the Template Withdrawal Message MUST be sent reliably, > using > SCTP-ordered delivery per [RFC5101], and as all IPFIX Messages > are sent in order within a stream (per the specon Template using an > Option > Template with the following Information Elements: > > SCOPE: Template ID > NOT-SCOPE: dataRecordsReliability > > The Data Record Reliability Option Template MUST be sent > reliably. > It's a template, so it must be sent reliably. This is redundant. Or do you mean the record described by this template? More seriously, I don't see the point of this Option. Can't the CP just deduce from the EP's actions whether it's sending the data for a given template reliably or not? Also, what additional functionality does the CP really gain by deducing this? Doesn't all the stream loss estimation stuff still work (well, on a per-group basis, anyway)? > > Expires January 1, 2009 [Page 11] > > Internet-Draft July > 2008 > > > The Option Data Record SHOULD be sent before the Data Record > that needs it so that it arrives first and is available for > the > Collector to use. And I presume it should also be sent reliably? > > > 4.2. New Information Element > > dataRecordsReliability > > Description: > The Data Records reliability associated with this > Template ID. The integer value 1 means that the Data > Records are sent reliably, while the integer value 2 > means that the Data Records are not sent reliably. > Abstract Data Type: boolean > Data Type Semantics: identifier > ElementId: xxx > Status: current Inconsistent. Is it an integer, or a boolean? > > > 4.3. SCTP > > This section introduces modifications compared to the "SCTP" > section 10.2 (and subsections) in [RFC5101]. More > specifically > the "Stream" section 10.2.4.3 See above about 2119 language and intended status. This frontmatter clearly reads as if it supercedes or updates 5101, which is not the case. > > PR-SCTP [RFC3758] MUST be implemented by all compliant > implementations. > > All IPFIX Messages MUST be sent in order within a stream. > > Depending on the application requirement, the Exporting > Process > MAY send Data Sets with full or partial reliability. > Unreliable > data transfer MAY be used where the application does not > require > reliable transmission or the use of a retransmission queue is > impractical due to resource restrictions at the Exporter. > > > If the Exporting Process requires to export a new Template but > there are no more free SCTP streams available, it SHOULD > attempt > to increase the number of outbound streams it is able to send > to, per [SCTP-RESET]. Alternatively, the Exporting Process > MAY > add the Template Set and Data Records to an existing stream at > the cost of diluting the granularity of Data Records loss. > > > > > > Expires January 1, 2009 [Page 12] > > Internet-Draft July > 2008 > > > 4.4. Template Withdrawal Message > > This section introduces Template Withdrawal Message-related > modifications compared to the Template Management section 8 in > [RFC5101]. As above. > > Templates that are not used anymore SHOULD be deleted. Before > reusing a Template ID, the Template MUST be deleted. In order > to delete an allocated Template, the Template is withdrawn > through the use of a Template Withdrawal Message. The > Template > Withdrawal Message MUST be sent on the same stream as the > Template Record. Inconsistent terminology. Template deletion == withdrawal? > > As the Template Withdrawal Message MUST be sent reliably, > using > SCTP-ordered delivery per [RFC5101], and as all IPFIX Messages > are sent in order within a stream (per the specififications in > this document), the IPFIX Message containing the Template > Withdrawal Message will not arrive at the Collecting Process > before any associated and previously sent Data Record. As a > consequence, no Data Records will be lost due to delayed > arrival > at the Collector. > > The Template ID from a withdrawn Template MAY be reused on the > same stream immediately after the Template Withdrawal > Message is > sent. This case is equivalent to the use of a Template Reuse > Delay value of 0. > > If the new definition of the Template ID is to be reused on a > different stream, the Template Withdrawal Message MUST NOT be > sent before the Template Reuse Delay. > > A Template Withdrawal Message to withdraw all Templates for > the > Observation Domain ID specified in the IPFIX Message header > MUST > NOT be used. > > Multiple Template IDs MAY be withdrawn with a single Template > Withdrawal Message at the condition that all the Template > IDs in > the Template Withdrawal Message are used on the same SCTP > stream. > > > 4.5. The Collecting Process's Side > > This section introduces modifications to the Collection > Process > as compared to section 9 in [RFC5101]. Modification of Collecting Process behavior is in conflict with the statement in the introduction that EPs using this spec interoperate with 5101 CPs. Which is it? > > > Expires January 1, 2009 [Page 13] > > Internet-Draft July > 2008 > > > The Collecting Process SHOULD listen for a new association > request from the Exporting Process. The Exporting Process > will > request a number of streams to use for export: the number of > streams SHOULD be equivalent to the number of simultaneous > Template Records used in the association. Note that the > Template > Records don't include the Options Template Records. This statement does not modify 5101 CP behavior and is superfluous. > > A Collecting Process SHOULD support the procedure for the > addition of an SCTP stream [SCTP-RESET]. > > The IPFIX protocol has a Sequence Number field in the IPFIX > Message header that increases with the number of IPFIX Data > Records in the IPFIX Message. A Collector may detect out-of- > sequence, dropped, or duplicate IPFIX Messages by tracking the > Sequence Number. As this Sequence Number is per SCTP stream, > the loss for the Data Records sent in that stream can be > calculated in case of partially-reliable export. A recommendation for how to do this calculation would be nice here. > > If the Collecting Process receives a Template Withdrawal > Message > on a different stream than the one on which the Template ID is > used, then the Collecting Process MUST reset the association > and > SHOULD log an error message. This is not possible without superceding 5101 and breaking interoperability with stock 5101 EPs. Therefore it must be removed. Furthermore (though I know we do it all over the protocol), CP- initiated Transport Session shutdown does not follow the general protocol design rule of "be conservative in what you send and liberal in what you accept". Adding a new condition where we do this would be less than ideal. But I accept that I lost this fight a while ago. :) > > The following sentences from [RFC5101] are not applicable in > this specification: > > "The Collecting Process normally receives Template Records > from the Exporting Process before receiving Data Records. > The Data Records are then decoded and stored by the > Collector. If the Template Records have not been > received at > the time Data Records are received, the Collecting Process >ications in > this document), the IPFIX Message containing the Template > Withdrawal Message will not arrive at the Collecting Process > before any associated and previously sent Data Record. As a > consequence, no Data Records will be lost due to delayed > arrival > at the Collector. > > The Template ID from a withdrawn Template MAY be reused on the > same stream immediately after the Template Withdrawal > Message is > sent. This case is equivalent to the use of a Template Reuse > Delay value of 0. > > If the new definition of the Template ID is to be reused on a > different stream, the Template Withdrawal Message MUST NOT be > sent before the Template Reuse Delay. > > A Template Withdrawal Message to withdraw all Templates for > the > Observation Domain ID specified in the IPFIX Message header > MUST > NOT be used. > > Multiple Template IDs MAY be withdrawn with a single Template > Withdrawal Message at the condition that all the Template > IDs in > the Template Withdrawal Message are used on the same SCTP > stream. > > > 4.5. The Collecting Process's Side > > This section introduces modifications to the Collection > Process > as compared to section 9 in [RFC5101]. Modification of Collecting Process behavior is in conflict with the statement in the introduction that EPs using this spec interoperate with 5101 CPs. Which is it? > > > Expires January 1, 2009 [Page 13] > > Internet-Draft July > 2008 > > > The Collecting Process SHOULD listen for a new association > request from the Exporting Process. The Exporting Process > will > request a number of streams to use for export: the number of > streams SHOULD be equivalent to the number of simultaneous > Template Records used in the association. Note that the > Template > Records don't include the Options Template Records. This statement does not modify 5101 CP behavior and is superfluous. > > A Collecting Process SHOULD support the procedure for the > addition of an SCTP stream [SCTP-RESET]. > > The IPFIX protocol has a Sequence Number field in the IPFIX > Message header that increases with the number of IPFIX Data > Records in the IPFIX Message. A Collector may detect out-of- > sequence, dropped, or duplicate IPFIX Messages by tracking the > Sequence Number. As this Sequence Number is per SCTP stream, > the loss for the Data Records sent in that stream can be > calculated in case of partially-reliable export. A recommendation for how to do this calculation would be nice here. > > If the Collecting Process receives a Template Withdrawal > Message > on a different stream than the one on which the Template ID is > used, then the Collecting Process MUST reset the association > and > SHOULD log an error message. This is not possible without superceding 5101 and breaking interoperability with stock 5101 EPs. Therefore it must be removed. Furthermore (though I know we do it all over the protocol), CP- initiated Transport Session shutdown does not follow the general protocol design rule of "be conservative in what you send and liberal in what you accept". Adding a new condition where we do this would be less than ideal. But I accept that I lost this fight a while ago. :) > > The following sentences from [RFC5101] are not applicable in > this specification: > > "The Collecting Process normally receives Template Records > from the Exporting Process before receiving Data Records. > The Data Records are then decoded and stored by the > Collector. If the Template Records have not been > received at > the time Data Records are received, the Collecting Process > MAY store the Data Records for a short period of time and > decode them after the Template Records are received." No need to remove this. CPs may still support buffering of template- less Data even if the EP doesn't require them to. Regards, Brian _______________________________________________ IPFIX mailing list IPFIX@ietf.org https://www.ietf.org/mailman/listinfo/ipfix MAY store the Data Records for a short period of time and > decode them after the Template Records are received." No need to remove this. CPs may still support buffering of template- less Data even if the EP doesn't require them to. Regards, Brian _______________________________________________ IPFIX mailing list IPFIX@ietf.org https://www.ietf.org/mailman/listinfo/ipfix From ipfix-arch@cisco.com Sun Aug 31 14:15:15 2008 Return-Path: X-Original-To: ietfarch-ipfix-archive@core3.amsl.com Delivered-To: ietfarch-ipfix-archive@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 931A73A683D for ; Sun, 31 Aug 2008 14:15:15 -0700 (PDT) X-Quarantine-ID: X-Virus-Scanned: amavisd-new at amsl.com X-Amavis-Alert: BAD HEADER, Non-encoded 8-bit data (char AE hex): From: VIAGRA \256 Official Site [...] X-Spam-Flag: NO X-Spam-Score: -48.082 X-Spam-Level: X-Spam-Status: No, score=-48.082 tagged_above=-999 required=5 tests=[BAYES_50=0.001, FH_RELAY_NODNS=1.451, HS_INDEX_PARAM=0.001, MIME_8BIT_HEADER=0.3, RAZOR2_CF_RANGE_51_100=0.5, RAZOR2_CF_RANGE_E8_51_100=1.5, RAZOR2_CHECK=0.5, RCVD_IN_BL_SPAMCOP_NET=1.96, RCVD_IN_PBL=0.905, RCVD_IN_XBL=3.033, RDNS_NONE=0.1, SARE_FROM_DRUGS=1.666, UNPARSEABLE_RELAY=0.001, URIBL_AB_SURBL=10, URIBL_BLACK=20, URIBL_JP_SURBL=10, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id e0jNSRDvjctA for ; Sun, 31 Aug 2008 14:15:14 -0700 (PDT) Received: from fatma-5edc5502f (unknown [88.227.51.79]) by core3.amsl.com (Postfix) with SMTP id 898033A683C for ; Sun, 31 Aug 2008 14:15:12 -0700 (PDT) Received: from Ginger Whitman (10.11.14.14) by fatma-5edc5502f (PowerMTA(TM) v3.2r4) id hfp75o25d78j36 for ; Mon, 1 Sep 2008 12:15:17 +0200 (envelope-from ) Message-ID: <20080901141517.5815.qmail@fatma-5edc5502f> To: Subject: RE: SALE 79%% OFF on Pfizer From: VIAGRA ® Official Site MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit Date: Sun, 31 Aug 2008 14:15:12 -0700 (PDT) Dear ipfix-archive@ietf.org, Best Price Only Today. http://kxw.wandburn.com?epi