From majordomo@mil.doit.wisc.edu Tue Jan 1 13:06:07 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA14178 for ; Tue, 1 Jan 2002 13:06:07 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16LSxc-0006xR-00 for ipfix-list@mil.doit.wisc.edu; Tue, 01 Jan 2002 11:44:24 -0600 Received: from ruby.he.net ([216.218.187.2]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 16LSxa-0006xJ-00 for ipfix@net.doit.wisc.edu; Tue, 01 Jan 2002 11:44:22 -0600 Received: from eif.net ([212.161.14.187] (may be forged)) by ruby.he.net (8.8.6/8.8.2) with SMTP id JAA06640; Tue, 1 Jan 2002 09:31:10 -0800 Message-Id: <200201011731.JAA06640@ruby.he.net> From: "HAPPY NEW YEAR FROM EIF" To: Subject: [ipfix] NEW YEAR EIF OFFER + CHASE OFFER Mime-Version: 1.0 Content-Type: text/html; charset="ISO-8859-1" Date: Tue, 1 Jan 2002 17:27:27 -0800 X-Priority: 1 (Highest) Content-Transfer-Encoding: 8bit Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 8bit Eif Security Solutions and Rapid Traffic Search Optimization

WISHING YOU ALL A VERY HAPPY AND PROSPEROUS NEW YEAR!

FREE PC FIREWALL AND ANTIVIRUS TO ALL THE HUMAN BEINGS CONTACTING US!

THANKS

I TAKE THIS OPPORTUNITY TO TAKE YOU THE MESSAGE FOR THE END OF THE YEAR OF THE PRESIDENT:

'THE ALMOST BEST FUTURE CAN BE MADE BETTER' G CRASTI PRESIDENT HYKSOS GROUP

Rob Tel + 39 32 00 25 80 44

Fax + 1 212 656 1546

Rob Photo

www.eif.net

Apply Now for the Chase Platinum Credit Card iCard Holiday Rewards_468 Shop Safely_468X60 Outtatown 468x60 Platinum Lollipops 468x60

NOTE: If you have asked to be removed from our mailing list, and are continuing to receive our emails, please send us the names of any older or alias email addresses. These sometimes are forwarded to the new mail address and we must delete these older or alias addresses from our list in order to stop mail from reaching your current address. We apologize for any inconvenience and appreciate your continued patience and cooperation.Your address is Opt in under G law.

If you require I will send you $ 6 + apologies having your address.

remove@eif.net -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Tue Jan 1 13:14:58 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA14302 for ; Tue, 1 Jan 2002 13:14:58 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16LT9K-0007E6-00 for ipfix-list@mil.doit.wisc.edu; Tue, 01 Jan 2002 11:56:31 -0600 Received: from rip.psg.com ([147.28.0.39]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 16LT9I-0007Dc-00 for ipfix@net.doit.wisc.edu; Tue, 01 Jan 2002 11:56:28 -0600 Received: from randy by rip.psg.com with local (Exim 3.33 #1) id 16LT8n-000Lnz-00 for ipfix@net.doit.wisc.edu; Tue, 01 Jan 2002 09:55:57 -0800 From: Randy Bush MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit To: ipfix@net.doit.wisc.edu Subject: [ipfix] spam Message-Id: Date: Tue, 01 Jan 2002 09:55:57 -0800 Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit sure would be nice if this list was subscriber-only -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Wed Jan 2 19:47:37 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id TAA13425 for ; Wed, 2 Jan 2002 19:47:37 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16Lvhw-0005wT-00 for ipfix-list@mil.doit.wisc.edu; Wed, 02 Jan 2002 18:26:08 -0600 Received: from dplonka by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16Lvhu-0005wM-00; Wed, 02 Jan 2002 18:26:06 -0600 Date: Wed, 2 Jan 2002 18:26:06 -0600 From: Dave Plonka To: ipfix@net.doit.wisc.edu Cc: Randy Bush Subject: list administrivia, Re: [ipfix] spam, etc. Message-ID: <20020102182606.A20656@doit.wisc.edu> Reply-To: ipfix@net.doit.wisc.edu References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 1.0.1i In-Reply-To: ; from randy@psg.com on Tue, Jan 01, 2002 at 09:55:57AM -0800 Organization: UW-Madison, DoIT, Network Services X-VMS-Error: %SYSTEM-W-QFACTIVE, disk quota file is already active X-Shakespearean-Insult: Thou puking motley-minded puttock Precedence: bulk Sender: majordomo listserver Two things: 1) On Tue, Jan 01, 2002 at 09:55:57AM -0800, Randy Bush wrote: > sure would be nice if this list was subscriber-only Yes, spam is rearing its ugly head here... I can try that. Some subscribers may get their messages delayed if they are not sending from the recipient address with which they subscribed. I'll test it out and send an announcement to the list before I change the list policy. In the WG chairs training sessions, it was suggested that if we do make it "subscriber-only", that rejected (non subscriber) postings should bounce to the admin (in this case me) which should then approve it if the content is pertinent. Apparently the IESG, etc. likes to be able to post to WG lists without having to endure the rigmarole of joining. 2) (to all list members) I haven't forgotten about the new lists for the architecture and the data model drafts that we mentioned at the last WG meeting in Salt Lake. I'm thinking that we'd be better off with keeping all the content in the same list (since it'll be really quiet here otherwise). Perhaps I'll make seperate entry point email addresses, so that if you email to "ipfix-arch" or "ipfix-data" (rather than just "ipfix") your message subject will be tagged appropriately, e.g. "Subject: [ipfix-arch] foobar" and "Subject: [ipfix-data] foobar", which is compatible with filtering, if need be. Let me know if that's objectionable, however, if so, please propose something better than 3 seperate mailing lists. ;^) Thanks, Dave -- plonka@doit.wisc.edu http://net.doit.wisc.edu/~plonka ARS:N9HZF Madison, WI -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Thu Jan 3 00:47:13 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id AAA18146 for ; Thu, 3 Jan 2002 00:47:13 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16M0Lf-0003u0-00 for ipfix-list@mil.doit.wisc.edu; Wed, 02 Jan 2002 23:23:27 -0600 Received: from c001-h001.c001.snv.cp.net ([209.228.32.115] helo=c001.snv.cp.net) by mil.doit.wisc.edu with smtp (Exim 3.13 #1) id 16M0Le-0003tc-00 for ipfix@net.doit.wisc.edu; Wed, 02 Jan 2002 23:23:26 -0600 Received: (cpmta 23661 invoked from network); 2 Jan 2002 21:22:53 -0800 Received: from 24.221.253.53 (HELO slc252750) by smtp.register-admin.com (209.228.32.115) with SMTP; 2 Jan 2002 21:22:53 -0800 X-Sent: 3 Jan 2002 05:22:53 GMT Message-ID: <001101c19416$f3e2c140$850f880a@slc252750> Reply-To: "K.C. Norseth" From: "K.C. Norseth" To: References: <20020102182606.A20656@doit.wisc.edu> Subject: Re: list administrivia, Re: [ipfix] spam, etc. Date: Wed, 2 Jan 2002 22:24:42 -0700 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.00.2615.200 X-MimeOLE: Produced By Microsoft MimeOLE V5.00.2615.200 Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Hi Dave, I hope your holidays were great. ----- Original Message ----- From: Dave Plonka To: Cc: Randy Bush Sent: Wednesday, January 02, 2002 5:26 PM Subject: list administrivia, Re: [ipfix] spam, etc. > Two things: > > 1) On Tue, Jan 01, 2002 at 09:55:57AM -0800, Randy Bush wrote: > > sure would be nice if this list was subscriber-only > > Yes, spam is rearing its ugly head here... > > I can try that. Some subscribers may get their messages delayed if > they are not sending from the recipient address with which they > subscribed. I'll test it out and send an announcement to the list > before I change the list policy. > > In the WG chairs training sessions, it was suggested that if we do > make it "subscriber-only", that rejected (non subscriber) postings > should bounce to the admin (in this case me) which should then > approve it if the content is pertinent. Apparently the IESG, etc. > likes to be able to post to WG lists without having to endure the > rigmarole of joining. Lets stop SPAM! Even the IETF lists are now regulated. About time. I've gotten porn on some of my wg mailing lists. I'll watch Monty Python if I want to see spam. If the IESG doesn't want to subscribe but wants to send an email (they do it very seldom), it can go through the moderator (slight delay) and they can be added to the list who can email. > 2) (to all list members) I haven't forgotten about the new lists for > the architecture and the data model drafts that we mentioned at the > last WG meeting in Salt Lake. > > I'm thinking that we'd be better off with keeping all the content in > the same list (since it'll be really quiet here otherwise). Perhaps > I'll make seperate entry point email addresses, so that if you email > to "ipfix-arch" or "ipfix-data" (rather than just "ipfix") your > message subject will be tagged appropriately, e.g. "Subject: > [ipfix-arch] foobar" and "Subject: [ipfix-data] foobar", which is > compatible with filtering, if need be. > > Let me know if that's objectionable, however, if so, please propose > something better than 3 seperate mailing lists. ;^) There is nothing wrong with 3 mailing lists. The 2 sublists are temporary. It like forming a committee, we won't get things started unless we can get a small group to start the focus. Lets form the groups, get them done and then we can shut down as soon as the first cut of the documents are done. If you don't an impromptu mail group would probably form of the main editors and then people could be left out by accident. Lets' do start the groups quickly, so we can get the task at hand done on time. K.C. > > Thanks, > Dave > > -- > plonka@doit.wisc.edu http://net.doit.wisc.edu/~plonka ARS:N9HZF Madison, WI > > -- > Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ > -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Thu Jan 3 00:51:31 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id AAA18174 for ; Thu, 3 Jan 2002 00:51:31 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16M0WQ-00044g-00 for ipfix-list@mil.doit.wisc.edu; Wed, 02 Jan 2002 23:34:34 -0600 Received: from rip.psg.com ([147.28.0.39]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 16M0WO-000442-00 for ipfix@net.doit.wisc.edu; Wed, 02 Jan 2002 23:34:33 -0600 Received: from randy by rip.psg.com with local (Exim 3.33 #1) id 16M0Vo-000Clp-00; Wed, 02 Jan 2002 21:33:56 -0800 From: Randy Bush MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit To: Dave Plonka Cc: ipfix@net.doit.wisc.edu Subject: Re: list administrivia, Re: [ipfix] spam, etc. References: <20020102182606.A20656@doit.wisc.edu> Message-Id: Date: Wed, 02 Jan 2002 21:33:56 -0800 Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit > In the WG chairs training sessions, it was suggested that if we do > make it "subscriber-only", that rejected (non subscriber) postings > should bounce to the admin (in this case me) which should then > approve it if the content is pertinent. Apparently the IESG, etc. > likes to be able to post to WG lists without having to endure the > rigmarole of joining. nope. we're all just bozos on this bus. the resons for that request are: o sometimes folk post from the wrong account - so approve it, or - tell them their error o sometimes folk like internet-drafts the rfc editor, ... send to a wg's list o sometimes a non-reader replies to a cross-post o etc randy -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Thu Jan 3 00:53:45 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id AAA18227 for ; Thu, 3 Jan 2002 00:53:45 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16M0YH-00047u-00 for ipfix-list@mil.doit.wisc.edu; Wed, 02 Jan 2002 23:36:29 -0600 Received: from rip.psg.com ([147.28.0.39]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 16M0YG-000470-00 for ipfix@net.doit.wisc.edu; Wed, 02 Jan 2002 23:36:28 -0600 Received: from randy by rip.psg.com with local (Exim 3.33 #1) id 16M0Xg-000CpD-00; Wed, 02 Jan 2002 21:35:52 -0800 From: Randy Bush MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit To: Dave Plonka Cc: ipfix@net.doit.wisc.edu Subject: Re: list administrivia, Re: [ipfix] spam, etc. References: <20020102182606.A20656@doit.wisc.edu> Message-Id: Date: Wed, 02 Jan 2002 21:35:52 -0800 Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit imiho, this list is not so busy that more are needed, but i am just a bozo on this bus, and this is a matter of taste. but, if there are other lists which members of the wg should be reading, then we need to tell the secretariat so they can be added to the wg's web page so that newcomers don't hear only part of the conversation. randy -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Sun Jan 6 00:57:23 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id AAA12636 for ; Sun, 6 Jan 2002 00:57:22 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16N5vH-0002IY-00 for ipfix-list@mil.doit.wisc.edu; Sat, 05 Jan 2002 23:32:43 -0600 Received: from host3.riverstonenet.com ([63.113.148.3] helo=exc-sc1.yagosys.com) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 16N5vF-0002Hy-00 for ipfix@net.doit.wisc.edu; Sat, 05 Jan 2002 23:32:41 -0600 Received: from riverstonenet.com (134.141.180.88 [134.141.180.88]) by exc-sc1.yagosys.com with SMTP (Microsoft Exchange Internet Mail Service Version 5.5.2653.13) id X7WVWTVG; Fri, 4 Jan 2002 03:42:28 -0800 Message-ID: <3C359529.BB680D93@riverstonenet.com> Date: Fri, 04 Jan 2002 06:42:33 -0500 From: calato@riverstonenet.com X-Mailer: Mozilla 4.75C-CCK-MCD NSCPCD475 [en] (X11; U; SunOS 5.6 sun4u) X-Accept-Language: en MIME-Version: 1.0 To: "ipfix@net.doit.wisc.edu" Subject: Re: list administrivia, Re: [ipfix] spam, etc. References: <20020102182606.A20656@doit.wisc.edu> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit I think one list is fine. -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Sun Jan 6 08:15:17 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id IAA23217 for ; Sun, 6 Jan 2002 08:15:16 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16NCrG-00053n-00 for ipfix-list@mil.doit.wisc.edu; Sun, 06 Jan 2002 06:57:02 -0600 Received: from central.switch.ch ([130.59.4.1]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 16NCrC-00052z-00 for ipfix@net.doit.wisc.edu; Sun, 06 Jan 2002 06:56:59 -0600 Received: from enigma ([130.59.4.36] helo=enigma.switch.ch) by central.switch.ch with esmtp (Exim 3.20 #1) id 16NCqg-0005jp-00; Sun, 06 Jan 2002 13:56:26 +0100 Received: (from leinen@localhost) by enigma.switch.ch (8.10.2+Sun/8.10.2) id g06CuQM02113; Sun, 6 Jan 2002 13:56:26 +0100 (MET) X-Authentication-Warning: enigma.switch.ch: leinen set sender to simon@limmat.switch.ch using -f To: Ganesh Sadasivan Cc: ipfix@net.doit.wisc.edu Subject: Re: [ipfix] IPFIX Requirement presentation References: <20011214150236.A29404@doit.wisc.edu> <3C1BDF61.B0BE8081@cisco.com> X-Face: 1Nk*r=:$IBBb8|TyRB'2WSY6u:BzMO7N)#id#-4_}MsU5?vTI?dez|JiutW4sKBLjp.l7,F 7QOld^hORRtpCUj)!cP]gtK_SyK5FW(+o"!or:v^C^]OxX^3+IPd\z,@ttmwYVO7l`6OXXYR` From: Simon Leinen In-Reply-To: <3C1BDF61.B0BE8081@cisco.com> Date: 06 Jan 2002 13:56:25 +0100 Message-ID: Lines: 57 User-Agent: Gnus/5.09 (Gnus v5.9.0) Emacs/21.1 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Precedence: bulk Sender: majordomo listserver >>>>> "gs" == Ganesh Sadasivan writes: > I want to bring up this point which I had raised on the > requirement spec. at the WG meeting. > It is from slide 5 on requirements where the flow definition has > been changed to include "incoming or outgoing interface" as a > MUST. The corresponding part of the requirements draft (ietf-ipfix-requirements-00) reads: 3.1. Interfaces The measuring device MUST be able to separate flows by the incoming interface or by the outgoing interface or by both of them. Note that this says "MUST be able to separate", not "MUST separate". Therefore I think your reasoning below doesn't apply: > There are cases like a SP is interested in getting all the > packest that go between 2 ASs. This does not have any strict > relevance to interface. An AS can be sourced by Multiple > interfaces in a router. Similar would be case if somebody > considers to define a flow based on other packet fields like > src/dst IP etc. In the general scheme of flow defintion ,I do not > see why there should be a restriction to include the interfaces > as a MUST. Claiming this as "most of the apps need the interfaces > today" is not a valid argument as this model is used to serve for > future apps. too. Therefore I request the WG to consider > removing this restriction. It is probably the case for *every* flow attribute that one can always come up with a sensible application that doesn't care about it. So it makes looks reasonable to make all flow attributes optional, in the sense that when an application isn't interested in them, the exporter doesn't have to send them in order to avoid waste of resources. This could be a requirement for the Flow Information Export protocol. But it also makes sense to make the *ability* of reporting certain attributes mandatory, so that applications can rely on them being available from all IPFIX compliant exporters. Personally I find the input and output interface sufficiently important to make their support mandatory. In particular the input interface is essential when one wants to do analysis of large-scale routing asymmetries. Input and output interfaces are also extremely useful when one collects flows at different points in the networks and wants to avoid counting the same flows twice. And the input interface cannot be derived from the source address and routing information, as can/must be done with e.g. the "source netmask", "upstream AS" or "source AS". Regards, -- Simon Leinen simon@babar.switch.ch SWITCH http://www.switch.ch/misc/leinen/ Computers hate being anthropomorphized. -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Sun Jan 6 20:44:40 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id UAA28311 for ; Sun, 6 Jan 2002 20:44:40 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16NOQ4-00046t-00 for ipfix-list@mil.doit.wisc.edu; Sun, 06 Jan 2002 19:17:44 -0600 Received: from d2cspimg001.smartpipes.com ([63.89.185.24]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 16NOQ3-00046R-00 for ipfix@net.doit.wisc.edu; Sun, 06 Jan 2002 19:17:43 -0600 Received: by D2CSPIMG001.smartpipes.com with Internet Mail Service (5.5.2653.19) id <44MDG1D1>; Mon, 7 Jan 2002 01:17:12 -0000 Message-ID: <4652644B98DFF34696801F8F3070D3FE0118855E@D2CSPEXM001.smartpipes.com> From: "Wang, Cliff" To: ipfix@net.doit.wisc.edu Subject: [ipfix] security requirement for IPFIX protocol Date: Mon, 7 Jan 2002 01:17:09 -0000 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01C19719.078DE1C0" Precedence: bulk Sender: majordomo listserver This message is in MIME format. Since your mail reader does not understand this format, some or all of this message may not be legible. ------_=_NextPart_001_01C19719.078DE1C0 Content-Type: text/plain In draft , Sec 5.3.3. Security Confidentiallity of transferred IPFIX data SHOULD be ensured. Integrity of transferred IPFIX data MUST be ensured. Authenticity of transferred IPFIX data MUST be ensured. Since we have started on the Architecture doc, I would like to further discuss the possible transport security solutions. At this point we potentially have several options: 1) The IPFIX protocol doesn't specify any security mechanism within the protocol. If security is desired (authentication/encryption/integrity), use the protection mechanism provided by other IETF WG solutions such as IPsec or TLS. 2) The IPFIX protocol specify the protection mechanism. Then the question will be at what level? At the data record level or at the packet level? Authentication only or encryption as well? Also Section 8 discusses some security considerations. Client (probe) to server (collection station) host level mutual authentication has not been fully discussed. I think this should fit into the WG charter: "Identify and address any security privacy concerns affecting flow data. Determine technology for securing the flow information export data, e.g. TLS. ". What in my mind is that in order to secure IPFIX data, you got to provide both host security (where data is stored) and transport security (where data is sent). However, we should only try to provide requirement and use existing solutions rather than building solutions from ground up. ------_=_NextPart_001_01C19719.078DE1C0 Content-Type: text/html Message

In draft <draft-ietf-ipfix-reqs-00.txt>, Sec 5.3.3. Security
 
   Confidentiallity of transferred IPFIX data SHOULD be ensured.
 
   Integrity of transferred IPFIX data MUST be ensured.
 
   Authenticity of transferred IPFIX data MUST be ensured.
 
 
Since we have started on the Architecture doc, I would like to further discuss the possible transport security solutions.
At this point we potentially have several options:
1) The IPFIX protocol doesn't specify any security mechanism within the protocol. If security is desired (authentication/encryption/integrity), use the protection mechanism provided by other IETF WG solutions such as IPsec or TLS.
2) The IPFIX protocol specify the protection mechanism. Then the question will be at what level? At the data record level or at the packet level? Authentication only or encryption as well?
 
 
Also Section 8 discusses some security considerations. Client (probe) to server (collection station) host level mutual authentication has not been fully discussed. I think this should fit into the WG charter:
"Identify and address any security privacy concerns affecting flow data. Determine technology for securing the flow information
export data, e.g. TLS. ".
 
What in my mind is that in order to secure IPFIX data, you got to provide both host security (where data is stored) and transport security (where data is sent). However, we should only try to provide requirement and use existing solutions rather than building solutions from ground up.
 
 
 
 
 
 
 
 

 
------_=_NextPart_001_01C19719.078DE1C0-- -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Sun Jan 6 22:44:30 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id WAA01194 for ; Sun, 6 Jan 2002 22:44:30 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16NQFz-0006U8-00 for ipfix-list@mil.doit.wisc.edu; Sun, 06 Jan 2002 21:15:27 -0600 Received: from dplonka by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16NQFw-0006U0-00 for ipfix@net.doit.wisc.edu; Sun, 06 Jan 2002 21:15:24 -0600 Date: Sun, 6 Jan 2002 21:15:24 -0600 From: Dave Plonka To: ipfix@net.doit.wisc.edu Subject: [ipfix] list policy changed to mitigate spam Message-ID: <20020106211524.A23326@doit.wisc.edu> Reply-To: plonka@doit.wisc.edu Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 1.0.1i Organization: UW-Madison, DoIT, Network Services X-VMS-Error: %SYSTEM-E-VA_IN_USE, virtual address already in use X-Shakespearean-Insult: Thou pribbling milk-livered dewberry Precedence: bulk Sender: majordomo listserver IPFIX members, As was recently suggested, the IPFIX mailing list policy has been changed so that posting requires the sender to be a current subscriber. Post attempts that do not appear to have been sent by a list subscriber will bounce to the list administrator (me), and I will endeavor to dispatch them, resending to the list if appropriate, in a timely fashion. Please let me know if you have trouble due to this change. Thanks, Dave -- plonka@doit.wisc.edu http://net.doit.wisc.edu/~plonka ARS:N9HZF Madison, WI -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Sun Jan 6 23:31:40 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id XAA01857 for ; Sun, 6 Jan 2002 23:31:40 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16NR8w-0007aL-00 for ipfix-list@mil.doit.wisc.edu; Sun, 06 Jan 2002 22:12:14 -0600 Received: from dplonka by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16NR8u-0007aF-00 for ipfix@net.doit.wisc.edu; Sun, 06 Jan 2002 22:12:12 -0600 Date: Sun, 6 Jan 2002 22:12:12 -0600 From: Dave Plonka To: ipfix@net.doit.wisc.edu Subject: [ipfix] new "volunteer" email aliases for architecture, data model efforts Message-ID: <20020106221212.A27462@doit.wisc.edu> Reply-To: plonka@doit.wisc.edu Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 1.0.1i Organization: UW-Madison, DoIT, Network Services X-VMS-Error: %SYSTEM-E-ARTRES, reserved arithmetic trap at PC=00000000, PS=00000472 X-Shakespearean-Insult: Thou gorbellied reeling-ripe baggage Precedence: bulk Sender: majordomo listserver IPFIX folks, I've created the new email aliases for the lists of "Architecture" and "Data Model" volunteers collected following the past WG meeting and in response to the solicitation in the minutes that were posted to the mailing list. The volunteers were as follows: * A D (A=Architecture, D=Data Model) - - X Carter Bullard X X Paul Calato X Will Eatherton X X Reinaldo Penno Filho X X Ram Gopal X X Simon Leinen X KC Norseth X X Juergen Quittek X X Ganesh Sadasivan X X Vamsi Valluri X X Tanja Zseby X Kevin Zhang X X Benoit Claise X Jean-Christophe Martin X Michelle Ma X X Tom Chen X Cliff Wang Here is a description, which I have added from the IPFIX web site, of the volunteer email aliases: Other Mailing Aliases The following are other email addresses pertinent to IPFIX. None of these should be used for general working group business as they are not archived publicly. They exist solely as a convenience to IPFIX participants so that need not maintain private entries for these folks in their address book: An alias for mailing the chairs: ipfix-chairs@net.doit.wisc.edu An alias used to contact volunteers for the IPFIX requirements: ipfix-req-volunteers@net.doit.wisc.edu An alias used to contact volunteers for the IPFIX architecture: ipfix-arch-volunteers@net.doit.wisc.edu An alias used to contact volunteers for the IPFIX data model: ipfix-data-volunteers@net.doit.wisc.edu Anyone wishing their address to be updated or removed from the aforementioned volunteer efforts should contact the chairs. When necessary, solicitation for additional volunteers will be done at a WG meeting AND in the draft meeting minutes, sent to the IPFIX mailing list, for those not able to attend the given meeting. Anyone is welcome to participate in IPFIX efforts. These "volunteer" aliases are merely a convenience so that any IPFIX participants can easily contact those volunteers when a status or other information is needed to be discussed in the public IPFIX mailing list proper. For future reference, this information can also be found on the IPFIX web site: http://ipfix.doit.wisc.edu/#Other_Mailing_Aliases Dave -- plonka@doit.wisc.edu http://net.doit.wisc.edu/~plonka ARS:N9HZF Madison, WI -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Sun Jan 6 23:40:47 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id XAA01997 for ; Sun, 6 Jan 2002 23:40:47 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16NRJv-00002I-00 for ipfix-list@mil.doit.wisc.edu; Sun, 06 Jan 2002 22:23:35 -0600 Received: from dplonka by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16NRJt-00002B-00 for ipfix@net.doit.wisc.edu; Sun, 06 Jan 2002 22:23:33 -0600 Date: Sun, 6 Jan 2002 22:23:33 -0600 From: Dave Plonka To: ipfix@net.doit.wisc.edu Subject: [ipfix] new aliases for IPFIX mailing list Message-ID: <20020106222333.B27462@doit.wisc.edu> Reply-To: plonka@doit.wisc.edu Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 1.0.1i Organization: UW-Madison, DoIT, Network Services X-VMS-Error: %SYSTEM-E-ARTRES, reserved arithmetic trap at PC=00000000, PS=00000472 X-Shakespearean-Insult: Thou gorbellied reeling-ripe baggage Precedence: bulk Sender: majordomo listserver IPFIX folks, As we work on the Architecture and Data Model efforts, we'll primarily continue to use the one public IPFIX mailing list so that everyone can participate and stay informed. However, I've created new aliases for the existing IPFIX mailing list which will cause the message Subject to be tagged appropriately for each of those those efforts, e.g. "[ipfix-req]", "[ipfix-arch]", or "[ipfix-data]". This was just an idea of mine, but I think it could help us to focus attention on particular threads of discussion in our areas of interest. Of course, email to "ipfix@net.doit.wisc.edu" will go to the list, just as it has in the past. Dave P.S. The aliases are described here: http://ipfix.doit.wisc.edu/#Mailing_List_Aliases_Convention and below: Mailing List Aliases, Conventions, and Policies There is only one IPFIX mailing list. General messages can be posted to the list by directing them to: ipfix@net.doit.wisc.edu However, because there are multiple efforts to prepare multiple documents within the IPFIX WG, there are multiple "entry points" which are simply aliases for that same list. The sole purpose of these aliases is to automatically tag the message Subject with a special prefix (e.g. "[ipfix-req]" for "IPFIX requirements"), so that the subject is clearer and the messages are able to be filtered based on the recipients interests, if the recipient wishes to do so. The following IPFIX mailing list aliases are available: Requirements ipfix-req@net.doit.wisc.edu Architecture ipfix-arch@net.doit.wisc.edu Data Model ipfix-data@net.doit.wisc.edu -- plonka@doit.wisc.edu http://net.doit.wisc.edu/~plonka ARS:N9HZF Madison, WI -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Mon Jan 7 15:29:07 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA27113 for ; Mon, 7 Jan 2002 15:29:07 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16NfxX-0005SR-00 for ipfix-list@mil.doit.wisc.edu; Mon, 07 Jan 2002 14:01:27 -0600 Received: from sj-msg-core-1.cisco.com ([171.71.163.11]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 16NfxW-0005Ru-00 for ipfix-arch@net.doit.wisc.edu; Mon, 07 Jan 2002 14:01:26 -0600 Received: from mira-sjcd-1.cisco.com (mira-sjcd-1.cisco.com [171.69.43.44]) by sj-msg-core-1.cisco.com (8.11.3/8.9.1) with ESMTP id g07K0sF22549 for ; Mon, 7 Jan 2002 12:00:55 -0800 (PST) Received: from cisco.com (dhcp-171-71-137-45.cisco.com [171.71.137.45]) by mira-sjcd-1.cisco.com (Mirapoint) with ESMTP id AAT89920; Mon, 7 Jan 2002 12:01:01 -0800 (PST) Message-ID: <3C3A0070.B37165BD@cisco.com> Date: Mon, 07 Jan 2002 12:09:20 -0800 From: Ganesh Sadasivan X-Mailer: Mozilla 4.76 [en]C-CCK-MCD (Windows NT 5.0; U) X-Accept-Language: en MIME-Version: 1.0 To: ipfix-arch@net.doit.wisc.edu Subject: [ipfix-arch] configuration information. Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Hi, One of the questions raised during the V9 presenatation if IETF was "how a collector would know whether or not the same packets or bytes were being reported in more than one flow record (e.g. using a different template)." (quoting from the minutes). This requires the collector to have configuration information regading the different templates, and the mode of selection of packets for each of these templates at each of the observation points. This information can be either 1. Communicated to the collector from the measuring device inband as a part of the export protocol. 2. The collector gets the required configuration out of band (.i.e not thru the export protocol. What are the thoughts of the group on this topic? If we are going by 1., should it be a part of the arch- spec? One think we need to keep in mind is that things like , details on method of packet selection could vary widely with implementations. Thanks Ganesh -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Mon Jan 7 16:23:53 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA28358 for ; Mon, 7 Jan 2002 16:23:53 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16Ngmf-0006Ph-00 for ipfix-list@mil.doit.wisc.edu; Mon, 07 Jan 2002 14:54:17 -0600 Received: from zrc2s0jx.nortelnetworks.com ([47.103.122.112]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 16Ngmd-0006PC-00 for ipfix-arch@net.doit.wisc.edu; Mon, 07 Jan 2002 14:54:15 -0600 Received: from zsc4c000.us.nortel.com (zsc4c000.us.nortel.com [47.81.138.47]) by zrc2s0jx.nortelnetworks.com (Switch-2.2.0/Switch-2.2.0) with ESMTP id g07Krat04881; Mon, 7 Jan 2002 14:53:36 -0600 (CST) Received: by zsc4c000.us.nortel.com with Internet Mail Service (5.5.2653.19) id ; Mon, 7 Jan 2002 12:51:47 -0800 Message-ID: <4F973E944951D311B6660008C7917CF00837E17A@zsc4c008.us.nortel.com> From: "Reinaldo Penno" To: "'Ganesh Sadasivan'" , "'ipfix-arch@net.doit.wisc.edu'" Subject: RE: [ipfix-arch] configuration information. Date: Mon, 7 Jan 2002 12:51:46 -0800 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01C197BD.1EA816B0" Precedence: bulk Sender: majordomo listserver This message is in MIME format. Since your mail reader does not understand this format, some or all of this message may not be legible. ------_=_NextPart_001_01C197BD.1EA816B0 Content-Type: text/plain; charset="iso-8859-1" Hello, How a collector would know that a priori? I mean, I can have such general templates that it would be very difficult to account for duplicate measurements (even in non real-time). If you have very specific templates such as srcIP, destIP, etc, it's not to hard find records in common. But you can have things like: src_AnyIP Dest_Yahoo.com Proto_Any and src_AnyIP Dest_AS3462 Proto_HTTP. How do you know a priori that the second rule will have records that are a subset of the first? Am I missing something? thanks, Reinaldo > -----Original Message----- > From: Ganesh Sadasivan [mailto:gsadasiv@cisco.com] > Sent: Monday, January 07, 2002 12:09 PM > To: ipfix-arch@net.doit.wisc.edu > Subject: [ipfix-arch] configuration information. > > > Hi, > One of the questions raised during the V9 presenatation if IETF was > "how a collector would know whether or not the same > packets or bytes were being reported in more than one flow record > (e.g. using a different template)." (quoting from the minutes). > This requires the collector to have configuration information > regading > the different templates, and the mode of selection of packets for > each > of these templates at each of the observation points. > This information can be either > 1. Communicated to the collector from the measuring device > inband as > a part > of the export protocol. > 2. The collector gets the required configuration out of band (.i.e > not thru > the export protocol. > > What are the thoughts of the group on this topic? > If we are going by 1., should it be a part of the arch- spec? One > think we > need to keep in mind is that things like , details on method of > packet > selection could vary widely with implementations. > > Thanks > Ganesh > > > > -- > Help mailto:majordomo@net.doit.wisc.edu and say "help" > in message body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ > ------_=_NextPart_001_01C197BD.1EA816B0 Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable RE: [ipfix-arch] configuration information.

Hello,

How a collector would know that a priori? I mean, I = can have such general templates that it would be very difficult to = account for duplicate measurements (even in non real-time).

If you have very specific templates such as srcIP, = destIP, etc, it's not to hard find records in common. But you can have = things like:

src_AnyIP   = Dest_Yahoo.com    Proto_Any   and
src_AnyIP   = Dest_AS3462       Proto_HTTP.

How do you know a priori that the second rule will = have records that are a subset of the first? Am I missing = something?

thanks,

Reinaldo



> -----Original Message-----
> From: Ganesh Sadasivan [mailto:gsadasiv@cisco.com]=
> Sent: Monday, January 07, 2002 12:09 PM
> To: ipfix-arch@net.doit.wisc.edu
> Subject: [ipfix-arch] configuration = information.
>
>
> Hi,
>    One of the questions raised = during the V9 presenatation if IETF was
>    "how a collector would = know whether or not the same
>    packets or bytes were being = reported in more than one flow record
>    (e.g. using a different = template)." (quoting from the minutes).
>    This requires the collector = to have configuration information
> regading
>    the different templates, and = the  mode of  selection of packets for
> each
>    of these templates at each of = the observation points.
>    This information can be = either
>    1. Communicated to the = collector from the measuring device
> inband as
> a part
>    of the export = protocol.
>    2. The collector gets the = required configuration out of band (.i.e
> not thru
>    the export protocol.
>
>    What are the thoughts of the = group on this topic?
>    If we are going by 1., should = it be a part of the arch- spec? One
> think we
>    need to keep in mind is that = things like , details on method of
> packet
>    selection could vary widely = with implementations.
>
> Thanks
> Ganesh
>
>
>
> --
> Help        = mailto:majordomo@net.doit.wi= sc.edu and say "help"
> in message body
> Unsubscribe mailto:majordomo@net.doit.wi= sc.edu and say
> "unsubscribe ipfix" in message = body
> Archive     http://ipfix.doit.wisc.edu/archive/
>

------_=_NextPart_001_01C197BD.1EA816B0-- -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Mon Jan 7 16:38:29 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA28667 for ; Mon, 7 Jan 2002 16:38:28 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16NhAC-0006uG-00 for ipfix-list@mil.doit.wisc.edu; Mon, 07 Jan 2002 15:18:36 -0600 Received: from dplonka by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16NhAB-0006uA-00 for ipfix-req@net.doit.wisc.edu; Mon, 07 Jan 2002 15:18:35 -0600 Date: Mon, 7 Jan 2002 15:18:35 -0600 From: Dave Plonka To: ipfix-req@net.doit.wisc.edu Subject: [ipfix-req] suggestion for requirements doc abstract Message-ID: <20020107151835.A19904@doit.wisc.edu> Reply-To: plonka@doit.wisc.edu Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 1.0.1i Organization: UW-Madison, DoIT, Network Services X-VMS-Error: %SYSTEM-W-SLICING_DISABLE, image slicing is disabled X-Shakespearean-Insult: Thou jarring swag-bellied giglet Precedence: bulk Sender: majordomo listserver IPFIX folks, The term "passive measurement" is commonly used by the measurement community and I have been thinking about how to include it in our requirements document to help clarify which systems are bound to these requirements. I suggest we reword the requirements abstract (see "http://www.ietf.org/internet-drafts/draft-ietf-ipfix-reqs-00.txt") to this: Abstract This memo defines requirements for network elements that export Internet Protocol flow information within a passive measurement framework. These network elements are devices such as routers, dedicated traffic measurement probes, and middleboxes. Then in the terminology section we can define "exporter" thusly: "exporter": a network element such as a router, dedicated traffic measurement probe, or middlebox, that exports IP flow information to one or more collectors. Note that "network element" is used similarly (and perhaps introduced?) in "A Simple Network Management Protocol" (RFC 1067). Lastly, in "draft-ietf-ipfix-reqs-00.txt" there are two instances (one being the present abstract) of the phrase "export [...] flow information out of". I suggest that we instead say "from" rather than "out of". Dave -- ":) You will make many changes before settling satisfactorily. :)" - the actual message (complete with smiley faces) that I received in a fortune cookie at dinner after the IPFIX WG meeting during IETF 52 in Salt Lake City -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Mon Jan 7 16:51:02 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA28972 for ; Mon, 7 Jan 2002 16:51:02 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16NhKJ-00076G-00 for ipfix-list@mil.doit.wisc.edu; Mon, 07 Jan 2002 15:29:03 -0600 Received: from sj-msg-core-2.cisco.com ([171.69.24.11]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 16NhKF-00075X-00 for ipfix-arch@net.doit.wisc.edu; Mon, 07 Jan 2002 15:28:59 -0600 Received: from mira-sjcd-1.cisco.com (mira-sjcd-1.cisco.com [171.69.43.44]) by sj-msg-core-2.cisco.com (8.11.3/8.9.1) with ESMTP id g07LSS407647; Mon, 7 Jan 2002 13:28:28 -0800 (PST) Received: from cisco.com (dhcp-171-71-137-45.cisco.com [171.71.137.45]) by mira-sjcd-1.cisco.com (Mirapoint) with ESMTP id AAT93065; Mon, 7 Jan 2002 13:28:35 -0800 (PST) Message-ID: <3C3A14F5.AD6A9AFC@cisco.com> Date: Mon, 07 Jan 2002 13:36:54 -0800 From: Ganesh Sadasivan X-Mailer: Mozilla 4.76 [en]C-CCK-MCD (Windows NT 5.0; U) X-Accept-Language: en MIME-Version: 1.0 To: Reinaldo Penno CC: "'ipfix-arch@net.doit.wisc.edu'" Subject: Re: [ipfix-arch] configuration information. References: <4F973E944951D311B6660008C7917CF00837E17A@zsc4c008.us.nortel.com> Content-Type: multipart/alternative; boundary="------------CC45A407E931E76F4C375CE1" Precedence: bulk Sender: majordomo listserver --------------CC45A407E931E76F4C375CE1 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Reinaldo, I agree with your point that it is non-trivial to derive the complete relationship between the packets collected various templates. My question to the group is that whether IPFIX should spend time on architecting how this detailed config information is sent to the collector so that the collector gets this information inband or whether this can be dealt by the collector in an adhoc manner. If we go by the former case to what level of details should the collector be made aware of about templates, packet selection and their relationships. Thanks Ganesh Reinaldo Penno wrote: > > > Hello, > > How a collector would know that a priori? I mean, I can have such > general templates that it would be very difficult to account for > duplicate measurements (even in non real-time). > > If you have very specific templates such as srcIP, destIP, etc, it's > not to hard find records in common. But you can have things like: > > src_AnyIP Dest_Yahoo.com Proto_Any and > src_AnyIP Dest_AS3462 Proto_HTTP. > > How do you know a priori that the second rule will have records that > are a subset of the first? Am I missing something? > > thanks, > > Reinaldo > > > > -----Original Message----- > > From: Ganesh Sadasivan [mailto:gsadasiv@cisco.com] > > Sent: Monday, January 07, 2002 12:09 PM > > To: ipfix-arch@net.doit.wisc.edu > > Subject: [ipfix-arch] configuration information. > > > > > > Hi, > > One of the questions raised during the V9 presenatation if IETF > was > > "how a collector would know whether or not the same > > packets or bytes were being reported in more than one flow record > > > (e.g. using a different template)." (quoting from the minutes). > > This requires the collector to have configuration information > > regading > > the different templates, and the mode of selection of packets > for > > each > > of these templates at each of the observation points. > > This information can be either > > 1. Communicated to the collector from the measuring device > > inband as > > a part > > of the export protocol. > > 2. The collector gets the required configuration out of band > (.i.e > > not thru > > the export protocol. > > > > What are the thoughts of the group on this topic? > > If we are going by 1., should it be a part of the arch- spec? One > > > think we > > need to keep in mind is that things like , details on method of > > packet > > selection could vary widely with implementations. > > > > Thanks > > Ganesh > > > > > > > > -- > > Help mailto:majordomo@net.doit.wisc.edu and say "help" > > in message body > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > "unsubscribe ipfix" in message body > > Archive http://ipfix.doit.wisc.edu/archive/ > > --------------CC45A407E931E76F4C375CE1 Content-Type: text/html; charset=us-ascii Content-Transfer-Encoding: 7bit Reinaldo,
I agree with your point that it is non-trivial to derive the complete relationship
between the packets collected various templates. My question to the group is that
whether IPFIX should spend time on architecting how this detailed config
information is sent to the collector so that the collector gets this information inband
or whether this can be dealt by the collector in an adhoc manner.
If we go by the former case to what level of details should the collector be
made aware of  about templates, packet selection and their relationships.

Thanks
Ganesh

Reinaldo Penno wrote:

 

Hello,

How a collector would know that a priori? I mean, I can have such general templates that it would be very difficult to account for duplicate measurements (even in non real-time).

If you have very specific templates such as srcIP, destIP, etc, it's not to hard find records in common. But you can have things like:

src_AnyIP   Dest_Yahoo.com    Proto_Any   and
src_AnyIP   Dest_AS3462       Proto_HTTP.

How do you know a priori that the second rule will have records that are a subset of the first? Am I missing something?

thanks,

Reinaldo
 

> -----Original Message-----
> From: Ganesh Sadasivan [mailto:gsadasiv@cisco.com]
> Sent: Monday, January 07, 2002 12:09 PM
> To: ipfix-arch@net.doit.wisc.edu
> Subject: [ipfix-arch] configuration information.
>
>
> Hi,
>    One of the questions raised during the V9 presenatation if IETF was
>    "how a collector would know whether or not the same
>    packets or bytes were being reported in more than one flow record
>    (e.g. using a different template)." (quoting from the minutes).
>    This requires the collector to have configuration information
> regading
>    the different templates, and the  mode of  selection of packets for
> each
>    of these templates at each of the observation points.
>    This information can be either
>    1. Communicated to the collector from the measuring device
> inband as
> a part
>    of the export protocol.
>    2. The collector gets the required configuration out of band (.i.e
> not thru
>    the export protocol.
>
>    What are the thoughts of the group on this topic?
>    If we are going by 1., should it be a part of the arch- spec? One
> think we
>    need to keep in mind is that things like , details on method of
> packet
>    selection could vary widely with implementations.
>
> Thanks
> Ganesh
>
>
>
> --
> Help        mailto:majordomo@net.doit.wisc.edu and say "help"
> in message body
> Unsubscribe mailto:majordomo@net.doit.wisc.edu and say
> "unsubscribe ipfix" in message body
> Archive     http://ipfix.doit.wisc.edu/archive/
>

--------------CC45A407E931E76F4C375CE1-- -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Mon Jan 7 17:14:32 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA29417 for ; Mon, 7 Jan 2002 17:14:32 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16Nhg5-0007XT-00 for ipfix-list@mil.doit.wisc.edu; Mon, 07 Jan 2002 15:51:33 -0600 Received: from dplonka by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16Nhg4-0007XO-00 for ipfix-req@net.doit.wisc.edu; Mon, 07 Jan 2002 15:51:32 -0600 Date: Mon, 7 Jan 2002 15:51:32 -0600 From: Dave Plonka To: ipfix-req@net.doit.wisc.edu Subject: [ipfix-req] unidirection vs. bidirectional flows Message-ID: <20020107155132.B19904@doit.wisc.edu> Reply-To: plonka@doit.wisc.edu Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 1.0.1i Organization: UW-Madison, DoIT, Network Services X-VMS-Error: %SYSTEM-W-SLICING_DISABLE, image slicing is disabled X-Shakespearean-Insult: Thou jarring swag-bellied giglet Precedence: bulk Sender: majordomo listserver IPFIX folks, As many of you recall, whether or not IPFIX should specify a unidirectional or bidirectional flow definition was a hot topic at the WG meeting. I think that specifying *something* gets us far along in communicating what a "flow" is. Personally, I would like us to specify one or the other and I am in favor a unidirection flow definition. Here are some thoughts about arguments made for bidirectional flows: A unidirectional flow definition means that the atomic flow report record is unidirection during the reporting/export phase. For instance, a single packet counter for a given protocol/srcaddr/srcport/dstaddr/dstport n-tuple. However, since the exporter is somewhat of a "black box", it need not be constrained to a unidirectional model internally. A unidirectional flow definition does not prevent a collector implementor from using a bi-directional notion of flows internally nor does it prevent it from being stored in a bidirectional way prior to export. Conversely if we specify a bidirectional definition, it does disqualify unidirectional implementations. For those that wish to perform measurements which may require bidirectional flow information (perhaps because start/end timestamp synchronization is required for flow pairs), the IPFIX data model and transport could define a mechanism by which the exporter could inform the collector about which unidirectional flow records are matched pairs. That is, the exporter could help the collector find the matched pairs of flow information, perhaps by exporting the flow records in pairs or by generating a numeric identifier that is an attribute of both flow records. Of course only bidirectional collectors would understand this convention, but a unidirection collector could ignore it and still make use of the exported flow information from an exporter that implemented this "bidirectional flow pairing" feature. Dave -- ":) You will make many changes before settling satisfactorily. :)" - the actual message (complete with smiley faces) that I received in a fortune cookie at dinner after the IPFIX WG meeting during IETF 52 in Salt Lake City -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Mon Jan 7 17:36:52 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA29798 for ; Mon, 7 Jan 2002 17:36:52 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16Ni3c-0000GP-00 for ipfix-list@mil.doit.wisc.edu; Mon, 07 Jan 2002 16:15:52 -0600 Received: from sj-msg-core-4.cisco.com ([171.71.163.10]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 16Ni3Y-0000Fj-00 for ipfix-req@net.doit.wisc.edu; Mon, 07 Jan 2002 16:15:48 -0600 Received: from postal.cisco.com (postal.cisco.com [171.71.160.26]) by sj-msg-core-4.cisco.com (8.11.3/8.9.1) with ESMTP id g07MFHJ12569; Mon, 7 Jan 2002 14:15:17 -0800 (PST) Received: from WILLW2K (dhcp-171-71-139-66.cisco.com [171.71.139.66]) by postal.cisco.com (Mirapoint) with SMTP id AAH60727; Mon, 7 Jan 2002 14:15:26 -0800 (PST) Reply-To: From: "Will Eatherton" To: , Subject: RE: [ipfix-req] unidirection vs. bidirectional flows Date: Mon, 7 Jan 2002 14:14:10 -0800 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="US-ASCII" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2910.0) In-Reply-To: <20020107155132.B19904@doit.wisc.edu> X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400 Importance: Normal Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit > Conversely if we specify a bidirectional definition, it does > disqualify unidirectional implementations. I think this is a key point and I agree that bidirectional definition disqualifies or at the very least encumbers unidirection implementations. Also based on currently developed and deployed traffic export mechanisms (i.e. netflow) unidirectional definition has a leg up. Will Eatherton > > For those that wish to perform measurements which may require > bidirectional flow information (perhaps because start/end timestamp > synchronization is required for flow pairs), the IPFIX data model and > transport could define a mechanism by which the exporter could inform > the collector about which unidirectional flow records are matched > pairs. That is, the exporter could help the collector find the matched > pairs of flow information, perhaps by exporting the flow records in > pairs or by generating a numeric identifier that is an attribute of > both flow records. Of course only bidirectional collectors would > understand this convention, but a unidirection collector could ignore > it and still make use of the exported flow information from an exporter > that implemented this "bidirectional flow pairing" feature. > > Dave > > -- > ":) You will make many changes before settling satisfactorily. :)" > - the actual message (complete with smiley faces) that I received in a > fortune cookie at dinner after the IPFIX WG meeting during IETF 52 > in Salt Lake City > > -- > Help mailto:majordomo@net.doit.wisc.edu and say "help" in > message body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Mon Jan 7 18:47:12 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id SAA01243 for ; Mon, 7 Jan 2002 18:47:12 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16NjEi-0001qo-00 for ipfix-list@mil.doit.wisc.edu; Mon, 07 Jan 2002 17:31:24 -0600 Received: from relay1.pair.com ([209.68.1.20] helo=relay.pair.com) by mil.doit.wisc.edu with smtp (Exim 3.13 #1) id 16NjEg-0001qf-00 for ipfix-req@net.doit.wisc.edu; Mon, 07 Jan 2002 17:31:22 -0600 Received: (qmail 43075 invoked from network); 7 Jan 2002 23:31:19 -0000 Received: from pool-162-83-252-213.ny5030.east.verizon.net (HELO sphynx) (162.83.252.213) by relay1.pair.com with SMTP; 7 Jan 2002 23:31:19 -0000 X-pair-Authenticated: 162.83.252.213 Reply-To: From: "Carter Bullard" To: , Subject: RE: [ipfix-req] unidirection vs. bidirectional flows Date: Mon, 7 Jan 2002 18:31:08 -0500 Organization: QoSient, LLC Message-ID: <5C8959A16A71B449AE793CF52FBBED66018FCB@ptah.newyork.qosient.com> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook, Build 10.0.3416 Importance: Normal In-Reply-To: <20020107155132.B19904@doit.wisc.edu> X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Gentle people, There is absolutely no reason for IPFIX to limit the type of flow data that it supports. The difference between uni-directional and bi-directional flow records is simply the type of metrics that are reported. The flow descriptors and keys are all the same. Because IPFIX must support reporting metrics that each vendors will not implement, such as jitter and one-way delay, which are the few metrics that are meaningful in uni-directional flow data models, supporting metrics that report on return traffic should not be a hardship for IPFIX. But, I believe that if you are going to make a design choice it should be motivated by design goals. What benefit do you gain by limiting IPFIX to a simple uni-directional flow model? Is a uni-directional flow data model the best data model for reporting all the metrics and network behaviors that relate to network flows? Is it the best data model for supporting all the existing commercial network flow data? But what about even the simplest of applications of flow data? Is a uni-directional flow data model the best solution for reporting the RTT of a ping volley? The empirical bulk transfer properties of a TCP connection? How about TCP packet retransmission? Existence of routing failures in an OSPF routed network? Application response time? Rerouting indications? Reachability fault detection? Access policy validation? Connectivity status? Even FTP supports both ASCII and binary data transfer. Handling both uni-directional data and bi-directional data in IPFIX should be pretty easy stuff. Carter Carter Bullard QoSient, LLC 300 E. 56th Street, Suite 18K New York, New York 10022 carter@qosient.com Phone +1 212 588-9133 Fax +1 212 588-9134 http://qosient.com > -----Original Message----- > From: majordomo listserver > [mailto:majordomo@mil.doit.wisc.edu] On Behalf Of Dave Plonka > Sent: Monday, January 07, 2002 4:52 PM > To: ipfix-req@net.doit.wisc.edu > Subject: [ipfix-req] unidirection vs. bidirectional flows > > > > IPFIX folks, > > As many of you recall, whether or not IPFIX should specify a > unidirectional or bidirectional flow definition was a hot > topic at the WG meeting. > > I think that specifying *something* gets us far along in > communicating what a "flow" is. Personally, I would like us > to specify one or the other and I am in favor a unidirection > flow definition. > > Here are some thoughts about arguments made for bidirectional flows: > > A unidirectional flow definition means that the atomic flow > report record is unidirection during the reporting/export > phase. For instance, a single packet counter for a given > protocol/srcaddr/srcport/dstaddr/dstport n-tuple. However, > since the exporter is somewhat of a "black box", it need not > be constrained to a unidirectional model internally. > > A unidirectional flow definition does not prevent a collector > implementor from using a bi-directional notion of flows > internally nor does it prevent it from being stored in a > bidirectional way prior to export. Conversely if we specify > a bidirectional definition, it does disqualify unidirectional > implementations. > > For those that wish to perform measurements which may require > bidirectional flow information (perhaps because start/end > timestamp synchronization is required for flow pairs), the > IPFIX data model and transport could define a mechanism by > which the exporter could inform the collector about which > unidirectional flow records are matched pairs. That is, the > exporter could help the collector find the matched pairs of > flow information, perhaps by exporting the flow records in > pairs or by generating a numeric identifier that is an > attribute of both flow records. Of course only bidirectional > collectors would understand this convention, but a > unidirection collector could ignore it and still make use of > the exported flow information from an exporter that > implemented this "bidirectional flow pairing" feature. > > Dave > > -- > ":) You will make many changes before settling satisfactorily. :)" > - the actual message (complete with smiley faces) that I > received in a > fortune cookie at dinner after the IPFIX WG meeting during IETF 52 > in Salt Lake City > > -- > Help mailto:majordomo@net.doit.wisc.edu and say "help" > in message body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ > > -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Mon Jan 7 19:19:48 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id TAA01617 for ; Mon, 7 Jan 2002 19:19:48 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16Njf6-0002N4-00 for ipfix-list@mil.doit.wisc.edu; Mon, 07 Jan 2002 17:58:40 -0600 Received: from sj-msg-core-1.cisco.com ([171.71.163.11]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 16Njf5-0002Mw-00 for ipfix-req@net.doit.wisc.edu; Mon, 07 Jan 2002 17:58:39 -0600 Received: from vvalluri-u10.cisco.com (vvalluri-u10.cisco.com [128.107.162.119]) by sj-msg-core-1.cisco.com (8.11.3/8.9.1) with ESMTP id g07NvVF21675; Mon, 7 Jan 2002 15:57:31 -0800 (PST) Date: Mon, 7 Jan 2002 15:57:30 -0800 (PST) From: Vamsidhar Valluri To: Carter Bullard cc: , Subject: RE: [ipfix-req] unidirection vs. bidirectional flows In-Reply-To: <5C8959A16A71B449AE793CF52FBBED66018FCB@ptah.newyork.qosient.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Precedence: bulk Sender: majordomo listserver On Mon, 7 Jan 2002, Carter Bullard wrote: > Gentle people, > There is absolutely no reason for IPFIX to limit the type > of flow data that it supports. The difference between > uni-directional and bi-directional flow records is simply > the type of metrics that are reported. The flow descriptors > and keys are all the same. Because IPFIX must support ^^^^^ If "keys" you are refering to are the fields used to identify a flow then "SRC interface" do not fit into the statement "The flow descriptors and keys are all same for both unidirectional and bi-directional flow records" mentioned above because SRC interface will make it unidirectional. -vamsi >reporting metrics that each vendors will not implement, such > as jitter and one-way delay, which are the few metrics that > are meaningful in uni-directional flow data models, supporting > metrics that report on return traffic should not be a > hardship for IPFIX. > > But, I believe that if you are going to make a design choice > it should be motivated by design goals. What benefit > do you gain by limiting IPFIX to a simple uni-directional > flow model? Is a uni-directional flow data model the best > data model for reporting all the metrics and network behaviors > that relate to network flows? Is it the best data model for > supporting all the existing commercial network flow data? > > But what about even the simplest of applications of flow data? > Is a uni-directional flow data model the best solution for > reporting the RTT of a ping volley? The empirical bulk transfer > properties of a TCP connection? How about TCP packet retransmission? > Existence of routing failures in an OSPF routed network? > Application response time? Rerouting indications? Reachability > fault detection? Access policy validation? Connectivity status? > > Even FTP supports both ASCII and binary data transfer. Handling > both uni-directional data and bi-directional data in IPFIX should > be pretty easy stuff. > > > Carter > > Carter Bullard > QoSient, LLC > 300 E. 56th Street, Suite 18K > New York, New York 10022 > > carter@qosient.com > Phone +1 212 588-9133 > Fax +1 212 588-9134 > http://qosient.com > > > > > > -----Original Message----- > > From: majordomo listserver > > [mailto:majordomo@mil.doit.wisc.edu] On Behalf Of Dave Plonka > > Sent: Monday, January 07, 2002 4:52 PM > > To: ipfix-req@net.doit.wisc.edu > > Subject: [ipfix-req] unidirection vs. bidirectional flows > > > > > > > > IPFIX folks, > > > > As many of you recall, whether or not IPFIX should specify a > > unidirectional or bidirectional flow definition was a hot > > topic at the WG meeting. > > > > I think that specifying *something* gets us far along in > > communicating what a "flow" is. Personally, I would like us > > to specify one or the other and I am in favor a unidirection > > flow definition. > > > > Here are some thoughts about arguments made for bidirectional flows: > > > > A unidirectional flow definition means that the atomic flow > > report record is unidirection during the reporting/export > > phase. For instance, a single packet counter for a given > > protocol/srcaddr/srcport/dstaddr/dstport n-tuple. However, > > since the exporter is somewhat of a "black box", it need not > > be constrained to a unidirectional model internally. > > > > A unidirectional flow definition does not prevent a collector > > implementor from using a bi-directional notion of flows > > internally nor does it prevent it from being stored in a > > bidirectional way prior to export. Conversely if we specify > > a bidirectional definition, it does disqualify unidirectional > > implementations. > > > > For those that wish to perform measurements which may require > > bidirectional flow information (perhaps because start/end > > timestamp synchronization is required for flow pairs), the > > IPFIX data model and transport could define a mechanism by > > which the exporter could inform the collector about which > > unidirectional flow records are matched pairs. That is, the > > exporter could help the collector find the matched pairs of > > flow information, perhaps by exporting the flow records in > > pairs or by generating a numeric identifier that is an > > attribute of both flow records. Of course only bidirectional > > collectors would understand this convention, but a > > unidirection collector could ignore it and still make use of > > the exported flow information from an exporter that > > implemented this "bidirectional flow pairing" feature. > > > > Dave > > > > -- > > ":) You will make many changes before settling satisfactorily. :)" > > - the actual message (complete with smiley faces) that I > > received in a > > fortune cookie at dinner after the IPFIX WG meeting during IETF 52 > > in Salt Lake City > > > > -- > > Help mailto:majordomo@net.doit.wisc.edu and say "help" > > in message body > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > "unsubscribe ipfix" in message body > > Archive http://ipfix.doit.wisc.edu/archive/ > > > > > > > > -- > Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ > -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Mon Jan 7 20:09:45 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id UAA02191 for ; Mon, 7 Jan 2002 20:09:45 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16NkVl-0003aS-00 for ipfix-list@mil.doit.wisc.edu; Mon, 07 Jan 2002 18:53:05 -0600 Received: from sj-msg-core-1.cisco.com ([171.71.163.11]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 16NkVj-0003Zp-00 for ipfix-req@net.doit.wisc.edu; Mon, 07 Jan 2002 18:53:03 -0600 Received: from vvalluri-u10.cisco.com (vvalluri-u10.cisco.com [128.107.162.119]) by sj-msg-core-1.cisco.com (8.11.3/8.9.1) with ESMTP id g080puF28453; Mon, 7 Jan 2002 16:51:56 -0800 (PST) Date: Mon, 7 Jan 2002 16:51:56 -0800 (PST) From: Vamsidhar Valluri To: Carter Bullard cc: , Subject: RE: [ipfix-req] unidirection vs. bidirectional flows In-Reply-To: <5C8959A16A71B449AE793CF52FBBED66018FCC@ptah.newyork.qosient.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Precedence: bulk Sender: majordomo listserver On Mon, 7 Jan 2002, Carter Bullard wrote: > Hey Vamsi, > Hmmm, is the interface a part of the flow defintion? > Not a problem. Lets look at how this may work. Suppose there > is a SRC interface and a DST interface, so that a flow is > simply all the packets moving from SRC -> DST. The return flow > would be all the packets moving from DST -> SRC. > > So the flow is reported as SRC <-> DST, with the simple > addition of the DST -> SRC metrics. Makes more sense than > independantly reporting two half-duplex flow records, which > could be skewed in time so badly, that the two records cannot > be correlated at all. Hi Carter, True, but there are certain high end platforms which will take considerable hit if destination interface has to become part of "keys". Other way of looking at it is to take away "interfaces" from keys so that bi-directional flows map to the same flow record but here we may lose interface information because birectional flows might take different interfaces at different point of time, if not then we are fine. -vamsi > > > Carter > > Carter Bullard > QoSient, LLC > 300 E. 56th Street, Suite 18K > New York, New York 10022 > > carter@qosient.com > Phone +1 212 588-9133 > Fax +1 212 588-9134 > http://qosient.com > > > > -----Original Message----- > > From: Vamsidhar Valluri [mailto:vvalluri@cisco.com] > > Sent: Monday, January 07, 2002 6:58 PM > > To: Carter Bullard > > Cc: plonka@doit.wisc.edu; ipfix-req@net.doit.wisc.edu > > Subject: RE: [ipfix-req] unidirection vs. bidirectional flows > > > > > > > > > > On Mon, 7 Jan 2002, Carter Bullard wrote: > > > > > Gentle people, > > > There is absolutely no reason for IPFIX to limit the type > > > of flow data that it supports. The difference between > > uni-directional > > > and bi-directional flow records is simply the type of > > metrics that are > > > reported. The flow descriptors and keys are all the same. Because > > > IPFIX must support > > ^^^^^ > > > > If "keys" you are refering to are the fields used to identify > > a flow then "SRC interface" do not fit into the statement > > "The flow descriptors and keys are all same for both > > unidirectional and bi-directional flow records" mentioned > > above because SRC interface will make it unidirectional. > > > > -vamsi > > > > > > >reporting metrics that each vendors will not implement, such > > as jitter > > >and one-way delay, which are the few metrics that are meaningful in > > >uni-directional flow data models, supporting metrics that report on > > >return traffic should not be a hardship for IPFIX. > > > > > > But, I believe that if you are going to make a design > > choice it should > > > be motivated by design goals. What benefit do you gain by limiting > > > IPFIX to a simple uni-directional > > > flow model? Is a uni-directional flow data model the best > > > data model for reporting all the metrics and network behaviors that > > > relate to network flows? Is it the best data model for > > supporting all > > > the existing commercial network flow data? > > > > > > But what about even the simplest of applications of flow data? Is a > > > uni-directional flow data model the best solution for reporting the > > > RTT of a ping volley? The empirical bulk transfer > > properties of a TCP > > > connection? How about TCP packet retransmission? Existence > > of routing > > > failures in an OSPF routed network? Application response time? > > > Rerouting indications? Reachability fault detection? > > Access policy > > > validation? Connectivity status? > > > > > > Even FTP supports both ASCII and binary data transfer. > > Handling both > > > uni-directional data and bi-directional data in IPFIX > > should be pretty > > > easy stuff. > > > > > > > > > Carter > > > > > > Carter Bullard > > > QoSient, LLC > > > 300 E. 56th Street, Suite 18K > > > New York, New York 10022 > > > > > > carter@qosient.com > > > Phone +1 212 588-9133 > > > Fax +1 212 588-9134 > > > http://qosient.com > > > > > > > > > > > > > > > > -----Original Message----- > > > > From: majordomo listserver > > [mailto:majordomo@mil.doit.wisc.edu] On > > > > Behalf Of Dave Plonka > > > > Sent: Monday, January 07, 2002 4:52 PM > > > > To: ipfix-req@net.doit.wisc.edu > > > > Subject: [ipfix-req] unidirection vs. bidirectional flows > > > > > > > > > > > > > > > > IPFIX folks, > > > > > > > > As many of you recall, whether or not IPFIX should specify a > > > > unidirectional or bidirectional flow definition was a hot > > topic at > > > > the WG meeting. > > > > > > > > I think that specifying *something* gets us far along in > > > > communicating what a "flow" is. Personally, I would like us to > > > > specify one or the other and I am in favor a unidirection flow > > > > definition. > > > > > > > > Here are some thoughts about arguments made for > > bidirectional flows: > > > > > > > > A unidirectional flow definition means that the atomic > > flow report > > > > record is unidirection during the reporting/export phase. For > > > > instance, a single packet counter for a given > > > > protocol/srcaddr/srcport/dstaddr/dstport n-tuple. However, since > > > > the exporter is somewhat of a "black box", it need not be > > > > constrained to a unidirectional model internally. > > > > > > > > A unidirectional flow definition does not prevent a collector > > > > implementor from using a bi-directional notion of flows > > internally > > > > nor does it prevent it from being stored in a bidirectional way > > > > prior to export. Conversely if we specify a bidirectional > > > > definition, it does disqualify unidirectional implementations. > > > > > > > > For those that wish to perform measurements which may require > > > > bidirectional flow information (perhaps because start/end > > timestamp > > > > synchronization is required for flow pairs), the IPFIX data model > > > > and transport could define a mechanism by which the > > exporter could > > > > inform the collector about which unidirectional flow records are > > > > matched pairs. That is, the exporter could help the > > collector find > > > > the matched pairs of flow information, perhaps by > > exporting the flow > > > > records in pairs or by generating a numeric identifier that is an > > > > attribute of both flow records. Of course only bidirectional > > > > collectors would understand this convention, but a > > > > unidirection collector could ignore it and still make use of > > > > the exported flow information from an exporter that > > > > implemented this "bidirectional flow pairing" feature. > > > > > > > > Dave > > > > > > > > -- > > > > ":) You will make many changes before settling satisfactorily. :)" > > > > - the actual message (complete with smiley faces) that I > > received > > > > in a > > > > fortune cookie at dinner after the IPFIX WG meeting > > during IETF 52 > > > > in Salt Lake City > > > > > > > > -- > > > > Help mailto:majordomo@net.doit.wisc.edu and say "help" > > > > in message body > > > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > "unsubscribe > > > > ipfix" in message body > > > > Archive http://ipfix.doit.wisc.edu/archive/ > > > > > > > > > > > > > > > > > > > > -- > > > Help mailto:majordomo@net.doit.wisc.edu and say > > "help" in message body > > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe > > > ipfix" in message body > > > Archive http://ipfix.doit.wisc.edu/archive/ > > > > > > > > > > > > -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Mon Jan 7 20:09:46 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id UAA02195 for ; Mon, 7 Jan 2002 20:09:45 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16NkLf-0003MJ-00 for ipfix-list@mil.doit.wisc.edu; Mon, 07 Jan 2002 18:42:39 -0600 Received: from relay1.pair.com ([209.68.1.20] helo=relay.pair.com) by mil.doit.wisc.edu with smtp (Exim 3.13 #1) id 16NkLc-0003MD-00 for ipfix-req@net.doit.wisc.edu; Mon, 07 Jan 2002 18:42:36 -0600 Received: (qmail 49514 invoked from network); 8 Jan 2002 00:42:35 -0000 Received: from pool-162-83-252-213.ny5030.east.verizon.net (HELO sphynx) (162.83.252.213) by relay1.pair.com with SMTP; 8 Jan 2002 00:42:35 -0000 X-pair-Authenticated: 162.83.252.213 Reply-To: From: "Carter Bullard" To: "'Vamsidhar Valluri'" Cc: , Subject: RE: [ipfix-req] unidirection vs. bidirectional flows Date: Mon, 7 Jan 2002 19:42:22 -0500 Organization: QoSient, LLC Message-ID: <5C8959A16A71B449AE793CF52FBBED66018FCC@ptah.newyork.qosient.com> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook, Build 10.0.3416 Importance: Normal In-Reply-To: X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Hey Vamsi, Hmmm, is the interface a part of the flow defintion? Not a problem. Lets look at how this may work. Suppose there is a SRC interface and a DST interface, so that a flow is simply all the packets moving from SRC -> DST. The return flow would be all the packets moving from DST -> SRC. So the flow is reported as SRC <-> DST, with the simple addition of the DST -> SRC metrics. Makes more sense than independantly reporting two half-duplex flow records, which could be skewed in time so badly, that the two records cannot be correlated at all. Carter Carter Bullard QoSient, LLC 300 E. 56th Street, Suite 18K New York, New York 10022 carter@qosient.com Phone +1 212 588-9133 Fax +1 212 588-9134 http://qosient.com > -----Original Message----- > From: Vamsidhar Valluri [mailto:vvalluri@cisco.com] > Sent: Monday, January 07, 2002 6:58 PM > To: Carter Bullard > Cc: plonka@doit.wisc.edu; ipfix-req@net.doit.wisc.edu > Subject: RE: [ipfix-req] unidirection vs. bidirectional flows > > > > > On Mon, 7 Jan 2002, Carter Bullard wrote: > > > Gentle people, > > There is absolutely no reason for IPFIX to limit the type > > of flow data that it supports. The difference between > uni-directional > > and bi-directional flow records is simply the type of > metrics that are > > reported. The flow descriptors and keys are all the same. Because > > IPFIX must support > ^^^^^ > > If "keys" you are refering to are the fields used to identify > a flow then "SRC interface" do not fit into the statement > "The flow descriptors and keys are all same for both > unidirectional and bi-directional flow records" mentioned > above because SRC interface will make it unidirectional. > > -vamsi > > > >reporting metrics that each vendors will not implement, such > as jitter > >and one-way delay, which are the few metrics that are meaningful in > >uni-directional flow data models, supporting metrics that report on > >return traffic should not be a hardship for IPFIX. > > > > But, I believe that if you are going to make a design > choice it should > > be motivated by design goals. What benefit do you gain by limiting > > IPFIX to a simple uni-directional > > flow model? Is a uni-directional flow data model the best > > data model for reporting all the metrics and network behaviors that > > relate to network flows? Is it the best data model for > supporting all > > the existing commercial network flow data? > > > > But what about even the simplest of applications of flow data? Is a > > uni-directional flow data model the best solution for reporting the > > RTT of a ping volley? The empirical bulk transfer > properties of a TCP > > connection? How about TCP packet retransmission? Existence > of routing > > failures in an OSPF routed network? Application response time? > > Rerouting indications? Reachability fault detection? > Access policy > > validation? Connectivity status? > > > > Even FTP supports both ASCII and binary data transfer. > Handling both > > uni-directional data and bi-directional data in IPFIX > should be pretty > > easy stuff. > > > > > > Carter > > > > Carter Bullard > > QoSient, LLC > > 300 E. 56th Street, Suite 18K > > New York, New York 10022 > > > > carter@qosient.com > > Phone +1 212 588-9133 > > Fax +1 212 588-9134 > > http://qosient.com > > > > > > > > > > > -----Original Message----- > > > From: majordomo listserver > [mailto:majordomo@mil.doit.wisc.edu] On > > > Behalf Of Dave Plonka > > > Sent: Monday, January 07, 2002 4:52 PM > > > To: ipfix-req@net.doit.wisc.edu > > > Subject: [ipfix-req] unidirection vs. bidirectional flows > > > > > > > > > > > > IPFIX folks, > > > > > > As many of you recall, whether or not IPFIX should specify a > > > unidirectional or bidirectional flow definition was a hot > topic at > > > the WG meeting. > > > > > > I think that specifying *something* gets us far along in > > > communicating what a "flow" is. Personally, I would like us to > > > specify one or the other and I am in favor a unidirection flow > > > definition. > > > > > > Here are some thoughts about arguments made for > bidirectional flows: > > > > > > A unidirectional flow definition means that the atomic > flow report > > > record is unidirection during the reporting/export phase. For > > > instance, a single packet counter for a given > > > protocol/srcaddr/srcport/dstaddr/dstport n-tuple. However, since > > > the exporter is somewhat of a "black box", it need not be > > > constrained to a unidirectional model internally. > > > > > > A unidirectional flow definition does not prevent a collector > > > implementor from using a bi-directional notion of flows > internally > > > nor does it prevent it from being stored in a bidirectional way > > > prior to export. Conversely if we specify a bidirectional > > > definition, it does disqualify unidirectional implementations. > > > > > > For those that wish to perform measurements which may require > > > bidirectional flow information (perhaps because start/end > timestamp > > > synchronization is required for flow pairs), the IPFIX data model > > > and transport could define a mechanism by which the > exporter could > > > inform the collector about which unidirectional flow records are > > > matched pairs. That is, the exporter could help the > collector find > > > the matched pairs of flow information, perhaps by > exporting the flow > > > records in pairs or by generating a numeric identifier that is an > > > attribute of both flow records. Of course only bidirectional > > > collectors would understand this convention, but a > > > unidirection collector could ignore it and still make use of > > > the exported flow information from an exporter that > > > implemented this "bidirectional flow pairing" feature. > > > > > > Dave > > > > > > -- > > > ":) You will make many changes before settling satisfactorily. :)" > > > - the actual message (complete with smiley faces) that I > received > > > in a > > > fortune cookie at dinner after the IPFIX WG meeting > during IETF 52 > > > in Salt Lake City > > > > > > -- > > > Help mailto:majordomo@net.doit.wisc.edu and say "help" > > > in message body > > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe > > > ipfix" in message body > > > Archive http://ipfix.doit.wisc.edu/archive/ > > > > > > > > > > > > > > -- > > Help mailto:majordomo@net.doit.wisc.edu and say > "help" in message body > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe > > ipfix" in message body > > Archive http://ipfix.doit.wisc.edu/archive/ > > > > > -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Tue Jan 8 00:48:38 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id AAA08121 for ; Tue, 8 Jan 2002 00:48:38 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16NopO-0001Na-00 for ipfix-list@mil.doit.wisc.edu; Mon, 07 Jan 2002 23:29:38 -0600 Received: from c001-h001.c001.snv.cp.net ([209.228.32.115] helo=c001.snv.cp.net) by mil.doit.wisc.edu with smtp (Exim 3.13 #1) id 16NopK-0001My-00 for ipfix@net.doit.wisc.edu; Mon, 07 Jan 2002 23:29:34 -0600 Received: (cpmta 29170 invoked from network); 7 Jan 2002 21:28:57 -0800 Received: from 24.221.253.53 (HELO slc252750) by smtp.register-admin.com (209.228.32.115) with SMTP; 7 Jan 2002 21:28:57 -0800 X-Sent: 8 Jan 2002 05:28:57 GMT Message-ID: <001901c19805$a6ff6b60$850f880a@slc252750> Reply-To: "K.C. Norseth" From: "K.C. Norseth" To: "Ganesh Sadasivan" , "Simon Leinen" Cc: References: <20011214150236.A29404@doit.wisc.edu><3C1BDF61.B0BE8081@cisco.com> Subject: Re: [ipfix] IPFIX Requirement presentation Date: Mon, 7 Jan 2002 22:30:57 -0700 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.00.2615.200 X-MimeOLE: Produced By Microsoft MimeOLE V5.00.2615.200 Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit In the case of an etxternal probe where there is only one interface and the probe is hooked to a mirrored port or bridge, the place where the probe is connected, ther is no incomoing or outgoing interface. What would the interface be that would satisfy MUST? K.C. ----- Original Message ----- From: Simon Leinen To: Ganesh Sadasivan Cc: Sent: Sunday, January 06, 2002 5:56 AM Subject: Re: [ipfix] IPFIX Requirement presentation > >>>>> "gs" == Ganesh Sadasivan writes: > > I want to bring up this point which I had raised on the > > requirement spec. at the WG meeting. > > It is from slide 5 on requirements where the flow definition has > > been changed to include "incoming or outgoing interface" as a > > MUST. > > The corresponding part of the requirements draft > (ietf-ipfix-requirements-00) reads: > > 3.1. Interfaces > > The measuring device MUST be able to separate flows by the incoming > interface or by the outgoing interface or by both of them. > > Note that this says "MUST be able to separate", not "MUST separate". > Therefore I think your reasoning below doesn't apply: > > > There are cases like a SP is interested in getting all the > > packest that go between 2 ASs. This does not have any strict > > relevance to interface. An AS can be sourced by Multiple > > interfaces in a router. Similar would be case if somebody > > considers to define a flow based on other packet fields like > > src/dst IP etc. In the general scheme of flow defintion ,I do not > > see why there should be a restriction to include the interfaces > > as a MUST. Claiming this as "most of the apps need the interfaces > > today" is not a valid argument as this model is used to serve for > > future apps. too. Therefore I request the WG to consider > > removing this restriction. > > It is probably the case for *every* flow attribute that one can always > come up with a sensible application that doesn't care about it. So it > makes looks reasonable to make all flow attributes optional, in the > sense that when an application isn't interested in them, the exporter > doesn't have to send them in order to avoid waste of resources. This > could be a requirement for the Flow Information Export protocol. > > But it also makes sense to make the *ability* of reporting certain > attributes mandatory, so that applications can rely on them being > available from all IPFIX compliant exporters. > > Personally I find the input and output interface sufficiently > important to make their support mandatory. In particular the input > interface is essential when one wants to do analysis of large-scale > routing asymmetries. Input and output interfaces are also extremely > useful when one collects flows at different points in the networks and > wants to avoid counting the same flows twice. And the input interface > cannot be derived from the source address and routing information, as > can/must be done with e.g. the "source netmask", "upstream AS" or > "source AS". > > Regards, > -- > Simon Leinen simon@babar.switch.ch > SWITCH http://www.switch.ch/misc/leinen/ > > Computers hate being anthropomorphized. > > -- > Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ > -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Tue Jan 8 07:01:43 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id HAA19398 for ; Tue, 8 Jan 2002 07:01:43 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16NuMG-0002MN-00 for ipfix-list@mil.doit.wisc.edu; Tue, 08 Jan 2002 05:23:56 -0600 Received: from yamato.ccrle.nec.de ([195.37.70.1]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 16NuMD-0002LV-00 for ipfix-req@net.doit.wisc.edu; Tue, 08 Jan 2002 05:23:53 -0600 Received: from wallace.heidelberg.ccrle.nec.de (root@wallace [192.168.102.1]) by yamato.ccrle.nec.de (8.11.6/8.10.1) with ESMTP id g08BNiR07832; Tue, 8 Jan 2002 12:23:44 +0100 (CET) Received: from [192.168.102.164] (beta.heidelberg.ccrle.nec.de [192.168.102.164]) by wallace.heidelberg.ccrle.nec.de (8.9.3/8.9.3/SuSE Linux 8.9.3-0.1) with ESMTP id MAA25074; Tue, 8 Jan 2002 12:23:20 +0100 Date: Tue, 08 Jan 2002 12:24:24 +0100 From: Juergen Quittek To: carter@qosient.com, "'Vamsidhar Valluri'" cc: plonka@doit.wisc.edu, ipfix-req@net.doit.wisc.edu Subject: RE: [ipfix-req] unidirection vs. bidirectional flows Message-ID: <1175350028.1010492664@[192.168.102.164]> In-Reply-To: <5C8959A16A71B449AE793CF52FBBED66018FCC@ptah.newyork.qosient.com> X-Mailer: Mulberry/2.0.8 (Win32) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Hi Carter, One of the very important constraints we applied to each requirements so far, is that it must be possible to for routers as well as for probes to implement the respective requirement with reasonable effort. (If anyone finds out that one or more of the requirements appears to be expensive, then please speak up!) Facing the hardware acrchitecture of a modern router, I consider a bi-directional flow model to be too expensive to be part of the requirements (and consequently also too expensive to be part of the general architecture on the exporter side). The reason is that at line interface cards you typically spent very different effort on treatment on incoming and outgoing packets. For incoming packets you do a lot of processing: checksums, TTL processing, routing based on cached routing tables, ... Here you touch header fields anyway and it is relatively easy and relatively cheap to add metering functions. For outgoing packets the treatment is different. You just try to get rid of them as fast as possible (except from scheduling thme first). Here it would be rather expensive to investigate header fields again and it might also reduce your maximum thoughput. Therefore a device just metering incoming packets should be able to match the requirements. Now, if we have a bi-directional flow model, you require such a device to merge all its input measurements performed at different line cards by looking for measured uni-directional flows belongig to the same bi-directinal one. This task might be done at a control processor, but it creates additional cost and it might slow down the system. I believe that merging of these measurements is a perfect job for an analysis application and not for a meter. Definitely it is nothing we should ask every meter to perform by making it a MUST requirement! Consequently, my position is that IF we add a flow model to the requirements, then it should be the uni-diractional one. However, I am not yet convinced, that we should specify the model at all in the requirements. Best wishes, Juergen -- Juergen Quittek quittek@ccrle.nec.de Tel: +49 6221 90511-15 NEC Europe Ltd., Network Laboratories Fax: +49 6221 90511-55 Adenauerplatz 6, 69115 Heidelberg, Germany http://www.ccrle.nec.de --On 07 January 2002 19:42 -0500 Carter Bullard wrote: > Hey Vamsi, > Hmmm, is the interface a part of the flow defintion? > Not a problem. Lets look at how this may work. Suppose there > is a SRC interface and a DST interface, so that a flow is > simply all the packets moving from SRC -> DST. The return flow > would be all the packets moving from DST -> SRC. > > So the flow is reported as SRC <-> DST, with the simple > addition of the DST -> SRC metrics. Makes more sense than > independantly reporting two half-duplex flow records, which > could be skewed in time so badly, that the two records cannot > be correlated at all. > > > Carter > > Carter Bullard > QoSient, LLC > 300 E. 56th Street, Suite 18K > New York, New York 10022 > > carter@qosient.com > Phone +1 212 588-9133 > Fax +1 212 588-9134 > http://qosient.com > > >> -----Original Message----- >> From: Vamsidhar Valluri [mailto:vvalluri@cisco.com] >> Sent: Monday, January 07, 2002 6:58 PM >> To: Carter Bullard >> Cc: plonka@doit.wisc.edu; ipfix-req@net.doit.wisc.edu >> Subject: RE: [ipfix-req] unidirection vs. bidirectional flows >> >> >> >> >> On Mon, 7 Jan 2002, Carter Bullard wrote: >> >> > Gentle people, >> > There is absolutely no reason for IPFIX to limit the type >> > of flow data that it supports. The difference between >> uni-directional >> > and bi-directional flow records is simply the type of >> metrics that are >> > reported. The flow descriptors and keys are all the same. Because >> > IPFIX must support >> ^^^^^ >> >> If "keys" you are refering to are the fields used to identify >> a flow then "SRC interface" do not fit into the statement >> "The flow descriptors and keys are all same for both >> unidirectional and bi-directional flow records" mentioned >> above because SRC interface will make it unidirectional. >> >> -vamsi >> >> >> > reporting metrics that each vendors will not implement, such >> as jitter >> > and one-way delay, which are the few metrics that are meaningful in >> > uni-directional flow data models, supporting metrics that report on >> > return traffic should not be a hardship for IPFIX. >> > >> > But, I believe that if you are going to make a design >> choice it should >> > be motivated by design goals. What benefit do you gain by limiting >> > IPFIX to a simple uni-directional >> > flow model? Is a uni-directional flow data model the best >> > data model for reporting all the metrics and network behaviors that >> > relate to network flows? Is it the best data model for >> supporting all >> > the existing commercial network flow data? >> > >> > But what about even the simplest of applications of flow data? Is a >> > uni-directional flow data model the best solution for reporting the >> > RTT of a ping volley? The empirical bulk transfer >> properties of a TCP >> > connection? How about TCP packet retransmission? Existence >> of routing >> > failures in an OSPF routed network? Application response time? >> > Rerouting indications? Reachability fault detection? >> Access policy >> > validation? Connectivity status? >> > >> > Even FTP supports both ASCII and binary data transfer. >> Handling both >> > uni-directional data and bi-directional data in IPFIX >> should be pretty >> > easy stuff. >> > >> > >> > Carter >> > >> > Carter Bullard >> > QoSient, LLC >> > 300 E. 56th Street, Suite 18K >> > New York, New York 10022 >> > >> > carter@qosient.com >> > Phone +1 212 588-9133 >> > Fax +1 212 588-9134 >> > http://qosient.com >> > >> > >> > >> > >> > > -----Original Message----- >> > > From: majordomo listserver >> [mailto:majordomo@mil.doit.wisc.edu] On >> > > Behalf Of Dave Plonka >> > > Sent: Monday, January 07, 2002 4:52 PM >> > > To: ipfix-req@net.doit.wisc.edu >> > > Subject: [ipfix-req] unidirection vs. bidirectional flows >> > > >> > > >> > > >> > > IPFIX folks, >> > > >> > > As many of you recall, whether or not IPFIX should specify a >> > > unidirectional or bidirectional flow definition was a hot >> topic at >> > > the WG meeting. >> > > >> > > I think that specifying *something* gets us far along in >> > > communicating what a "flow" is. Personally, I would like us to >> > > specify one or the other and I am in favor a unidirection flow >> > > definition. >> > > >> > > Here are some thoughts about arguments made for >> bidirectional flows: >> > > >> > > A unidirectional flow definition means that the atomic >> flow report >> > > record is unidirection during the reporting/export phase. For >> > > instance, a single packet counter for a given >> > > protocol/srcaddr/srcport/dstaddr/dstport n-tuple. However, since >> > > the exporter is somewhat of a "black box", it need not be >> > > constrained to a unidirectional model internally. >> > > >> > > A unidirectional flow definition does not prevent a collector >> > > implementor from using a bi-directional notion of flows >> internally >> > > nor does it prevent it from being stored in a bidirectional way >> > > prior to export. Conversely if we specify a bidirectional >> > > definition, it does disqualify unidirectional implementations. >> > > >> > > For those that wish to perform measurements which may require >> > > bidirectional flow information (perhaps because start/end >> timestamp >> > > synchronization is required for flow pairs), the IPFIX data model >> > > and transport could define a mechanism by which the >> exporter could >> > > inform the collector about which unidirectional flow records are >> > > matched pairs. That is, the exporter could help the >> collector find >> > > the matched pairs of flow information, perhaps by >> exporting the flow >> > > records in pairs or by generating a numeric identifier that is an >> > > attribute of both flow records. Of course only bidirectional >> > > collectors would understand this convention, but a >> > > unidirection collector could ignore it and still make use of >> > > the exported flow information from an exporter that >> > > implemented this "bidirectional flow pairing" feature. >> > > >> > > Dave >> > > >> > > -- >> > > ":) You will make many changes before settling satisfactorily. :)" >> > > - the actual message (complete with smiley faces) that I >> received >> > > in a >> > > fortune cookie at dinner after the IPFIX WG meeting >> during IETF 52 >> > > in Salt Lake City >> > > >> > > -- >> > > Help mailto:majordomo@net.doit.wisc.edu and say "help" >> > > in message body >> > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say >> "unsubscribe >> > > ipfix" in message body >> > > Archive http://ipfix.doit.wisc.edu/archive/ >> > > >> > > >> > >> > >> > >> > -- >> > Help mailto:majordomo@net.doit.wisc.edu and say >> "help" in message body >> > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe >> > ipfix" in message body >> > Archive http://ipfix.doit.wisc.edu/archive/ >> > >> >> >> > > > > -- > Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ > -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Tue Jan 8 08:03:47 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id IAA20003 for ; Tue, 8 Jan 2002 08:03:46 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16NvT3-0004IA-00 for ipfix-list@mil.doit.wisc.edu; Tue, 08 Jan 2002 06:35:01 -0600 Received: from relay1.pair.com ([209.68.1.20] helo=relay.pair.com) by mil.doit.wisc.edu with smtp (Exim 3.13 #1) id 16NvT1-0004Hw-00 for ipfix-req@net.doit.wisc.edu; Tue, 08 Jan 2002 06:34:59 -0600 Received: (qmail 2320 invoked from network); 8 Jan 2002 12:34:57 -0000 Received: from pool-162-83-252-213.ny5030.east.verizon.net (HELO sphynx) (162.83.252.213) by relay1.pair.com with SMTP; 8 Jan 2002 12:34:57 -0000 X-pair-Authenticated: 162.83.252.213 Reply-To: From: "Carter Bullard" To: "'Juergen Quittek'" Cc: , Subject: RE: [ipfix-req] unidirection vs. bidirectional flows Date: Tue, 8 Jan 2002 07:34:47 -0500 Organization: QoSient, LLC Message-ID: <5C8959A16A71B449AE793CF52FBBED66018FCF@ptah.newyork.qosient.com> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook, Build 10.0.3416 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 In-Reply-To: <1175350028.1010492664@[192.168.102.164]> Importance: Normal Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Hey Juergen, Just a note, the majority of switches and routers today (> 90%) cannot support uni-directional flow record reporting. However, virtually all switches and routers (> 99%) support RMON-style interface statistics, which are L1/L2 bi-directional flow metrics. I believe that adoption of a uni-directional flow model excludes basically the entire population of legacy networking devices that could easily report their existing bi-directional flow metrics using IPFIX. Wouldn't this violate your fundamental constraint? Carter Carter Bullard QoSient, LLC 300 E. 56th Street, Suite 18K New York, New York 10022 carter@qosient.com Phone +1 212 588-9133 Fax +1 212 588-9134 http://qosient.com > -----Original Message----- > From: majordomo listserver > [mailto:majordomo@mil.doit.wisc.edu] On Behalf Of Juergen Quittek > Sent: Tuesday, January 08, 2002 6:24 AM > To: carter@qosient.com; 'Vamsidhar Valluri' > Cc: plonka@doit.wisc.edu; ipfix-req@net.doit.wisc.edu > Subject: RE: [ipfix-req] unidirection vs. bidirectional flows > > > Hi Carter, > > One of the very important constraints we applied to each > requirements so far, is that it must be possible to for > routers as well as for probes to implement the respective > requirement with reasonable effort. > > (If anyone finds out that one or more of the requirements > appears to be expensive, then please speak up!) > > Facing the hardware acrchitecture of a modern router, > I consider a bi-directional flow model to be too expensive > to be part of the requirements (and consequently also > too expensive to be part of the general architecture on > the exporter side). > > The reason is that at line interface cards you typically > spent very different effort on treatment on incoming and > outgoing packets. For incoming packets you do a lot of > processing: checksums, TTL processing, routing based on > cached routing tables, ... Here you touch header fields > anyway and it is relatively easy and relatively cheap to > add metering functions. For outgoing packets the treatment > is different. You just try to get rid of them as fast as > possible (except from scheduling thme first). Here it would > be rather expensive to investigate header fields again and > it might also reduce your maximum thoughput. Therefore a > device just metering incoming packets should be able to > match the requirements. > > Now, if we have a bi-directional flow model, you require > such a device to merge all its input measurements performed > at different line cards by looking for measured uni-directional > flows belongig to the same bi-directinal one. This task > might be done at a control processor, but it creates > additional cost and it might slow down the system. I believe > that merging of these measurements is a perfect job for an > analysis application and not for a meter. > > Definitely it is nothing we should ask every meter to perform > by making it a MUST requirement! > > Consequently, my position is that IF we add a flow model to > the requirements, then it should be the uni-diractional one. > However, I am not yet convinced, that we should specify the > model at all in the requirements. > > Best wishes, > > Juergen > -- > Juergen Quittek quittek@ccrle.nec.de Tel: +49 6221 90511-15 > NEC Europe Ltd., Network Laboratories Fax: +49 6221 90511-55 > Adenauerplatz 6, 69115 Heidelberg, Germany http://www.ccrle.nec.de > > > --On 07 January 2002 19:42 -0500 Carter Bullard > wrote: > > > Hey Vamsi, > > Hmmm, is the interface a part of the flow defintion? > > Not a problem. Lets look at how this may work. Suppose there > > is a SRC interface and a DST interface, so that a flow is > > simply all the packets moving from SRC -> DST. The return flow > > would be all the packets moving from DST -> SRC. > > > > So the flow is reported as SRC <-> DST, with the simple > > addition of the DST -> SRC metrics. Makes more sense than > > independantly reporting two half-duplex flow records, which > > could be skewed in time so badly, that the two records cannot > > be correlated at all. > > > > > > Carter > > > > Carter Bullard > > QoSient, LLC > > 300 E. 56th Street, Suite 18K > > New York, New York 10022 > > > > carter@qosient.com > > Phone +1 212 588-9133 > > Fax +1 212 588-9134 > > http://qosient.com > > > > > >> -----Original Message----- > >> From: Vamsidhar Valluri [mailto:vvalluri@cisco.com] > >> Sent: Monday, January 07, 2002 6:58 PM > >> To: Carter Bullard > >> Cc: plonka@doit.wisc.edu; ipfix-req@net.doit.wisc.edu > >> Subject: RE: [ipfix-req] unidirection vs. bidirectional flows > >> > >> > >> > >> > >> On Mon, 7 Jan 2002, Carter Bullard wrote: > >> > >> > Gentle people, > >> > There is absolutely no reason for IPFIX to limit the type > >> > of flow data that it supports. The difference between > >> uni-directional > >> > and bi-directional flow records is simply the type of > >> metrics that are > >> > reported. The flow descriptors and keys are all the > same. Because > >> > IPFIX must support > >> ^^^^^ > >> > >> If "keys" you are refering to are the fields used to identify > >> a flow then "SRC interface" do not fit into the statement > >> "The flow descriptors and keys are all same for both > >> unidirectional and bi-directional flow records" mentioned > >> above because SRC interface will make it unidirectional. > >> > >> -vamsi > >> > >> > >> > reporting metrics that each vendors will not implement, such > >> as jitter > >> > and one-way delay, which are the few metrics that are > meaningful in > >> > uni-directional flow data models, supporting metrics > that report on > >> > return traffic should not be a hardship for IPFIX. > >> > > >> > But, I believe that if you are going to make a design > >> choice it should > >> > be motivated by design goals. What benefit do you gain > by limiting > >> > IPFIX to a simple uni-directional > >> > flow model? Is a uni-directional flow data model the best > >> > data model for reporting all the metrics and network > behaviors that > >> > relate to network flows? Is it the best data model for > >> supporting all > >> > the existing commercial network flow data? > >> > > >> > But what about even the simplest of applications of flow > data? Is a > >> > uni-directional flow data model the best solution for > reporting the > >> > RTT of a ping volley? The empirical bulk transfer > >> properties of a TCP > >> > connection? How about TCP packet retransmission? Existence > >> of routing > >> > failures in an OSPF routed network? Application response time? > >> > Rerouting indications? Reachability fault detection? > >> Access policy > >> > validation? Connectivity status? > >> > > >> > Even FTP supports both ASCII and binary data transfer. > >> Handling both > >> > uni-directional data and bi-directional data in IPFIX > >> should be pretty > >> > easy stuff. > >> > > >> > > >> > Carter > >> > > >> > Carter Bullard > >> > QoSient, LLC > >> > 300 E. 56th Street, Suite 18K > >> > New York, New York 10022 > >> > > >> > carter@qosient.com > >> > Phone +1 212 588-9133 > >> > Fax +1 212 588-9134 > >> > http://qosient.com > >> > > >> > > >> > > >> > > >> > > -----Original Message----- > >> > > From: majordomo listserver > >> [mailto:majordomo@mil.doit.wisc.edu] On > >> > > Behalf Of Dave Plonka > >> > > Sent: Monday, January 07, 2002 4:52 PM > >> > > To: ipfix-req@net.doit.wisc.edu > >> > > Subject: [ipfix-req] unidirection vs. bidirectional flows > >> > > > >> > > > >> > > > >> > > IPFIX folks, > >> > > > >> > > As many of you recall, whether or not IPFIX should specify a > >> > > unidirectional or bidirectional flow definition was a hot > >> topic at > >> > > the WG meeting. > >> > > > >> > > I think that specifying *something* gets us far along in > >> > > communicating what a "flow" is. Personally, I would like us to > >> > > specify one or the other and I am in favor a unidirection flow > >> > > definition. > >> > > > >> > > Here are some thoughts about arguments made for > >> bidirectional flows: > >> > > > >> > > A unidirectional flow definition means that the atomic > >> flow report > >> > > record is unidirection during the reporting/export phase. For > >> > > instance, a single packet counter for a given > >> > > protocol/srcaddr/srcport/dstaddr/dstport n-tuple. > However, since > >> > > the exporter is somewhat of a "black box", it need not be > >> > > constrained to a unidirectional model internally. > >> > > > >> > > A unidirectional flow definition does not prevent a collector > >> > > implementor from using a bi-directional notion of flows > >> internally > >> > > nor does it prevent it from being stored in a bidirectional way > >> > > prior to export. Conversely if we specify a bidirectional > >> > > definition, it does disqualify unidirectional implementations. > >> > > > >> > > For those that wish to perform measurements which may require > >> > > bidirectional flow information (perhaps because start/end > >> timestamp > >> > > synchronization is required for flow pairs), the IPFIX > data model > >> > > and transport could define a mechanism by which the > >> exporter could > >> > > inform the collector about which unidirectional flow > records are > >> > > matched pairs. That is, the exporter could help the > >> collector find > >> > > the matched pairs of flow information, perhaps by > >> exporting the flow > >> > > records in pairs or by generating a numeric identifier > that is an > >> > > attribute of both flow records. Of course only bidirectional > >> > > collectors would understand this convention, but a > >> > > unidirection collector could ignore it and still make use of > >> > > the exported flow information from an exporter that > >> > > implemented this "bidirectional flow pairing" feature. > >> > > > >> > > Dave > >> > > > >> > > -- > >> > > ":) You will make many changes before settling > satisfactorily. :)" > >> > > - the actual message (complete with smiley faces) that I > >> received > >> > > in a > >> > > fortune cookie at dinner after the IPFIX WG meeting > >> during IETF 52 > >> > > in Salt Lake City > >> > > > >> > > -- > >> > > Help mailto:majordomo@net.doit.wisc.edu and say "help" > >> > > in message body > >> > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > >> "unsubscribe > >> > > ipfix" in message body > >> > > Archive http://ipfix.doit.wisc.edu/archive/ > >> > > > >> > > > >> > > >> > > >> > > >> > -- > >> > Help mailto:majordomo@net.doit.wisc.edu and say > >> "help" in message body > >> > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe > >> > ipfix" in message body > >> > Archive http://ipfix.doit.wisc.edu/archive/ > >> > > >> > >> > >> > > > > > > > > -- > > Help mailto:majordomo@net.doit.wisc.edu and say > "help" in message body > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > "unsubscribe ipfix" in message body > > Archive http://ipfix.doit.wisc.edu/archive/ > > > > > > -- > Help mailto:majordomo@net.doit.wisc.edu and say "help" > in message body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ > > -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Tue Jan 8 08:46:52 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id IAA20760 for ; Tue, 8 Jan 2002 08:46:52 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16NwCt-0005GC-00 for ipfix-list@mil.doit.wisc.edu; Tue, 08 Jan 2002 07:22:23 -0600 Received: from yamato.ccrle.nec.de ([195.37.70.1]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 16NwCr-0005FD-00 for ipfix-req@net.doit.wisc.edu; Tue, 08 Jan 2002 07:22:21 -0600 Received: from wallace.heidelberg.ccrle.nec.de (root@wallace [192.168.102.1]) by yamato.ccrle.nec.de (8.11.6/8.10.1) with ESMTP id g08DMDR08262; Tue, 8 Jan 2002 14:22:13 +0100 (CET) Received: from [192.168.102.164] (beta.heidelberg.ccrle.nec.de [192.168.102.164]) by wallace.heidelberg.ccrle.nec.de (8.9.3/8.9.3/SuSE Linux 8.9.3-0.1) with ESMTP id OAA26612; Tue, 8 Jan 2002 14:21:50 +0100 Date: Tue, 08 Jan 2002 14:22:53 +0100 From: Juergen Quittek To: carter@qosient.com cc: plonka@doit.wisc.edu, ipfix-req@net.doit.wisc.edu Subject: RE: [ipfix-req] unidirection vs. bidirectional flows Message-ID: <1182459298.1010499773@[192.168.102.164]> In-Reply-To: <5C8959A16A71B449AE793CF52FBBED66018FCF@ptah.newyork.qosient.com> X-Mailer: Mulberry/2.0.8 (Win32) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit --On 08 January 2002 07:34 -0500 Carter Bullard wrote: > Hey Juergen, > Just a note, the majority of switches and routers > today (> 90%) cannot support uni-directional flow record What routers are you talking about? Isn't NetFlow available for most CisCo and Juniper boxes? > reporting. However, virtually all switches and routers > (> 99%) support RMON-style interface statistics, which > are L1/L2 bi-directional flow metrics. > > I believe that adoption of a uni-directional flow model > excludes basically the entire population of legacy > networking devices that could easily report their existing > bi-directional flow metrics using IPFIX. > > Wouldn't this violate your fundamental constraint? Not at all. No one is trying to forbid existing technologies. But this is about IPFIX requirements. I said I am not sure whether we should specify uni- or bi-directional in the requirements at all, but I am sure I do not want to have "expensive" requirements. If you explain to me how to make bi-directional flows cheap, I will find it easier considering them as a potential requirement. Juergen -- Juergen Quittek quittek@ccrle.nec.de Tel: +49 6221 90511-15 NEC Europe Ltd., Network Laboratories Fax: +49 6221 90511-55 Adenauerplatz 6, 69115 Heidelberg, Germany http://www.ccrle.nec.de > > Carter > > Carter Bullard > QoSient, LLC > 300 E. 56th Street, Suite 18K > New York, New York 10022 > > carter@qosient.com > Phone +1 212 588-9133 > Fax +1 212 588-9134 > http://qosient.com > > >> -----Original Message----- >> From: majordomo listserver >> [mailto:majordomo@mil.doit.wisc.edu] On Behalf Of Juergen Quittek >> Sent: Tuesday, January 08, 2002 6:24 AM >> To: carter@qosient.com; 'Vamsidhar Valluri' >> Cc: plonka@doit.wisc.edu; ipfix-req@net.doit.wisc.edu >> Subject: RE: [ipfix-req] unidirection vs. bidirectional flows >> >> >> Hi Carter, >> >> One of the very important constraints we applied to each >> requirements so far, is that it must be possible to for >> routers as well as for probes to implement the respective >> requirement with reasonable effort. >> >> (If anyone finds out that one or more of the requirements >> appears to be expensive, then please speak up!) >> >> Facing the hardware acrchitecture of a modern router, >> I consider a bi-directional flow model to be too expensive >> to be part of the requirements (and consequently also >> too expensive to be part of the general architecture on >> the exporter side). >> >> The reason is that at line interface cards you typically >> spent very different effort on treatment on incoming and >> outgoing packets. For incoming packets you do a lot of >> processing: checksums, TTL processing, routing based on >> cached routing tables, ... Here you touch header fields >> anyway and it is relatively easy and relatively cheap to >> add metering functions. For outgoing packets the treatment >> is different. You just try to get rid of them as fast as >> possible (except from scheduling thme first). Here it would >> be rather expensive to investigate header fields again and >> it might also reduce your maximum thoughput. Therefore a >> device just metering incoming packets should be able to >> match the requirements. >> >> Now, if we have a bi-directional flow model, you require >> such a device to merge all its input measurements performed >> at different line cards by looking for measured uni-directional >> flows belongig to the same bi-directinal one. This task >> might be done at a control processor, but it creates >> additional cost and it might slow down the system. I believe >> that merging of these measurements is a perfect job for an >> analysis application and not for a meter. >> >> Definitely it is nothing we should ask every meter to perform >> by making it a MUST requirement! >> >> Consequently, my position is that IF we add a flow model to >> the requirements, then it should be the uni-diractional one. >> However, I am not yet convinced, that we should specify the >> model at all in the requirements. >> >> Best wishes, >> >> Juergen >> -- >> Juergen Quittek quittek@ccrle.nec.de Tel: +49 6221 90511-15 >> NEC Europe Ltd., Network Laboratories Fax: +49 6221 90511-55 >> Adenauerplatz 6, 69115 Heidelberg, Germany http://www.ccrle.nec.de >> >> >> --On 07 January 2002 19:42 -0500 Carter Bullard >> wrote: >> >> > Hey Vamsi, >> > Hmmm, is the interface a part of the flow defintion? >> > Not a problem. Lets look at how this may work. Suppose there >> > is a SRC interface and a DST interface, so that a flow is >> > simply all the packets moving from SRC -> DST. The return flow >> > would be all the packets moving from DST -> SRC. >> > >> > So the flow is reported as SRC <-> DST, with the simple >> > addition of the DST -> SRC metrics. Makes more sense than >> > independantly reporting two half-duplex flow records, which >> > could be skewed in time so badly, that the two records cannot >> > be correlated at all. >> > >> > >> > Carter >> > >> > Carter Bullard >> > QoSient, LLC >> > 300 E. 56th Street, Suite 18K >> > New York, New York 10022 >> > >> > carter@qosient.com >> > Phone +1 212 588-9133 >> > Fax +1 212 588-9134 >> > http://qosient.com >> > >> > >> >> -----Original Message----- >> >> From: Vamsidhar Valluri [mailto:vvalluri@cisco.com] >> >> Sent: Monday, January 07, 2002 6:58 PM >> >> To: Carter Bullard >> >> Cc: plonka@doit.wisc.edu; ipfix-req@net.doit.wisc.edu >> >> Subject: RE: [ipfix-req] unidirection vs. bidirectional flows >> >> >> >> >> >> >> >> >> >> On Mon, 7 Jan 2002, Carter Bullard wrote: >> >> >> >> > Gentle people, >> >> > There is absolutely no reason for IPFIX to limit the type >> >> > of flow data that it supports. The difference between >> >> uni-directional >> >> > and bi-directional flow records is simply the type of >> >> metrics that are >> >> > reported. The flow descriptors and keys are all the >> same. Because >> >> > IPFIX must support >> >> ^^^^^ >> >> >> >> If "keys" you are refering to are the fields used to identify >> >> a flow then "SRC interface" do not fit into the statement >> >> "The flow descriptors and keys are all same for both >> >> unidirectional and bi-directional flow records" mentioned >> >> above because SRC interface will make it unidirectional. >> >> >> >> -vamsi >> >> >> >> >> >> > reporting metrics that each vendors will not implement, such >> >> as jitter >> >> > and one-way delay, which are the few metrics that are >> meaningful in >> >> > uni-directional flow data models, supporting metrics >> that report on >> >> > return traffic should not be a hardship for IPFIX. >> >> > >> >> > But, I believe that if you are going to make a design >> >> choice it should >> >> > be motivated by design goals. What benefit do you gain >> by limiting >> >> > IPFIX to a simple uni-directional >> >> > flow model? Is a uni-directional flow data model the best >> >> > data model for reporting all the metrics and network >> behaviors that >> >> > relate to network flows? Is it the best data model for >> >> supporting all >> >> > the existing commercial network flow data? >> >> > >> >> > But what about even the simplest of applications of flow >> data? Is a >> >> > uni-directional flow data model the best solution for >> reporting the >> >> > RTT of a ping volley? The empirical bulk transfer >> >> properties of a TCP >> >> > connection? How about TCP packet retransmission? Existence >> >> of routing >> >> > failures in an OSPF routed network? Application response time? >> >> > Rerouting indications? Reachability fault detection? >> >> Access policy >> >> > validation? Connectivity status? >> >> > >> >> > Even FTP supports both ASCII and binary data transfer. >> >> Handling both >> >> > uni-directional data and bi-directional data in IPFIX >> >> should be pretty >> >> > easy stuff. >> >> > >> >> > >> >> > Carter >> >> > >> >> > Carter Bullard >> >> > QoSient, LLC >> >> > 300 E. 56th Street, Suite 18K >> >> > New York, New York 10022 >> >> > >> >> > carter@qosient.com >> >> > Phone +1 212 588-9133 >> >> > Fax +1 212 588-9134 >> >> > http://qosient.com >> >> > >> >> > >> >> > >> >> > >> >> > > -----Original Message----- >> >> > > From: majordomo listserver >> >> [mailto:majordomo@mil.doit.wisc.edu] On >> >> > > Behalf Of Dave Plonka >> >> > > Sent: Monday, January 07, 2002 4:52 PM >> >> > > To: ipfix-req@net.doit.wisc.edu >> >> > > Subject: [ipfix-req] unidirection vs. bidirectional flows >> >> > > >> >> > > >> >> > > >> >> > > IPFIX folks, >> >> > > >> >> > > As many of you recall, whether or not IPFIX should specify a >> >> > > unidirectional or bidirectional flow definition was a hot >> >> topic at >> >> > > the WG meeting. >> >> > > >> >> > > I think that specifying *something* gets us far along in >> >> > > communicating what a "flow" is. Personally, I would like us to >> >> > > specify one or the other and I am in favor a unidirection flow >> >> > > definition. >> >> > > >> >> > > Here are some thoughts about arguments made for >> >> bidirectional flows: >> >> > > >> >> > > A unidirectional flow definition means that the atomic >> >> flow report >> >> > > record is unidirection during the reporting/export phase. For >> >> > > instance, a single packet counter for a given >> >> > > protocol/srcaddr/srcport/dstaddr/dstport n-tuple. >> However, since >> >> > > the exporter is somewhat of a "black box", it need not be >> >> > > constrained to a unidirectional model internally. >> >> > > >> >> > > A unidirectional flow definition does not prevent a collector >> >> > > implementor from using a bi-directional notion of flows >> >> internally >> >> > > nor does it prevent it from being stored in a bidirectional way >> >> > > prior to export. Conversely if we specify a bidirectional >> >> > > definition, it does disqualify unidirectional implementations. >> >> > > >> >> > > For those that wish to perform measurements which may require >> >> > > bidirectional flow information (perhaps because start/end >> >> timestamp >> >> > > synchronization is required for flow pairs), the IPFIX >> data model >> >> > > and transport could define a mechanism by which the >> >> exporter could >> >> > > inform the collector about which unidirectional flow >> records are >> >> > > matched pairs. That is, the exporter could help the >> >> collector find >> >> > > the matched pairs of flow information, perhaps by >> >> exporting the flow >> >> > > records in pairs or by generating a numeric identifier >> that is an >> >> > > attribute of both flow records. Of course only bidirectional >> >> > > collectors would understand this convention, but a >> >> > > unidirection collector could ignore it and still make use of >> >> > > the exported flow information from an exporter that >> >> > > implemented this "bidirectional flow pairing" feature. >> >> > > >> >> > > Dave >> >> > > >> >> > > -- >> >> > > ":) You will make many changes before settling >> satisfactorily. :)" >> >> > > - the actual message (complete with smiley faces) that I >> >> received >> >> > > in a >> >> > > fortune cookie at dinner after the IPFIX WG meeting >> >> during IETF 52 >> >> > > in Salt Lake City >> >> > > >> >> > > -- >> >> > > Help mailto:majordomo@net.doit.wisc.edu and say "help" >> >> > > in message body >> >> > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say >> >> "unsubscribe >> >> > > ipfix" in message body >> >> > > Archive http://ipfix.doit.wisc.edu/archive/ >> >> > > >> >> > > >> >> > >> >> > >> >> > >> >> > -- >> >> > Help mailto:majordomo@net.doit.wisc.edu and say >> >> "help" in message body >> >> > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say >> "unsubscribe >> >> > ipfix" in message body >> >> > Archive http://ipfix.doit.wisc.edu/archive/ >> >> > >> >> >> >> >> >> >> > >> > >> > >> > -- >> > Help mailto:majordomo@net.doit.wisc.edu and say >> "help" in message body >> > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say >> > "unsubscribe ipfix" in message body >> > Archive http://ipfix.doit.wisc.edu/archive/ >> > >> >> >> >> -- >> Help mailto:majordomo@net.doit.wisc.edu and say "help" >> in message body >> Unsubscribe mailto:majordomo@net.doit.wisc.edu and say >> "unsubscribe ipfix" in message body >> Archive http://ipfix.doit.wisc.edu/archive/ >> >> > > > > -- > Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ > -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Tue Jan 8 10:28:02 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id KAA25340 for ; Tue, 8 Jan 2002 10:28:02 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16NxsZ-0007i5-00 for ipfix-list@mil.doit.wisc.edu; Tue, 08 Jan 2002 09:09:31 -0600 Received: from bru-cse-222.cisco.com ([144.254.8.48]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 16NxsX-0007hI-00 for ipfix-req@net.doit.wisc.edu; Tue, 08 Jan 2002 09:09:30 -0600 Received: (from bclaise@localhost) by bru-cse-222.cisco.com (8.10.2+Sun/CA/950118) id g08F8FI11407; Tue, 8 Jan 2002 16:08:15 +0100 (CET) Date: Tue, 8 Jan 2002 16:08:15 +0100 (CET) From: Benoit Claise Message-Id: <200201081508.g08F8FI11407@bru-cse-222.cisco.com> To: carter@qosient.com, quittek@ccrle.nec.de Subject: RE: [ipfix-req] unidirection vs. bidirectional flows Cc: plonka@doit.wisc.edu, ipfix-req@net.doit.wisc.edu X-Sun-Charset: US-ASCII Precedence: bulk Sender: majordomo listserver Carter and all, I'm not sure I understand the note below From the last WG meeting minutes, I can read: "We were reminded that this group was chartered to standardize existing practice, and that specifying new measurement techniques is out of scope." > > Just a note, the majority of switches and routers > > today (> 90%) cannot support uni-directional flow record > > What routers are you talking about? Isn't NetFlow available > for most CisCo and Juniper boxes? > > > reporting. However, virtually all switches and routers > > (> 99%) support RMON-style interface statistics, which > > are L1/L2 bi-directional flow metrics. > > > > I believe that adoption of a uni-directional flow model > > excludes basically the entire population of legacy > > networking devices that could easily report their existing > > bi-directional flow metrics using IPFIX. > > > > Wouldn't this violate your fundamental constraint? > > Not at all. No one is trying to forbid existing technologies. > But this is about IPFIX requirements. I said I am not sure > whether we should specify uni- or bi-directional in the > requirements at all, but I am sure I do not want to have > "expensive" requirements. If you explain to me how to make > bi-directional flows cheap, I will find it easier considering > them as a potential requirement From the requirements draft: 1.1 IP Flows There are several definitions of the term 'flow' being used by the Internet community. Within this document we use the following one: A flow is a set of packets passing an observation point in the network during a certain time interval. All packets belonging to a particular flow have a set of common properties derived from the data contained in the packet and from the packet treatment at the observation point. The observation point may be a network interface of a device or an entire router, a probe, or a middlebox with several interfaces. Properties derived from packet treatment include for example the interface at which the packet arrived, the interface the packet will be forwarded to and potentially some other information. If the observation point is interface 1 of line card 1 and the output interface interface the interface 1 of line card 2, this is extremely difficult (if not impossible) to correlate the flow going forth and the flow going back. Because most of the processing for a "ipfix like" protocol will be done on the incoming line card. As a conclusion, I think this only solution is that a flow is unidirectional. Regards, Benoit. > > Juergen > -- > Juergen Quittek quittek@ccrle.nec.de Tel: +49 6221 90511-15 > NEC Europe Ltd., Network Laboratories Fax: +49 6221 90511-55 > Adenauerplatz 6, 69115 Heidelberg, Germany http://www.ccrle.nec.de > > > > > Carter > > > > Carter Bullard > > QoSient, LLC > > 300 E. 56th Street, Suite 18K > > New York, New York 10022 > > > > carter@qosient.com > > Phone +1 212 588-9133 > > Fax +1 212 588-9134 > > http://qosient.com > > > > > >> -----Original Message----- > >> From: majordomo listserver > >> [mailto:majordomo@mil.doit.wisc.edu] On Behalf Of Juergen Quittek > >> Sent: Tuesday, January 08, 2002 6:24 AM > >> To: carter@qosient.com; 'Vamsidhar Valluri' > >> Cc: plonka@doit.wisc.edu; ipfix-req@net.doit.wisc.edu > >> Subject: RE: [ipfix-req] unidirection vs. bidirectional flows > >> > >> > >> Hi Carter, > >> > >> One of the very important constraints we applied to each > >> requirements so far, is that it must be possible to for > >> routers as well as for probes to implement the respective > >> requirement with reasonable effort. > >> > >> (If anyone finds out that one or more of the requirements > >> appears to be expensive, then please speak up!) > >> > >> Facing the hardware acrchitecture of a modern router, > >> I consider a bi-directional flow model to be too expensive > >> to be part of the requirements (and consequently also > >> too expensive to be part of the general architecture on > >> the exporter side). > >> > >> The reason is that at line interface cards you typically > >> spent very different effort on treatment on incoming and > >> outgoing packets. For incoming packets you do a lot of > >> processing: checksums, TTL processing, routing based on > >> cached routing tables, ... Here you touch header fields > >> anyway and it is relatively easy and relatively cheap to > >> add metering functions. For outgoing packets the treatment > >> is different. You just try to get rid of them as fast as > >> possible (except from scheduling thme first). Here it would > >> be rather expensive to investigate header fields again and > >> it might also reduce your maximum thoughput. Therefore a > >> device just metering incoming packets should be able to > >> match the requirements. > >> > >> Now, if we have a bi-directional flow model, you require > >> such a device to merge all its input measurements performed > >> at different line cards by looking for measured uni-directional > >> flows belongig to the same bi-directinal one. This task > >> might be done at a control processor, but it creates > >> additional cost and it might slow down the system. I believe > >> that merging of these measurements is a perfect job for an > >> analysis application and not for a meter. > >> > >> Definitely it is nothing we should ask every meter to perform > >> by making it a MUST requirement! > >> > >> Consequently, my position is that IF we add a flow model to > >> the requirements, then it should be the uni-diractional one. > >> However, I am not yet convinced, that we should specify the > >> model at all in the requirements. > >> > >> Best wishes, > >> > >> Juergen > >> -- > >> Juergen Quittek quittek@ccrle.nec.de Tel: +49 6221 90511-15 > >> NEC Europe Ltd., Network Laboratories Fax: +49 6221 90511-55 > >> Adenauerplatz 6, 69115 Heidelberg, Germany http://www.ccrle.nec.de > >> > >> > >> --On 07 January 2002 19:42 -0500 Carter Bullard > >> wrote: > >> > >> > Hey Vamsi, > >> > Hmmm, is the interface a part of the flow defintion? > >> > Not a problem. Lets look at how this may work. Suppose there > >> > is a SRC interface and a DST interface, so that a flow is > >> > simply all the packets moving from SRC -> DST. The return flow > >> > would be all the packets moving from DST -> SRC. > >> > > >> > So the flow is reported as SRC <-> DST, with the simple > >> > addition of the DST -> SRC metrics. Makes more sense than > >> > independantly reporting two half-duplex flow records, which > >> > could be skewed in time so badly, that the two records cannot > >> > be correlated at all. > >> > > >> > > >> > Carter > >> > > >> > Carter Bullard > >> > QoSient, LLC > >> > 300 E. 56th Street, Suite 18K > >> > New York, New York 10022 > >> > > >> > carter@qosient.com > >> > Phone +1 212 588-9133 > >> > Fax +1 212 588-9134 > >> > http://qosient.com > >> > > >> > > >> >> -----Original Message----- > >> >> From: Vamsidhar Valluri [mailto:vvalluri@cisco.com] > >> >> Sent: Monday, January 07, 2002 6:58 PM > >> >> To: Carter Bullard > >> >> Cc: plonka@doit.wisc.edu; ipfix-req@net.doit.wisc.edu > >> >> Subject: RE: [ipfix-req] unidirection vs. bidirectional flows > >> >> > >> >> > >> >> > >> >> > >> >> On Mon, 7 Jan 2002, Carter Bullard wrote: > >> >> > >> >> > Gentle people, > >> >> > There is absolutely no reason for IPFIX to limit the type > >> >> > of flow data that it supports. The difference between > >> >> uni-directional > >> >> > and bi-directional flow records is simply the type of > >> >> metrics that are > >> >> > reported. The flow descriptors and keys are all the > >> same. Because > >> >> > IPFIX must support > >> >> ^^^^^ > >> >> > >> >> If "keys" you are refering to are the fields used to identify > >> >> a flow then "SRC interface" do not fit into the statement > >> >> "The flow descriptors and keys are all same for both > >> >> unidirectional and bi-directional flow records" mentioned > >> >> above because SRC interface will make it unidirectional. > >> >> > >> >> -vamsi > >> >> > >> >> > >> >> > reporting metrics that each vendors will not implement, such > >> >> as jitter > >> >> > and one-way delay, which are the few metrics that are > >> meaningful in > >> >> > uni-directional flow data models, supporting metrics > >> that report on > >> >> > return traffic should not be a hardship for IPFIX. > >> >> > > >> >> > But, I believe that if you are going to make a design > >> >> choice it should > >> >> > be motivated by design goals. What benefit do you gain > >> by limiting > >> >> > IPFIX to a simple uni-directional > >> >> > flow model? Is a uni-directional flow data model the best > >> >> > data model for reporting all the metrics and network > >> behaviors that > >> >> > relate to network flows? Is it the best data model for > >> >> supporting all > >> >> > the existing commercial network flow data? > >> >> > > >> >> > But what about even the simplest of applications of flow > >> data? Is a > >> >> > uni-directional flow data model the best solution for > >> reporting the > >> >> > RTT of a ping volley? The empirical bulk transfer > >> >> properties of a TCP > >> >> > connection? How about TCP packet retransmission? Existence > >> >> of routing > >> >> > failures in an OSPF routed network? Application response time? > >> >> > Rerouting indications? Reachability fault detection? > >> >> Access policy > >> >> > validation? Connectivity status? > >> >> > > >> >> > Even FTP supports both ASCII and binary data transfer. > >> >> Handling both > >> >> > uni-directional data and bi-directional data in IPFIX > >> >> should be pretty > >> >> > easy stuff. > >> >> > > >> >> > > >> >> > Carter > >> >> > > >> >> > Carter Bullard > >> >> > QoSient, LLC > >> >> > 300 E. 56th Street, Suite 18K > >> >> > New York, New York 10022 > >> >> > > >> >> > carter@qosient.com > >> >> > Phone +1 212 588-9133 > >> >> > Fax +1 212 588-9134 > >> >> > http://qosient.com > >> >> > > >> >> > > >> >> > > >> >> > > >> >> > > -----Original Message----- > >> >> > > From: majordomo listserver > >> >> [mailto:majordomo@mil.doit.wisc.edu] On > >> >> > > Behalf Of Dave Plonka > >> >> > > Sent: Monday, January 07, 2002 4:52 PM > >> >> > > To: ipfix-req@net.doit.wisc.edu > >> >> > > Subject: [ipfix-req] unidirection vs. bidirectional flows > >> >> > > > >> >> > > > >> >> > > > >> >> > > IPFIX folks, > >> >> > > > >> >> > > As many of you recall, whether or not IPFIX should specify a > >> >> > > unidirectional or bidirectional flow definition was a hot > >> >> topic at > >> >> > > the WG meeting. > >> >> > > > >> >> > > I think that specifying *something* gets us far along in > >> >> > > communicating what a "flow" is. Personally, I would like us to > >> >> > > specify one or the other and I am in favor a unidirection flow > >> >> > > definition. > >> >> > > > >> >> > > Here are some thoughts about arguments made for > >> >> bidirectional flows: > >> >> > > > >> >> > > A unidirectional flow definition means that the atomic > >> >> flow report > >> >> > > record is unidirection during the reporting/export phase. For > >> >> > > instance, a single packet counter for a given > >> >> > > protocol/srcaddr/srcport/dstaddr/dstport n-tuple. > >> However, since > >> >> > > the exporter is somewhat of a "black box", it need not be > >> >> > > constrained to a unidirectional model internally. > >> >> > > > >> >> > > A unidirectional flow definition does not prevent a collector > >> >> > > implementor from using a bi-directional notion of flows > >> >> internally > >> >> > > nor does it prevent it from being stored in a bidirectional way > >> >> > > prior to export. Conversely if we specify a bidirectional > >> >> > > definition, it does disqualify unidirectional implementations. > >> >> > > > >> >> > > For those that wish to perform measurements which may require > >> >> > > bidirectional flow information (perhaps because start/end > >> >> timestamp > >> >> > > synchronization is required for flow pairs), the IPFIX > >> data model > >> >> > > and transport could define a mechanism by which the > >> >> exporter could > >> >> > > inform the collector about which unidirectional flow > >> records are > >> >> > > matched pairs. That is, the exporter could help the > >> >> collector find > >> >> > > the matched pairs of flow information, perhaps by > >> >> exporting the flow > >> >> > > records in pairs or by generating a numeric identifier > >> that is an > >> >> > > attribute of both flow records. Of course only bidirectional > >> >> > > collectors would understand this convention, but a > >> >> > > unidirection collector could ignore it and still make use of > >> >> > > the exported flow information from an exporter that > >> >> > > implemented this "bidirectional flow pairing" feature. > >> >> > > > >> >> > > Dave > >> >> > > > >> >> > > -- > >> >> > > ":) You will make many changes before settling > >> satisfactorily. :)" > >> >> > > - the actual message (complete with smiley faces) that I > >> >> received > >> >> > > in a > >> >> > > fortune cookie at dinner after the IPFIX WG meeting > >> >> during IETF 52 > >> >> > > in Salt Lake City > >> >> > > > >> >> > > -- > >> >> > > Help mailto:majordomo@net.doit.wisc.edu and say "help" > >> >> > > in message body > >> >> > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > >> >> "unsubscribe > >> >> > > ipfix" in message body > >> >> > > Archive http://ipfix.doit.wisc.edu/archive/ > >> >> > > > >> >> > > > >> >> > > >> >> > > >> >> > > >> >> > -- > >> >> > Help mailto:majordomo@net.doit.wisc.edu and say > >> >> "help" in message body > >> >> > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > >> "unsubscribe > >> >> > ipfix" in message body > >> >> > Archive http://ipfix.doit.wisc.edu/archive/ > >> >> > > >> >> > >> >> > >> >> > >> > > >> > > >> > > >> > -- > >> > Help mailto:majordomo@net.doit.wisc.edu and say > >> "help" in message body > >> > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > >> > "unsubscribe ipfix" in message body > >> > Archive http://ipfix.doit.wisc.edu/archive/ > >> > > >> > >> > >> > >> -- > >> Help mailto:majordomo@net.doit.wisc.edu and say "help" > >> in message body > >> Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > >> "unsubscribe ipfix" in message body > >> Archive http://ipfix.doit.wisc.edu/archive/ > >> > >> > > > > > > > > -- > > Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > "unsubscribe ipfix" in message body > > Archive http://ipfix.doit.wisc.edu/archive/ > > > > > > -- > Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ > -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Tue Jan 8 10:52:02 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id KAA26488 for ; Tue, 8 Jan 2002 10:52:02 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16NyE7-0000Jy-00 for ipfix-list@mil.doit.wisc.edu; Tue, 08 Jan 2002 09:31:47 -0600 Received: from zrc2s0jx.nortelnetworks.com ([47.103.122.112]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 16NyE4-0000JC-00 for ipfix-arch@net.doit.wisc.edu; Tue, 08 Jan 2002 09:31:44 -0600 Received: from zsc4c000.us.nortel.com (zsc4c000.us.nortel.com [47.81.138.47]) by zrc2s0jx.nortelnetworks.com (Switch-2.2.0/Switch-2.2.0) with ESMTP id g08FV0019442; Tue, 8 Jan 2002 09:31:00 -0600 (CST) Received: by zsc4c000.us.nortel.com with Internet Mail Service (5.5.2653.19) id ; Tue, 8 Jan 2002 07:29:10 -0800 Message-ID: <4F973E944951D311B6660008C7917CF0083E0758@zsc4c008.us.nortel.com> From: "Reinaldo Penno" To: "'Ganesh Sadasivan'" Cc: "'ipfix-arch@net.doit.wisc.edu'" Subject: RE: [ipfix-arch] configuration information. Date: Tue, 8 Jan 2002 07:29:09 -0800 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01C19859.37D14F20" Precedence: bulk Sender: majordomo listserver This message is in MIME format. Since your mail reader does not understand this format, some or all of this message may not be legible. ------_=_NextPart_001_01C19859.37D14F20 Content-Type: text/plain; charset="iso-8859-1" -----Original Message----- From: Ganesh Sadasivan [mailto:gsadasiv@cisco.com] Sent: Monday, January 07, 2002 1:37 PM To: Penno, Reinaldo [SC9:T327:EXCH] Cc: 'ipfix-arch@net.doit.wisc.edu' Subject: Re: [ipfix-arch] configuration information. Reinaldo, I agree with your point that it is non-trivial to derive the complete relationship between the packets collected various templates. My question to the group is that whether IPFIX should spend time on architecting how this detailed config information is sent to the collector so that the collector gets this information inband or whether this can be dealt by the collector in an adhoc manner. Hello Ganesh, okay, so let me rephrase my comment. Unless someone has some solution in its sleeve to do this automatically my vote would be not to spend time on this now since we might come with a solution to pass this info to the collector but there is no practical use. I do not know how a router/switch/device would derive the relationship unless it's based on (a lot of) manual configuration. In my previous example you would have to query ARIN databases, find the networks assigned to a specific AS, see if they have ranges in common to the first template (in specific directions), etc, etc. thanks, Reinaldo If we go by the former case to what level of details should the collector be made aware of about templates, packet selection and their relationships. Thanks Ganesh Reinaldo Penno wrote: Hello, How a collector would know that a priori? I mean, I can have such general templates that it would be very difficult to account for duplicate measurements (even in non real-time). If you have very specific templates such as srcIP, destIP, etc, it's not to hard find records in common. But you can have things like: src_AnyIP Dest_Yahoo.com Proto_Any and src_AnyIP Dest_AS3462 Proto_HTTP. How do you know a priori that the second rule will have records that are a subset of the first? Am I missing something? thanks, Reinaldo > -----Original Message----- > From: Ganesh Sadasivan [ mailto:gsadasiv@cisco.com ] > Sent: Monday, January 07, 2002 12:09 PM > To: ipfix-arch@net.doit.wisc.edu > Subject: [ipfix-arch] configuration information. > > > Hi, > One of the questions raised during the V9 presenatation if IETF was > "how a collector would know whether or not the same > packets or bytes were being reported in more than one flow record > (e.g. using a different template)." (quoting from the minutes). > This requires the collector to have configuration information > regading > the different templates, and the mode of selection of packets for > each > of these templates at each of the observation points. > This information can be either > 1. Communicated to the collector from the measuring device > inband as > a part > of the export protocol. > 2. The collector gets the required configuration out of band (.i.e > not thru > the export protocol. > > What are the thoughts of the group on this topic? > If we are going by 1., should it be a part of the arch- spec? One > think we > need to keep in mind is that things like , details on method of > packet > selection could vary widely with implementations. > > Thanks > Ganesh > > > > -- > Help mailto:majordomo@net.doit.wisc.edu and say "help" > in message body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ > ------_=_NextPart_001_01C19859.37D14F20 Content-Type: text/html; charset="iso-8859-1"
 
-----Original Message-----
From: Ganesh Sadasivan [mailto:gsadasiv@cisco.com]
Sent: Monday, January 07, 2002 1:37 PM
To: Penno, Reinaldo [SC9:T327:EXCH]
Cc: 'ipfix-arch@net.doit.wisc.edu'
Subject: Re: [ipfix-arch] configuration information.

Reinaldo,
I agree with your point that it is non-trivial to derive the complete relationship
between the packets collected various templates. My question to the group is that
whether IPFIX should spend time on architecting how this detailed config
information is sent to the collector so that the collector gets this information inband
or whether this can be dealt by the collector in an adhoc manner.  
 
Hello Ganesh,
 
okay, so let me rephrase my comment. Unless someone has some solution in its sleeve to do this automatically my vote would be not to spend time on this now since we might come with a solution to pass this info to the collector but there is no practical use. I do not know how a router/switch/device would derive the relationship unless it's based on (a lot of) manual configuration.
 
In my previous example you would have to query ARIN databases, find the networks assigned to a specific AS, see if they have ranges in common to the first template (in specific directions), etc, etc.
 
thanks,
 
Reinaldo
 
If we go by the former case to what level of details should the collector be
made aware of  about templates, packet selection and their relationships.

Thanks
Ganesh

Reinaldo Penno wrote:

 

Hello,

How a collector would know that a priori? I mean, I can have such general templates that it would be very difficult to account for duplicate measurements (even in non real-time).

If you have very specific templates such as srcIP, destIP, etc, it's not to hard find records in common. But you can have things like:

src_AnyIP   Dest_Yahoo.com    Proto_Any   and
src_AnyIP   Dest_AS3462       Proto_HTTP.

How do you know a priori that the second rule will have records that are a subset of the first? Am I missing something?

thanks,

Reinaldo
 

> -----Original Message-----
> From: Ganesh Sadasivan [mailto:gsadasiv@cisco.com]
> Sent: Monday, January 07, 2002 12:09 PM
> To: ipfix-arch@net.doit.wisc.edu
> Subject: [ipfix-arch] configuration information.
>
>
> Hi,
>    One of the questions raised during the V9 presenatation if IETF was
>    "how a collector would know whether or not the same
>    packets or bytes were being reported in more than one flow record
>    (e.g. using a different template)." (quoting from the minutes).
>    This requires the collector to have configuration information
> regading
>    the different templates, and the  mode of  selection of packets for
> each
>    of these templates at each of the observation points.
>    This information can be either
>    1. Communicated to the collector from the measuring device
> inband as
> a part
>    of the export protocol.
>    2. The collector gets the required configuration out of band (.i.e
> not thru
>    the export protocol.
>
>    What are the thoughts of the group on this topic?
>    If we are going by 1., should it be a part of the arch- spec? One
> think we
>    need to keep in mind is that things like , details on method of
> packet
>    selection could vary widely with implementations.
>
> Thanks
> Ganesh
>
>
>
> --
> Help        mailto:majordomo@net.doit.wisc.edu and say "help"
> in message body
> Unsubscribe mailto:majordomo@net.doit.wisc.edu and say
> "unsubscribe ipfix" in message body
> Archive     http://ipfix.doit.wisc.edu/archive/
>

------_=_NextPart_001_01C19859.37D14F20-- -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Tue Jan 8 11:08:34 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA27353 for ; Tue, 8 Jan 2002 11:08:34 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16NyV8-0000hS-00 for ipfix-list@mil.doit.wisc.edu; Tue, 08 Jan 2002 09:49:22 -0600 Received: from host3.riverstonenet.com ([63.113.148.3] helo=exc-sc1.yagosys.com) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 16NyV7-0000hG-00 for ipfix@net.doit.wisc.edu; Tue, 08 Jan 2002 09:49:21 -0600 Received: from riverstonenet.com (134.141.180.100 [134.141.180.100]) by exc-sc1.yagosys.com with SMTP (Microsoft Exchange Internet Mail Service Version 5.5.2653.13) id X7WVYKLW; Tue, 8 Jan 2002 07:48:18 -0800 Message-ID: <3C3B14C7.E398C708@riverstonenet.com> Date: Tue, 08 Jan 2002 10:48:23 -0500 From: calato@riverstonenet.com X-Mailer: Mozilla 4.75C-CCK-MCD NSCPCD475 [en] (X11; U; SunOS 5.6 sun4u) X-Accept-Language: en MIME-Version: 1.0 To: "ipfix@net.doit.wisc.edu" Subject: Re: [ipfix] security requirement for IPFIX protocol References: <4652644B98DFF34696801F8F3070D3FE0118855E@D2CSPEXM001.smartpipes.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit "Wang, Cliff" wrote: > > In draft , Sec 5.3.3. Security > > Confidentiallity of transferred IPFIX data SHOULD be ensured. > > Integrity of transferred IPFIX data MUST be ensured. > > Authenticity of transferred IPFIX data MUST be ensured. > > > Since we have started on the Architecture doc, I would like to further > discuss the possible transport security solutions. > At this point we potentially have several options: > 1) The IPFIX protocol doesn't specify any security mechanism within > the protocol. If security is desired > (authentication/encryption/integrity), use the protection mechanism > provided by other IETF WG solutions such as IPsec or TLS. > 2) The IPFIX protocol specify the protection mechanism. Then the > question will be at what level? At the data record level or at the > packet level? Authentication only or encryption as well? If you look at the Internet draft for LFAP it has an extensive section on security. We could use that as a **starting** point. In a nutshell it says use IPsec where possible. But also includes a lightweight way to transport the data securely via the protocol itself. It also describes the various attacks that are intended to be prevented with the lightweight version. > > > Also Section 8 discusses some security considerations. Client (probe) > to server (collection station) host level mutual authentication has > not been fully discussed. I think this should fit into the WG charter: > "Identify and address any security privacy concerns affecting flow > data. Determine technology for securing the flow information > export data, e.g. TLS. ". > > What in my mind is that in order to secure IPFIX data, you got to > provide both host security (where data is stored) and transport > security (where data is sent). However, we should only try to provide > requirement and use existing solutions rather than building solutions > from ground up. I don't think protection of data stored on the server or the client is in our charter. We only need to deal with transporting the data securely. > > > > > > > > > > -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Tue Jan 8 11:12:00 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA27494 for ; Tue, 8 Jan 2002 11:12:00 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16NyX7-0000io-00 for ipfix-list@mil.doit.wisc.edu; Tue, 08 Jan 2002 09:51:25 -0600 Received: from relay1.pair.com ([209.68.1.20] helo=relay.pair.com) by mil.doit.wisc.edu with smtp (Exim 3.13 #1) id 16NyX6-0000ij-00 for ipfix-req@net.doit.wisc.edu; Tue, 08 Jan 2002 09:51:24 -0600 Received: (qmail 25804 invoked from network); 8 Jan 2002 15:51:22 -0000 Received: from pool-162-83-252-213.ny5030.east.verizon.net (HELO sphynx) (162.83.252.213) by relay1.pair.com with SMTP; 8 Jan 2002 15:51:22 -0000 X-pair-Authenticated: 162.83.252.213 Reply-To: From: "Carter Bullard" To: "'Juergen Quittek'" Cc: , Subject: RE: [ipfix-req] unidirection vs. bidirectional flows Date: Tue, 8 Jan 2002 10:51:13 -0500 Organization: QoSient, LLC Message-ID: <5C8959A16A71B449AE793CF52FBBED6603BE17@ptah.newyork.qosient.com> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook, Build 10.0.3416 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 In-Reply-To: <1182459298.1010499773@[192.168.102.164]> Importance: Normal Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Hey Juergen, You know, you have to buy special cards to get netflow support in a Cisco router. I'm interested in how your going to make uni-directional flow record reporting cheap. Carter Carter Bullard QoSient, LLC 300 E. 56th Street, Suite 18K New York, New York 10022 carter@qosient.com Phone +1 212 588-9133 Fax +1 212 588-9134 http://qosient.com > -----Original Message----- > From: Juergen Quittek [mailto:quittek@ccrle.nec.de] > Sent: Tuesday, January 08, 2002 8:23 AM > To: carter@qosient.com > Cc: plonka@doit.wisc.edu; ipfix-req@net.doit.wisc.edu > Subject: RE: [ipfix-req] unidirection vs. bidirectional flows > > > --On 08 January 2002 07:34 -0500 Carter Bullard wrote: > > > Hey Juergen, > > Just a note, the majority of switches and routers > > today (> 90%) cannot support uni-directional flow record > > What routers are you talking about? Isn't NetFlow available > for most CisCo and Juniper boxes? > > > reporting. However, virtually all switches and routers > > (> 99%) support RMON-style interface statistics, which > > are L1/L2 bi-directional flow metrics. > > > > I believe that adoption of a uni-directional flow model excludes > > basically the entire population of legacy networking devices that > > could easily report their existing bi-directional flow > metrics using > > IPFIX. > > > > Wouldn't this violate your fundamental constraint? > > Not at all. No one is trying to forbid existing technologies. > But this is about IPFIX requirements. I said I am not sure > whether we should specify uni- or bi-directional in the > requirements at all, but I am sure I do not want to have > "expensive" requirements. If you explain to me how to make > bi-directional flows cheap, I will find it easier considering > them as a potential requirement. > > Juergen > -- > Juergen Quittek quittek@ccrle.nec.de Tel: +49 6221 90511-15 > NEC Europe Ltd., Network Laboratories Fax: +49 6221 90511-55 > Adenauerplatz 6, 69115 Heidelberg, Germany http://www.ccrle.nec.de > > > > > Carter > > > > Carter Bullard > > QoSient, LLC > > 300 E. 56th Street, Suite 18K > > New York, New York 10022 > > > > carter@qosient.com > > Phone +1 212 588-9133 > > Fax +1 212 588-9134 > > http://qosient.com > > > > > >> -----Original Message----- > >> From: majordomo listserver [mailto:majordomo@mil.doit.wisc.edu] On > >> Behalf Of Juergen Quittek > >> Sent: Tuesday, January 08, 2002 6:24 AM > >> To: carter@qosient.com; 'Vamsidhar Valluri' > >> Cc: plonka@doit.wisc.edu; ipfix-req@net.doit.wisc.edu > >> Subject: RE: [ipfix-req] unidirection vs. bidirectional flows > >> > >> > >> Hi Carter, > >> > >> One of the very important constraints we applied to each > requirements > >> so far, is that it must be possible to for routers as well as for > >> probes to implement the respective requirement with reasonable > >> effort. > >> > >> (If anyone finds out that one or more of the requirements > appears to > >> be expensive, then please speak up!) > >> > >> Facing the hardware acrchitecture of a modern router, > >> I consider a bi-directional flow model to be too expensive > to be part > >> of the requirements (and consequently also too expensive > to be part > >> of the general architecture on the exporter side). > >> > >> The reason is that at line interface cards you typically > spent very > >> different effort on treatment on incoming and outgoing > packets. For > >> incoming packets you do a lot of > >> processing: checksums, TTL processing, routing based on cached > >> routing tables, ... Here you touch header fields anyway and it is > >> relatively easy and relatively cheap to add metering > functions. For > >> outgoing packets the treatment is different. You just try > to get rid > >> of them as fast as possible (except from scheduling thme > first). Here > >> it would be rather expensive to investigate header fields again and > >> it might also reduce your maximum thoughput. Therefore a > >> device just metering incoming packets should be able to > >> match the requirements. > >> > >> Now, if we have a bi-directional flow model, you require such a > >> device to merge all its input measurements performed at different > >> line cards by looking for measured uni-directional flows > belongig to > >> the same bi-directinal one. This task might be done at a control > >> processor, but it creates additional cost and it might > slow down the > >> system. I believe that merging of these measurements is a > perfect job > >> for an analysis application and not for a meter. > >> > >> Definitely it is nothing we should ask every meter to perform by > >> making it a MUST requirement! > >> > >> Consequently, my position is that IF we add a flow model to the > >> requirements, then it should be the uni-diractional one. > However, I > >> am not yet convinced, that we should specify the model at > all in the > >> requirements. > >> > >> Best wishes, > >> > >> Juergen > >> -- > >> Juergen Quittek quittek@ccrle.nec.de Tel: +49 6221 90511-15 > >> NEC Europe Ltd., Network Laboratories Fax: +49 6221 90511-55 > >> Adenauerplatz 6, 69115 Heidelberg, Germany > http://www.ccrle.nec.de > >> > >> > >> --On 07 January 2002 19:42 -0500 Carter Bullard > > >> wrote: > >> > >> > Hey Vamsi, > >> > Hmmm, is the interface a part of the flow defintion? Not a > >> > problem. Lets look at how this may work. Suppose there > is a SRC > >> > interface and a DST interface, so that a flow is simply all the > >> > packets moving from SRC -> DST. The return flow would > be all the > >> > packets moving from DST -> SRC. > >> > > >> > So the flow is reported as SRC <-> DST, with the > simple addition > >> > of the DST -> SRC metrics. Makes more sense than independantly > >> > reporting two half-duplex flow records, which could be skewed in > >> > time so badly, that the two records cannot be correlated at all. > >> > > >> > > >> > Carter > >> > > >> > Carter Bullard > >> > QoSient, LLC > >> > 300 E. 56th Street, Suite 18K > >> > New York, New York 10022 > >> > > >> > carter@qosient.com > >> > Phone +1 212 588-9133 > >> > Fax +1 212 588-9134 > >> > http://qosient.com > >> > > >> > > >> >> -----Original Message----- > >> >> From: Vamsidhar Valluri [mailto:vvalluri@cisco.com] > >> >> Sent: Monday, January 07, 2002 6:58 PM > >> >> To: Carter Bullard > >> >> Cc: plonka@doit.wisc.edu; ipfix-req@net.doit.wisc.edu > >> >> Subject: RE: [ipfix-req] unidirection vs. bidirectional flows > >> >> > >> >> > >> >> > >> >> > >> >> On Mon, 7 Jan 2002, Carter Bullard wrote: > >> >> > >> >> > Gentle people, > >> >> > There is absolutely no reason for IPFIX to limit the type of > >> >> > flow data that it supports. The difference between > >> >> uni-directional > >> >> > and bi-directional flow records is simply the type of > >> >> metrics that are > >> >> > reported. The flow descriptors and keys are all the > >> same. Because > >> >> > IPFIX must support > >> >> ^^^^^ > >> >> > >> >> If "keys" you are refering to are the fields used to identify a > >> >> flow then "SRC interface" do not fit into the statement > "The flow > >> >> descriptors and keys are all same for both unidirectional and > >> >> bi-directional flow records" mentioned above because > SRC interface > >> >> will make it unidirectional. > >> >> > >> >> -vamsi > >> >> > >> >> > >> >> > reporting metrics that each vendors will not implement, such > >> >> as jitter > >> >> > and one-way delay, which are the few metrics that are > >> meaningful in > >> >> > uni-directional flow data models, supporting metrics > >> that report on > >> >> > return traffic should not be a hardship for IPFIX. > >> >> > > >> >> > But, I believe that if you are going to make a design > >> >> choice it should > >> >> > be motivated by design goals. What benefit do you gain > >> by limiting > >> >> > IPFIX to a simple uni-directional > >> >> > flow model? Is a uni-directional flow data model the best > >> >> > data model for reporting all the metrics and network > >> behaviors that > >> >> > relate to network flows? Is it the best data model for > >> >> supporting all > >> >> > the existing commercial network flow data? > >> >> > > >> >> > But what about even the simplest of applications of flow > >> data? Is a > >> >> > uni-directional flow data model the best solution for > >> reporting the > >> >> > RTT of a ping volley? The empirical bulk transfer > >> >> properties of a TCP > >> >> > connection? How about TCP packet retransmission? Existence > >> >> of routing > >> >> > failures in an OSPF routed network? Application > response time? > >> >> > Rerouting indications? Reachability fault detection? > >> >> Access policy > >> >> > validation? Connectivity status? > >> >> > > >> >> > Even FTP supports both ASCII and binary data transfer. > >> >> Handling both > >> >> > uni-directional data and bi-directional data in IPFIX > >> >> should be pretty > >> >> > easy stuff. > >> >> > > >> >> > > >> >> > Carter > >> >> > > >> >> > Carter Bullard > >> >> > QoSient, LLC > >> >> > 300 E. 56th Street, Suite 18K > >> >> > New York, New York 10022 > >> >> > > >> >> > carter@qosient.com > >> >> > Phone +1 212 588-9133 > >> >> > Fax +1 212 588-9134 > >> >> > http://qosient.com > >> >> > > >> >> > > >> >> > > >> >> > > >> >> > > -----Original Message----- > >> >> > > From: majordomo listserver > >> >> [mailto:majordomo@mil.doit.wisc.edu] On > >> >> > > Behalf Of Dave Plonka > >> >> > > Sent: Monday, January 07, 2002 4:52 PM > >> >> > > To: ipfix-req@net.doit.wisc.edu > >> >> > > Subject: [ipfix-req] unidirection vs. bidirectional flows > >> >> > > > >> >> > > > >> >> > > > >> >> > > IPFIX folks, > >> >> > > > >> >> > > As many of you recall, whether or not IPFIX should > specify a > >> >> > > unidirectional or bidirectional flow definition was a hot > >> >> topic at > >> >> > > the WG meeting. > >> >> > > > >> >> > > I think that specifying *something* gets us far along in > >> >> > > communicating what a "flow" is. Personally, I > would like us > >> >> > > to specify one or the other and I am in favor a > unidirection > >> >> > > flow definition. > >> >> > > > >> >> > > Here are some thoughts about arguments made for > >> >> bidirectional flows: > >> >> > > > >> >> > > A unidirectional flow definition means that the atomic > >> >> flow report > >> >> > > record is unidirection during the reporting/export > phase. For > >> >> > > instance, a single packet counter for a given > >> >> > > protocol/srcaddr/srcport/dstaddr/dstport n-tuple. > >> However, since > >> >> > > the exporter is somewhat of a "black box", it need not be > >> >> > > constrained to a unidirectional model internally. > >> >> > > > >> >> > > A unidirectional flow definition does not prevent a > collector > >> >> > > implementor from using a bi-directional notion of flows > >> >> internally > >> >> > > nor does it prevent it from being stored in a bidirectional > >> >> > > way prior to export. Conversely if we specify a > bidirectional > >> >> > > definition, it does disqualify unidirectional > implementations. > >> >> > > > >> >> > > For those that wish to perform measurements which > may require > >> >> > > bidirectional flow information (perhaps because start/end > >> >> timestamp > >> >> > > synchronization is required for flow pairs), the IPFIX > >> data model > >> >> > > and transport could define a mechanism by which the > >> >> exporter could > >> >> > > inform the collector about which unidirectional flow > >> records are > >> >> > > matched pairs. That is, the exporter could help the > >> >> collector find > >> >> > > the matched pairs of flow information, perhaps by > >> >> exporting the flow > >> >> > > records in pairs or by generating a numeric identifier > >> that is an > >> >> > > attribute of both flow records. Of course only > bidirectional > >> >> > > collectors would understand this convention, but a > >> >> > > unidirection collector could ignore it and still > make use of > >> >> > > the exported flow information from an exporter that > >> >> > > implemented this "bidirectional flow pairing" feature. > >> >> > > > >> >> > > Dave > >> >> > > > >> >> > > -- > >> >> > > ":) You will make many changes before settling > >> satisfactorily. :)" > >> >> > > - the actual message (complete with smiley faces) that I > >> >> received > >> >> > > in a > >> >> > > fortune cookie at dinner after the IPFIX WG meeting > >> >> during IETF 52 > >> >> > > in Salt Lake City > >> >> > > > >> >> > > -- > >> >> > > Help mailto:majordomo@net.doit.wisc.edu and > say "help" > >> >> > > in message body > >> >> > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > >> >> "unsubscribe > >> >> > > ipfix" in message body > >> >> > > Archive http://ipfix.doit.wisc.edu/archive/ > >> >> > > > >> >> > > > >> >> > > >> >> > > >> >> > > >> >> > -- > >> >> > Help mailto:majordomo@net.doit.wisc.edu and say > >> >> "help" in message body > >> >> > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > >> "unsubscribe > >> >> > ipfix" in message body > >> >> > Archive http://ipfix.doit.wisc.edu/archive/ > >> >> > > >> >> > >> >> > >> >> > >> > > >> > > >> > > >> > -- > >> > Help mailto:majordomo@net.doit.wisc.edu and say > >> "help" in message body > >> > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe > >> > ipfix" in message body > >> > Archive http://ipfix.doit.wisc.edu/archive/ > >> > > >> > >> > >> > >> -- > >> Help mailto:majordomo@net.doit.wisc.edu and say "help" > >> in message body > >> Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe > >> ipfix" in message body > >> Archive http://ipfix.doit.wisc.edu/archive/ > >> > >> > > > > > > > > -- > > Help mailto:majordomo@net.doit.wisc.edu and say > "help" in message body > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe > > ipfix" in message body > > Archive http://ipfix.doit.wisc.edu/archive/ > > > > > > -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Tue Jan 8 11:16:45 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA27756 for ; Tue, 8 Jan 2002 11:16:44 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16NycH-0000qG-00 for ipfix-list@mil.doit.wisc.edu; Tue, 08 Jan 2002 09:56:45 -0600 Received: from sj-msg-core-1.cisco.com ([171.71.163.11]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 16NycF-0000po-00 for ipfix-req@net.doit.wisc.edu; Tue, 08 Jan 2002 09:56:43 -0600 Received: from vvalluri-u10.cisco.com (vvalluri-u10.cisco.com [128.107.162.119]) by sj-msg-core-1.cisco.com (8.11.3/8.9.1) with ESMTP id g08FtMF28429; Tue, 8 Jan 2002 07:55:22 -0800 (PST) Date: Tue, 8 Jan 2002 07:55:22 -0800 (PST) From: Vamsidhar Valluri To: Carter Bullard cc: "'Juergen Quittek'" , , Subject: RE: [ipfix-req] unidirection vs. bidirectional flows In-Reply-To: <5C8959A16A71B449AE793CF52FBBED6603BE17@ptah.newyork.qosient.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Precedence: bulk Sender: majordomo listserver Carter, Catalyst platform may need cards but c4500, c7200, RSP, GSR routers do not need special cards. -vamsi On Tue, 8 Jan 2002, Carter Bullard wrote: > Hey Juergen, > You know, you have to buy special cards to get netflow > support in a Cisco router. I'm interested in how your > going to make uni-directional flow record reporting cheap. > > Carter > > Carter Bullard > QoSient, LLC > 300 E. 56th Street, Suite 18K > New York, New York 10022 > > carter@qosient.com > Phone +1 212 588-9133 > Fax +1 212 588-9134 > http://qosient.com > > > -----Original Message----- > > From: Juergen Quittek [mailto:quittek@ccrle.nec.de] > > Sent: Tuesday, January 08, 2002 8:23 AM > > To: carter@qosient.com > > Cc: plonka@doit.wisc.edu; ipfix-req@net.doit.wisc.edu > > Subject: RE: [ipfix-req] unidirection vs. bidirectional flows > > > > > > --On 08 January 2002 07:34 -0500 Carter Bullard wrote: > > > > > Hey Juergen, > > > Just a note, the majority of switches and routers > > > today (> 90%) cannot support uni-directional flow record > > > > What routers are you talking about? Isn't NetFlow available > > for most CisCo and Juniper boxes? > > > > > reporting. However, virtually all switches and routers > > > (> 99%) support RMON-style interface statistics, which > > > are L1/L2 bi-directional flow metrics. > > > > > > I believe that adoption of a uni-directional flow model excludes > > > basically the entire population of legacy networking devices that > > > could easily report their existing bi-directional flow > > metrics using > > > IPFIX. > > > > > > Wouldn't this violate your fundamental constraint? > > > > Not at all. No one is trying to forbid existing technologies. > > But this is about IPFIX requirements. I said I am not sure > > whether we should specify uni- or bi-directional in the > > requirements at all, but I am sure I do not want to have > > "expensive" requirements. If you explain to me how to make > > bi-directional flows cheap, I will find it easier considering > > them as a potential requirement. > > > > Juergen > > -- > > Juergen Quittek quittek@ccrle.nec.de Tel: +49 6221 90511-15 > > NEC Europe Ltd., Network Laboratories Fax: +49 6221 90511-55 > > Adenauerplatz 6, 69115 Heidelberg, Germany http://www.ccrle.nec.de > > > > > > > > Carter > > > > > > Carter Bullard > > > QoSient, LLC > > > 300 E. 56th Street, Suite 18K > > > New York, New York 10022 > > > > > > carter@qosient.com > > > Phone +1 212 588-9133 > > > Fax +1 212 588-9134 > > > http://qosient.com > > > > > > > > >> -----Original Message----- > > >> From: majordomo listserver [mailto:majordomo@mil.doit.wisc.edu] On > > >> Behalf Of Juergen Quittek > > >> Sent: Tuesday, January 08, 2002 6:24 AM > > >> To: carter@qosient.com; 'Vamsidhar Valluri' > > >> Cc: plonka@doit.wisc.edu; ipfix-req@net.doit.wisc.edu > > >> Subject: RE: [ipfix-req] unidirection vs. bidirectional flows > > >> > > >> > > >> Hi Carter, > > >> > > >> One of the very important constraints we applied to each > > requirements > > >> so far, is that it must be possible to for routers as well as for > > >> probes to implement the respective requirement with reasonable > > >> effort. > > >> > > >> (If anyone finds out that one or more of the requirements > > appears to > > >> be expensive, then please speak up!) > > >> > > >> Facing the hardware acrchitecture of a modern router, > > >> I consider a bi-directional flow model to be too expensive > > to be part > > >> of the requirements (and consequently also too expensive > > to be part > > >> of the general architecture on the exporter side). > > >> > > >> The reason is that at line interface cards you typically > > spent very > > >> different effort on treatment on incoming and outgoing > > packets. For > > >> incoming packets you do a lot of > > >> processing: checksums, TTL processing, routing based on cached > > >> routing tables, ... Here you touch header fields anyway and it is > > >> relatively easy and relatively cheap to add metering > > functions. For > > >> outgoing packets the treatment is different. You just try > > to get rid > > >> of them as fast as possible (except from scheduling thme > > first). Here > > >> it would be rather expensive to investigate header fields again and > > >> it might also reduce your maximum thoughput. Therefore a > > >> device just metering incoming packets should be able to > > >> match the requirements. > > >> > > >> Now, if we have a bi-directional flow model, you require such a > > >> device to merge all its input measurements performed at different > > >> line cards by looking for measured uni-directional flows > > belongig to > > >> the same bi-directinal one. This task might be done at a control > > >> processor, but it creates additional cost and it might > > slow down the > > >> system. I believe that merging of these measurements is a > > perfect job > > >> for an analysis application and not for a meter. > > >> > > >> Definitely it is nothing we should ask every meter to perform by > > >> making it a MUST requirement! > > >> > > >> Consequently, my position is that IF we add a flow model to the > > >> requirements, then it should be the uni-diractional one. > > However, I > > >> am not yet convinced, that we should specify the model at > > all in the > > >> requirements. > > >> > > >> Best wishes, > > >> > > >> Juergen > > >> -- > > >> Juergen Quittek quittek@ccrle.nec.de Tel: +49 6221 90511-15 > > >> NEC Europe Ltd., Network Laboratories Fax: +49 6221 90511-55 > > >> Adenauerplatz 6, 69115 Heidelberg, Germany > > http://www.ccrle.nec.de > > >> > > >> > > >> --On 07 January 2002 19:42 -0500 Carter Bullard > > > > >> wrote: > > >> > > >> > Hey Vamsi, > > >> > Hmmm, is the interface a part of the flow defintion? Not a > > >> > problem. Lets look at how this may work. Suppose there > > is a SRC > > >> > interface and a DST interface, so that a flow is simply all the > > >> > packets moving from SRC -> DST. The return flow would > > be all the > > >> > packets moving from DST -> SRC. > > >> > > > >> > So the flow is reported as SRC <-> DST, with the > > simple addition > > >> > of the DST -> SRC metrics. Makes more sense than independantly > > >> > reporting two half-duplex flow records, which could be skewed in > > >> > time so badly, that the two records cannot be correlated at all. > > >> > > > >> > > > >> > Carter > > >> > > > >> > Carter Bullard > > >> > QoSient, LLC > > >> > 300 E. 56th Street, Suite 18K > > >> > New York, New York 10022 > > >> > > > >> > carter@qosient.com > > >> > Phone +1 212 588-9133 > > >> > Fax +1 212 588-9134 > > >> > http://qosient.com > > >> > > > >> > > > >> >> -----Original Message----- > > >> >> From: Vamsidhar Valluri [mailto:vvalluri@cisco.com] > > >> >> Sent: Monday, January 07, 2002 6:58 PM > > >> >> To: Carter Bullard > > >> >> Cc: plonka@doit.wisc.edu; ipfix-req@net.doit.wisc.edu > > >> >> Subject: RE: [ipfix-req] unidirection vs. bidirectional flows > > >> >> > > >> >> > > >> >> > > >> >> > > >> >> On Mon, 7 Jan 2002, Carter Bullard wrote: > > >> >> > > >> >> > Gentle people, > > >> >> > There is absolutely no reason for IPFIX to limit the type of > > >> >> > flow data that it supports. The difference between > > >> >> uni-directional > > >> >> > and bi-directional flow records is simply the type of > > >> >> metrics that are > > >> >> > reported. The flow descriptors and keys are all the > > >> same. Because > > >> >> > IPFIX must support > > >> >> ^^^^^ > > >> >> > > >> >> If "keys" you are refering to are the fields used to identify a > > >> >> flow then "SRC interface" do not fit into the statement > > "The flow > > >> >> descriptors and keys are all same for both unidirectional and > > >> >> bi-directional flow records" mentioned above because > > SRC interface > > >> >> will make it unidirectional. > > >> >> > > >> >> -vamsi > > >> >> > > >> >> > > >> >> > reporting metrics that each vendors will not implement, such > > >> >> as jitter > > >> >> > and one-way delay, which are the few metrics that are > > >> meaningful in > > >> >> > uni-directional flow data models, supporting metrics > > >> that report on > > >> >> > return traffic should not be a hardship for IPFIX. > > >> >> > > > >> >> > But, I believe that if you are going to make a design > > >> >> choice it should > > >> >> > be motivated by design goals. What benefit do you gain > > >> by limiting > > >> >> > IPFIX to a simple uni-directional > > >> >> > flow model? Is a uni-directional flow data model the best > > >> >> > data model for reporting all the metrics and network > > >> behaviors that > > >> >> > relate to network flows? Is it the best data model for > > >> >> supporting all > > >> >> > the existing commercial network flow data? > > >> >> > > > >> >> > But what about even the simplest of applications of flow > > >> data? Is a > > >> >> > uni-directional flow data model the best solution for > > >> reporting the > > >> >> > RTT of a ping volley? The empirical bulk transfer > > >> >> properties of a TCP > > >> >> > connection? How about TCP packet retransmission? Existence > > >> >> of routing > > >> >> > failures in an OSPF routed network? Application > > response time? > > >> >> > Rerouting indications? Reachability fault detection? > > >> >> Access policy > > >> >> > validation? Connectivity status? > > >> >> > > > >> >> > Even FTP supports both ASCII and binary data transfer. > > >> >> Handling both > > >> >> > uni-directional data and bi-directional data in IPFIX > > >> >> should be pretty > > >> >> > easy stuff. > > >> >> > > > >> >> > > > >> >> > Carter > > >> >> > > > >> >> > Carter Bullard > > >> >> > QoSient, LLC > > >> >> > 300 E. 56th Street, Suite 18K > > >> >> > New York, New York 10022 > > >> >> > > > >> >> > carter@qosient.com > > >> >> > Phone +1 212 588-9133 > > >> >> > Fax +1 212 588-9134 > > >> >> > http://qosient.com > > >> >> > > > >> >> > > > >> >> > > > >> >> > > > >> >> > > -----Original Message----- > > >> >> > > From: majordomo listserver > > >> >> [mailto:majordomo@mil.doit.wisc.edu] On > > >> >> > > Behalf Of Dave Plonka > > >> >> > > Sent: Monday, January 07, 2002 4:52 PM > > >> >> > > To: ipfix-req@net.doit.wisc.edu > > >> >> > > Subject: [ipfix-req] unidirection vs. bidirectional flows > > >> >> > > > > >> >> > > > > >> >> > > > > >> >> > > IPFIX folks, > > >> >> > > > > >> >> > > As many of you recall, whether or not IPFIX should > > specify a > > >> >> > > unidirectional or bidirectional flow definition was a hot > > >> >> topic at > > >> >> > > the WG meeting. > > >> >> > > > > >> >> > > I think that specifying *something* gets us far along in > > >> >> > > communicating what a "flow" is. Personally, I > > would like us > > >> >> > > to specify one or the other and I am in favor a > > unidirection > > >> >> > > flow definition. > > >> >> > > > > >> >> > > Here are some thoughts about arguments made for > > >> >> bidirectional flows: > > >> >> > > > > >> >> > > A unidirectional flow definition means that the atomic > > >> >> flow report > > >> >> > > record is unidirection during the reporting/export > > phase. For > > >> >> > > instance, a single packet counter for a given > > >> >> > > protocol/srcaddr/srcport/dstaddr/dstport n-tuple. > > >> However, since > > >> >> > > the exporter is somewhat of a "black box", it need not be > > >> >> > > constrained to a unidirectional model internally. > > >> >> > > > > >> >> > > A unidirectional flow definition does not prevent a > > collector > > >> >> > > implementor from using a bi-directional notion of flows > > >> >> internally > > >> >> > > nor does it prevent it from being stored in a bidirectional > > >> >> > > way prior to export. Conversely if we specify a > > bidirectional > > >> >> > > definition, it does disqualify unidirectional > > implementations. > > >> >> > > > > >> >> > > For those that wish to perform measurements which > > may require > > >> >> > > bidirectional flow information (perhaps because start/end > > >> >> timestamp > > >> >> > > synchronization is required for flow pairs), the IPFIX > > >> data model > > >> >> > > and transport could define a mechanism by which the > > >> >> exporter could > > >> >> > > inform the collector about which unidirectional flow > > >> records are > > >> >> > > matched pairs. That is, the exporter could help the > > >> >> collector find > > >> >> > > the matched pairs of flow information, perhaps by > > >> >> exporting the flow > > >> >> > > records in pairs or by generating a numeric identifier > > >> that is an > > >> >> > > attribute of both flow records. Of course only > > bidirectional > > >> >> > > collectors would understand this convention, but a > > >> >> > > unidirection collector could ignore it and still > > make use of > > >> >> > > the exported flow information from an exporter that > > >> >> > > implemented this "bidirectional flow pairing" feature. > > >> >> > > > > >> >> > > Dave > > >> >> > > > > >> >> > > -- > > >> >> > > ":) You will make many changes before settling > > >> satisfactorily. :)" > > >> >> > > - the actual message (complete with smiley faces) that I > > >> >> received > > >> >> > > in a > > >> >> > > fortune cookie at dinner after the IPFIX WG meeting > > >> >> during IETF 52 > > >> >> > > in Salt Lake City > > >> >> > > > > >> >> > > -- > > >> >> > > Help mailto:majordomo@net.doit.wisc.edu and > > say "help" > > >> >> > > in message body > > >> >> > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > >> >> "unsubscribe > > >> >> > > ipfix" in message body > > >> >> > > Archive http://ipfix.doit.wisc.edu/archive/ > > >> >> > > > > >> >> > > > > >> >> > > > >> >> > > > >> >> > > > >> >> > -- > > >> >> > Help mailto:majordomo@net.doit.wisc.edu and say > > >> >> "help" in message body > > >> >> > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > >> "unsubscribe > > >> >> > ipfix" in message body > > >> >> > Archive http://ipfix.doit.wisc.edu/archive/ > > >> >> > > > >> >> > > >> >> > > >> >> > > >> > > > >> > > > >> > > > >> > -- > > >> > Help mailto:majordomo@net.doit.wisc.edu and say > > >> "help" in message body > > >> > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > "unsubscribe > > >> > ipfix" in message body > > >> > Archive http://ipfix.doit.wisc.edu/archive/ > > >> > > > >> > > >> > > >> > > >> -- > > >> Help mailto:majordomo@net.doit.wisc.edu and say "help" > > >> in message body > > >> Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > "unsubscribe > > >> ipfix" in message body > > >> Archive http://ipfix.doit.wisc.edu/archive/ > > >> > > >> > > > > > > > > > > > > -- > > > Help mailto:majordomo@net.doit.wisc.edu and say > > "help" in message body > > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe > > > ipfix" in message body > > > Archive http://ipfix.doit.wisc.edu/archive/ > > > > > > > > > > > > > > > -- > Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ > -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Tue Jan 8 11:16:51 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA27766 for ; Tue, 8 Jan 2002 11:16:45 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16NydT-0000sD-00 for ipfix-list@mil.doit.wisc.edu; Tue, 08 Jan 2002 09:57:59 -0600 Received: from bru-cse-222.cisco.com ([144.254.8.48]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 16NydS-0000rF-00 for ipfix-req@net.doit.wisc.edu; Tue, 08 Jan 2002 09:57:58 -0600 Received: (from bclaise@localhost) by bru-cse-222.cisco.com (8.10.2+Sun/CA/950118) id g08FuK613751; Tue, 8 Jan 2002 16:56:20 +0100 (CET) Date: Tue, 8 Jan 2002 16:56:20 +0100 (CET) From: Benoit Claise Message-Id: <200201081556.g08FuK613751@bru-cse-222.cisco.com> To: quittek@ccrle.nec.de, carter@qosient.com Subject: RE: [ipfix-req] unidirection vs. bidirectional flows Cc: plonka@doit.wisc.edu, ipfix-req@net.doit.wisc.edu X-Sun-Charset: US-ASCII Precedence: bulk Sender: majordomo listserver Hi Carter, > > Hey Juergen, > You know, you have to buy special cards to get netflow > support in a Cisco router. I'm interested in how your > going to make uni-directional flow record reporting cheap. This is not correct! This function is part of the software. I know of only device which needs a special daughter card, and I don't think this still applies. Regards, Benoit. > > Carter > > Carter Bullard > QoSient, LLC > 300 E. 56th Street, Suite 18K > New York, New York 10022 > > carter@qosient.com > Phone +1 212 588-9133 > Fax +1 212 588-9134 > http://qosient.com > > > -----Original Message----- > > From: Juergen Quittek [mailto:quittek@ccrle.nec.de] > > Sent: Tuesday, January 08, 2002 8:23 AM > > To: carter@qosient.com > > Cc: plonka@doit.wisc.edu; ipfix-req@net.doit.wisc.edu > > Subject: RE: [ipfix-req] unidirection vs. bidirectional flows > > > > > > --On 08 January 2002 07:34 -0500 Carter Bullard wrote: > > > > > Hey Juergen, > > > Just a note, the majority of switches and routers > > > today (> 90%) cannot support uni-directional flow record > > > > What routers are you talking about? Isn't NetFlow available > > for most CisCo and Juniper boxes? > > > > > reporting. However, virtually all switches and routers > > > (> 99%) support RMON-style interface statistics, which > > > are L1/L2 bi-directional flow metrics. > > > > > > I believe that adoption of a uni-directional flow model excludes > > > basically the entire population of legacy networking devices that > > > could easily report their existing bi-directional flow > > metrics using > > > IPFIX. > > > > > > Wouldn't this violate your fundamental constraint? > > > > Not at all. No one is trying to forbid existing technologies. > > But this is about IPFIX requirements. I said I am not sure > > whether we should specify uni- or bi-directional in the > > requirements at all, but I am sure I do not want to have > > "expensive" requirements. If you explain to me how to make > > bi-directional flows cheap, I will find it easier considering > > them as a potential requirement. > > > > Juergen > > -- > > Juergen Quittek quittek@ccrle.nec.de Tel: +49 6221 90511-15 > > NEC Europe Ltd., Network Laboratories Fax: +49 6221 90511-55 > > Adenauerplatz 6, 69115 Heidelberg, Germany http://www.ccrle.nec.de > > > > > > > > Carter > > > > > > Carter Bullard > > > QoSient, LLC > > > 300 E. 56th Street, Suite 18K > > > New York, New York 10022 > > > > > > carter@qosient.com > > > Phone +1 212 588-9133 > > > Fax +1 212 588-9134 > > > http://qosient.com > > > > > > > > >> -----Original Message----- > > >> From: majordomo listserver [mailto:majordomo@mil.doit.wisc.edu] On > > >> Behalf Of Juergen Quittek > > >> Sent: Tuesday, January 08, 2002 6:24 AM > > >> To: carter@qosient.com; 'Vamsidhar Valluri' > > >> Cc: plonka@doit.wisc.edu; ipfix-req@net.doit.wisc.edu > > >> Subject: RE: [ipfix-req] unidirection vs. bidirectional flows > > >> > > >> > > >> Hi Carter, > > >> > > >> One of the very important constraints we applied to each > > requirements > > >> so far, is that it must be possible to for routers as well as for > > >> probes to implement the respective requirement with reasonable > > >> effort. > > >> > > >> (If anyone finds out that one or more of the requirements > > appears to > > >> be expensive, then please speak up!) > > >> > > >> Facing the hardware acrchitecture of a modern router, > > >> I consider a bi-directional flow model to be too expensive > > to be part > > >> of the requirements (and consequently also too expensive > > to be part > > >> of the general architecture on the exporter side). > > >> > > >> The reason is that at line interface cards you typically > > spent very > > >> different effort on treatment on incoming and outgoing > > packets. For > > >> incoming packets you do a lot of > > >> processing: checksums, TTL processing, routing based on cached > > >> routing tables, ... Here you touch header fields anyway and it is > > >> relatively easy and relatively cheap to add metering > > functions. For > > >> outgoing packets the treatment is different. You just try > > to get rid > > >> of them as fast as possible (except from scheduling thme > > first). Here > > >> it would be rather expensive to investigate header fields again and > > >> it might also reduce your maximum thoughput. Therefore a > > >> device just metering incoming packets should be able to > > >> match the requirements. > > >> > > >> Now, if we have a bi-directional flow model, you require such a > > >> device to merge all its input measurements performed at different > > >> line cards by looking for measured uni-directional flows > > belongig to > > >> the same bi-directinal one. This task might be done at a control > > >> processor, but it creates additional cost and it might > > slow down the > > >> system. I believe that merging of these measurements is a > > perfect job > > >> for an analysis application and not for a meter. > > >> > > >> Definitely it is nothing we should ask every meter to perform by > > >> making it a MUST requirement! > > >> > > >> Consequently, my position is that IF we add a flow model to the > > >> requirements, then it should be the uni-diractional one. > > However, I > > >> am not yet convinced, that we should specify the model at > > all in the > > >> requirements. > > >> > > >> Best wishes, > > >> > > >> Juergen > > >> -- > > >> Juergen Quittek quittek@ccrle.nec.de Tel: +49 6221 90511-15 > > >> NEC Europe Ltd., Network Laboratories Fax: +49 6221 90511-55 > > >> Adenauerplatz 6, 69115 Heidelberg, Germany > > http://www.ccrle.nec.de > > >> > > >> > > >> --On 07 January 2002 19:42 -0500 Carter Bullard > > > > >> wrote: > > >> > > >> > Hey Vamsi, > > >> > Hmmm, is the interface a part of the flow defintion? Not a > > >> > problem. Lets look at how this may work. Suppose there > > is a SRC > > >> > interface and a DST interface, so that a flow is simply all the > > >> > packets moving from SRC -> DST. The return flow would > > be all the > > >> > packets moving from DST -> SRC. > > >> > > > >> > So the flow is reported as SRC <-> DST, with the > > simple addition > > >> > of the DST -> SRC metrics. Makes more sense than independantly > > >> > reporting two half-duplex flow records, which could be skewed in > > >> > time so badly, that the two records cannot be correlated at all. > > >> > > > >> > > > >> > Carter > > >> > > > >> > Carter Bullard > > >> > QoSient, LLC > > >> > 300 E. 56th Street, Suite 18K > > >> > New York, New York 10022 > > >> > > > >> > carter@qosient.com > > >> > Phone +1 212 588-9133 > > >> > Fax +1 212 588-9134 > > >> > http://qosient.com > > >> > > > >> > > > >> >> -----Original Message----- > > >> >> From: Vamsidhar Valluri [mailto:vvalluri@cisco.com] > > >> >> Sent: Monday, January 07, 2002 6:58 PM > > >> >> To: Carter Bullard > > >> >> Cc: plonka@doit.wisc.edu; ipfix-req@net.doit.wisc.edu > > >> >> Subject: RE: [ipfix-req] unidirection vs. bidirectional flows > > >> >> > > >> >> > > >> >> > > >> >> > > >> >> On Mon, 7 Jan 2002, Carter Bullard wrote: > > >> >> > > >> >> > Gentle people, > > >> >> > There is absolutely no reason for IPFIX to limit the type of > > >> >> > flow data that it supports. The difference between > > >> >> uni-directional > > >> >> > and bi-directional flow records is simply the type of > > >> >> metrics that are > > >> >> > reported. The flow descriptors and keys are all the > > >> same. Because > > >> >> > IPFIX must support > > >> >> ^^^^^ > > >> >> > > >> >> If "keys" you are refering to are the fields used to identify a > > >> >> flow then "SRC interface" do not fit into the statement > > "The flow > > >> >> descriptors and keys are all same for both unidirectional and > > >> >> bi-directional flow records" mentioned above because > > SRC interface > > >> >> will make it unidirectional. > > >> >> > > >> >> -vamsi > > >> >> > > >> >> > > >> >> > reporting metrics that each vendors will not implement, such > > >> >> as jitter > > >> >> > and one-way delay, which are the few metrics that are > > >> meaningful in > > >> >> > uni-directional flow data models, supporting metrics > > >> that report on > > >> >> > return traffic should not be a hardship for IPFIX. > > >> >> > > > >> >> > But, I believe that if you are going to make a design > > >> >> choice it should > > >> >> > be motivated by design goals. What benefit do you gain > > >> by limiting > > >> >> > IPFIX to a simple uni-directional > > >> >> > flow model? Is a uni-directional flow data model the best > > >> >> > data model for reporting all the metrics and network > > >> behaviors that > > >> >> > relate to network flows? Is it the best data model for > > >> >> supporting all > > >> >> > the existing commercial network flow data? > > >> >> > > > >> >> > But what about even the simplest of applications of flow > > >> data? Is a > > >> >> > uni-directional flow data model the best solution for > > >> reporting the > > >> >> > RTT of a ping volley? The empirical bulk transfer > > >> >> properties of a TCP > > >> >> > connection? How about TCP packet retransmission? Existence > > >> >> of routing > > >> >> > failures in an OSPF routed network? Application > > response time? > > >> >> > Rerouting indications? Reachability fault detection? > > >> >> Access policy > > >> >> > validation? Connectivity status? > > >> >> > > > >> >> > Even FTP supports both ASCII and binary data transfer. > > >> >> Handling both > > >> >> > uni-directional data and bi-directional data in IPFIX > > >> >> should be pretty > > >> >> > easy stuff. > > >> >> > > > >> >> > > > >> >> > Carter > > >> >> > > > >> >> > Carter Bullard > > >> >> > QoSient, LLC > > >> >> > 300 E. 56th Street, Suite 18K > > >> >> > New York, New York 10022 > > >> >> > > > >> >> > carter@qosient.com > > >> >> > Phone +1 212 588-9133 > > >> >> > Fax +1 212 588-9134 > > >> >> > http://qosient.com > > >> >> > > > >> >> > > > >> >> > > > >> >> > > > >> >> > > -----Original Message----- > > >> >> > > From: majordomo listserver > > >> >> [mailto:majordomo@mil.doit.wisc.edu] On > > >> >> > > Behalf Of Dave Plonka > > >> >> > > Sent: Monday, January 07, 2002 4:52 PM > > >> >> > > To: ipfix-req@net.doit.wisc.edu > > >> >> > > Subject: [ipfix-req] unidirection vs. bidirectional flows > > >> >> > > > > >> >> > > > > >> >> > > > > >> >> > > IPFIX folks, > > >> >> > > > > >> >> > > As many of you recall, whether or not IPFIX should > > specify a > > >> >> > > unidirectional or bidirectional flow definition was a hot > > >> >> topic at > > >> >> > > the WG meeting. > > >> >> > > > > >> >> > > I think that specifying *something* gets us far along in > > >> >> > > communicating what a "flow" is. Personally, I > > would like us > > >> >> > > to specify one or the other and I am in favor a > > unidirection > > >> >> > > flow definition. > > >> >> > > > > >> >> > > Here are some thoughts about arguments made for > > >> >> bidirectional flows: > > >> >> > > > > >> >> > > A unidirectional flow definition means that the atomic > > >> >> flow report > > >> >> > > record is unidirection during the reporting/export > > phase. For > > >> >> > > instance, a single packet counter for a given > > >> >> > > protocol/srcaddr/srcport/dstaddr/dstport n-tuple. > > >> However, since > > >> >> > > the exporter is somewhat of a "black box", it need not be > > >> >> > > constrained to a unidirectional model internally. > > >> >> > > > > >> >> > > A unidirectional flow definition does not prevent a > > collector > > >> >> > > implementor from using a bi-directional notion of flows > > >> >> internally > > >> >> > > nor does it prevent it from being stored in a bidirectional > > >> >> > > way prior to export. Conversely if we specify a > > bidirectional > > >> >> > > definition, it does disqualify unidirectional > > implementations. > > >> >> > > > > >> >> > > For those that wish to perform measurements which > > may require > > >> >> > > bidirectional flow information (perhaps because start/end > > >> >> timestamp > > >> >> > > synchronization is required for flow pairs), the IPFIX > > >> data model > > >> >> > > and transport could define a mechanism by which the > > >> >> exporter could > > >> >> > > inform the collector about which unidirectional flow > > >> records are > > >> >> > > matched pairs. That is, the exporter could help the > > >> >> collector find > > >> >> > > the matched pairs of flow information, perhaps by > > >> >> exporting the flow > > >> >> > > records in pairs or by generating a numeric identifier > > >> that is an > > >> >> > > attribute of both flow records. Of course only > > bidirectional > > >> >> > > collectors would understand this convention, but a > > >> >> > > unidirection collector could ignore it and still > > make use of > > >> >> > > the exported flow information from an exporter that > > >> >> > > implemented this "bidirectional flow pairing" feature. > > >> >> > > > > >> >> > > Dave > > >> >> > > > > >> >> > > -- > > >> >> > > ":) You will make many changes before settling > > >> satisfactorily. :)" > > >> >> > > - the actual message (complete with smiley faces) that I > > >> >> received > > >> >> > > in a > > >> >> > > fortune cookie at dinner after the IPFIX WG meeting > > >> >> during IETF 52 > > >> >> > > in Salt Lake City > > >> >> > > > > >> >> > > -- > > >> >> > > Help mailto:majordomo@net.doit.wisc.edu and > > say "help" > > >> >> > > in message body > > >> >> > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > >> >> "unsubscribe > > >> >> > > ipfix" in message body > > >> >> > > Archive http://ipfix.doit.wisc.edu/archive/ > > >> >> > > > > >> >> > > > > >> >> > > > >> >> > > > >> >> > > > >> >> > -- > > >> >> > Help mailto:majordomo@net.doit.wisc.edu and say > > >> >> "help" in message body > > >> >> > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > >> "unsubscribe > > >> >> > ipfix" in message body > > >> >> > Archive http://ipfix.doit.wisc.edu/archive/ > > >> >> > > > >> >> > > >> >> > > >> >> > > >> > > > >> > > > >> > > > >> > -- > > >> > Help mailto:majordomo@net.doit.wisc.edu and say > > >> "help" in message body > > >> > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > "unsubscribe > > >> > ipfix" in message body > > >> > Archive http://ipfix.doit.wisc.edu/archive/ > > >> > > > >> > > >> > > >> > > >> -- > > >> Help mailto:majordomo@net.doit.wisc.edu and say "help" > > >> in message body > > >> Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > "unsubscribe > > >> ipfix" in message body > > >> Archive http://ipfix.doit.wisc.edu/archive/ > > >> > > >> > > > > > > > > > > > > -- > > > Help mailto:majordomo@net.doit.wisc.edu and say > > "help" in message body > > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe > > > ipfix" in message body > > > Archive http://ipfix.doit.wisc.edu/archive/ > > > > > > > > > > > > > > > -- > Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ > -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Tue Jan 8 11:19:10 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA27900 for ; Tue, 8 Jan 2002 11:19:09 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16NyfM-0000v3-00 for ipfix-list@mil.doit.wisc.edu; Tue, 08 Jan 2002 09:59:56 -0600 Received: from yamato.ccrle.nec.de ([195.37.70.1]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 16NyfJ-0000uF-00 for ipfix-req@net.doit.wisc.edu; Tue, 08 Jan 2002 09:59:53 -0600 Received: from wallace.heidelberg.ccrle.nec.de (root@wallace [192.168.102.1]) by yamato.ccrle.nec.de (8.11.6/8.10.1) with ESMTP id g08FxjR09414; Tue, 8 Jan 2002 16:59:45 +0100 (CET) Received: from [192.168.102.164] (beta.heidelberg.ccrle.nec.de [192.168.102.164]) by wallace.heidelberg.ccrle.nec.de (8.9.3/8.9.3/SuSE Linux 8.9.3-0.1) with ESMTP id QAA28715; Tue, 8 Jan 2002 16:59:22 +0100 Date: Tue, 08 Jan 2002 17:00:26 +0100 From: Juergen Quittek To: carter@qosient.com cc: plonka@doit.wisc.edu, ipfix-req@net.doit.wisc.edu Subject: RE: [ipfix-req] unidirection vs. bidirectional flows Message-ID: <1191912232.1010509226@[192.168.102.164]> In-Reply-To: <5C8959A16A71B449AE793CF52FBBED6603BE17@ptah.newyork.qosient.com> X-Mailer: Mulberry/2.0.8 (Win32) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Hi Carter, I'm not making anything cheaper as it is, I'm just claiming that bi-directional measurements in big routers are expensive, IMO too expensive. Juergen -- Juergen Quittek quittek@ccrle.nec.de Tel: +49 6221 90511-15 NEC Europe Ltd., Network Laboratories Fax: +49 6221 90511-55 Adenauerplatz 6, 69115 Heidelberg, Germany http://www.ccrle.nec.de --On 08 January 2002 10:51 -0500 Carter Bullard wrote: > Hey Juergen, > You know, you have to buy special cards to get netflow > support in a Cisco router. I'm interested in how your > going to make uni-directional flow record reporting cheap. > > Carter > > Carter Bullard > QoSient, LLC > 300 E. 56th Street, Suite 18K > New York, New York 10022 > > carter@qosient.com > Phone +1 212 588-9133 > Fax +1 212 588-9134 > http://qosient.com > >> -----Original Message----- >> From: Juergen Quittek [mailto:quittek@ccrle.nec.de] >> Sent: Tuesday, January 08, 2002 8:23 AM >> To: carter@qosient.com >> Cc: plonka@doit.wisc.edu; ipfix-req@net.doit.wisc.edu >> Subject: RE: [ipfix-req] unidirection vs. bidirectional flows >> >> >> --On 08 January 2002 07:34 -0500 Carter Bullard wrote: >> >> > Hey Juergen, >> > Just a note, the majority of switches and routers >> > today (> 90%) cannot support uni-directional flow record >> >> What routers are you talking about? Isn't NetFlow available >> for most CisCo and Juniper boxes? >> >> > reporting. However, virtually all switches and routers >> > (> 99%) support RMON-style interface statistics, which >> > are L1/L2 bi-directional flow metrics. >> > >> > I believe that adoption of a uni-directional flow model excludes >> > basically the entire population of legacy networking devices that >> > could easily report their existing bi-directional flow >> metrics using >> > IPFIX. >> > >> > Wouldn't this violate your fundamental constraint? >> >> Not at all. No one is trying to forbid existing technologies. >> But this is about IPFIX requirements. I said I am not sure >> whether we should specify uni- or bi-directional in the >> requirements at all, but I am sure I do not want to have >> "expensive" requirements. If you explain to me how to make >> bi-directional flows cheap, I will find it easier considering >> them as a potential requirement. >> >> Juergen >> -- >> Juergen Quittek quittek@ccrle.nec.de Tel: +49 6221 90511-15 >> NEC Europe Ltd., Network Laboratories Fax: +49 6221 90511-55 >> Adenauerplatz 6, 69115 Heidelberg, Germany http://www.ccrle.nec.de >> >> > >> > Carter >> > >> > Carter Bullard >> > QoSient, LLC >> > 300 E. 56th Street, Suite 18K >> > New York, New York 10022 >> > >> > carter@qosient.com >> > Phone +1 212 588-9133 >> > Fax +1 212 588-9134 >> > http://qosient.com >> > >> > >> >> -----Original Message----- >> >> From: majordomo listserver [mailto:majordomo@mil.doit.wisc.edu] On >> >> Behalf Of Juergen Quittek >> >> Sent: Tuesday, January 08, 2002 6:24 AM >> >> To: carter@qosient.com; 'Vamsidhar Valluri' >> >> Cc: plonka@doit.wisc.edu; ipfix-req@net.doit.wisc.edu >> >> Subject: RE: [ipfix-req] unidirection vs. bidirectional flows >> >> >> >> >> >> Hi Carter, >> >> >> >> One of the very important constraints we applied to each >> requirements >> >> so far, is that it must be possible to for routers as well as for >> >> probes to implement the respective requirement with reasonable >> >> effort. >> >> >> >> (If anyone finds out that one or more of the requirements >> appears to >> >> be expensive, then please speak up!) >> >> >> >> Facing the hardware acrchitecture of a modern router, >> >> I consider a bi-directional flow model to be too expensive >> to be part >> >> of the requirements (and consequently also too expensive >> to be part >> >> of the general architecture on the exporter side). >> >> >> >> The reason is that at line interface cards you typically >> spent very >> >> different effort on treatment on incoming and outgoing >> packets. For >> >> incoming packets you do a lot of >> >> processing: checksums, TTL processing, routing based on cached >> >> routing tables, ... Here you touch header fields anyway and it is >> >> relatively easy and relatively cheap to add metering >> functions. For >> >> outgoing packets the treatment is different. You just try >> to get rid >> >> of them as fast as possible (except from scheduling thme >> first). Here >> >> it would be rather expensive to investigate header fields again and >> >> it might also reduce your maximum thoughput. Therefore a >> >> device just metering incoming packets should be able to >> >> match the requirements. >> >> >> >> Now, if we have a bi-directional flow model, you require such a >> >> device to merge all its input measurements performed at different >> >> line cards by looking for measured uni-directional flows >> belongig to >> >> the same bi-directinal one. This task might be done at a control >> >> processor, but it creates additional cost and it might >> slow down the >> >> system. I believe that merging of these measurements is a >> perfect job >> >> for an analysis application and not for a meter. >> >> >> >> Definitely it is nothing we should ask every meter to perform by >> >> making it a MUST requirement! >> >> >> >> Consequently, my position is that IF we add a flow model to the >> >> requirements, then it should be the uni-diractional one. >> However, I >> >> am not yet convinced, that we should specify the model at >> all in the >> >> requirements. >> >> >> >> Best wishes, >> >> >> >> Juergen >> >> -- >> >> Juergen Quittek quittek@ccrle.nec.de Tel: +49 6221 90511-15 >> >> NEC Europe Ltd., Network Laboratories Fax: +49 6221 90511-55 >> >> Adenauerplatz 6, 69115 Heidelberg, Germany >> http://www.ccrle.nec.de >> >> >> >> >> >> --On 07 January 2002 19:42 -0500 Carter Bullard >> >> >> wrote: >> >> >> >> > Hey Vamsi, >> >> > Hmmm, is the interface a part of the flow defintion? Not a >> >> > problem. Lets look at how this may work. Suppose there >> is a SRC >> >> > interface and a DST interface, so that a flow is simply all the >> >> > packets moving from SRC -> DST. The return flow would >> be all the >> >> > packets moving from DST -> SRC. >> >> > >> >> > So the flow is reported as SRC <-> DST, with the >> simple addition >> >> > of the DST -> SRC metrics. Makes more sense than independantly >> >> > reporting two half-duplex flow records, which could be skewed in >> >> > time so badly, that the two records cannot be correlated at all. >> >> > >> >> > >> >> > Carter >> >> > >> >> > Carter Bullard >> >> > QoSient, LLC >> >> > 300 E. 56th Street, Suite 18K >> >> > New York, New York 10022 >> >> > >> >> > carter@qosient.com >> >> > Phone +1 212 588-9133 >> >> > Fax +1 212 588-9134 >> >> > http://qosient.com >> >> > >> >> > >> >> >> -----Original Message----- >> >> >> From: Vamsidhar Valluri [mailto:vvalluri@cisco.com] >> >> >> Sent: Monday, January 07, 2002 6:58 PM >> >> >> To: Carter Bullard >> >> >> Cc: plonka@doit.wisc.edu; ipfix-req@net.doit.wisc.edu >> >> >> Subject: RE: [ipfix-req] unidirection vs. bidirectional flows >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> On Mon, 7 Jan 2002, Carter Bullard wrote: >> >> >> >> >> >> > Gentle people, >> >> >> > There is absolutely no reason for IPFIX to limit the type of >> >> >> > flow data that it supports. The difference between >> >> >> uni-directional >> >> >> > and bi-directional flow records is simply the type of >> >> >> metrics that are >> >> >> > reported. The flow descriptors and keys are all the >> >> same. Because >> >> >> > IPFIX must support >> >> >> ^^^^^ >> >> >> >> >> >> If "keys" you are refering to are the fields used to identify a >> >> >> flow then "SRC interface" do not fit into the statement >> "The flow >> >> >> descriptors and keys are all same for both unidirectional and >> >> >> bi-directional flow records" mentioned above because >> SRC interface >> >> >> will make it unidirectional. >> >> >> >> >> >> -vamsi >> >> >> >> >> >> >> >> >> > reporting metrics that each vendors will not implement, such >> >> >> as jitter >> >> >> > and one-way delay, which are the few metrics that are >> >> meaningful in >> >> >> > uni-directional flow data models, supporting metrics >> >> that report on >> >> >> > return traffic should not be a hardship for IPFIX. >> >> >> > >> >> >> > But, I believe that if you are going to make a design >> >> >> choice it should >> >> >> > be motivated by design goals. What benefit do you gain >> >> by limiting >> >> >> > IPFIX to a simple uni-directional >> >> >> > flow model? Is a uni-directional flow data model the best >> >> >> > data model for reporting all the metrics and network >> >> behaviors that >> >> >> > relate to network flows? Is it the best data model for >> >> >> supporting all >> >> >> > the existing commercial network flow data? >> >> >> > >> >> >> > But what about even the simplest of applications of flow >> >> data? Is a >> >> >> > uni-directional flow data model the best solution for >> >> reporting the >> >> >> > RTT of a ping volley? The empirical bulk transfer >> >> >> properties of a TCP >> >> >> > connection? How about TCP packet retransmission? Existence >> >> >> of routing >> >> >> > failures in an OSPF routed network? Application >> response time? >> >> >> > Rerouting indications? Reachability fault detection? >> >> >> Access policy >> >> >> > validation? Connectivity status? >> >> >> > >> >> >> > Even FTP supports both ASCII and binary data transfer. >> >> >> Handling both >> >> >> > uni-directional data and bi-directional data in IPFIX >> >> >> should be pretty >> >> >> > easy stuff. >> >> >> > >> >> >> > >> >> >> > Carter >> >> >> > >> >> >> > Carter Bullard >> >> >> > QoSient, LLC >> >> >> > 300 E. 56th Street, Suite 18K >> >> >> > New York, New York 10022 >> >> >> > >> >> >> > carter@qosient.com >> >> >> > Phone +1 212 588-9133 >> >> >> > Fax +1 212 588-9134 >> >> >> > http://qosient.com >> >> >> > >> >> >> > >> >> >> > >> >> >> > >> >> >> > > -----Original Message----- >> >> >> > > From: majordomo listserver >> >> >> [mailto:majordomo@mil.doit.wisc.edu] On >> >> >> > > Behalf Of Dave Plonka >> >> >> > > Sent: Monday, January 07, 2002 4:52 PM >> >> >> > > To: ipfix-req@net.doit.wisc.edu >> >> >> > > Subject: [ipfix-req] unidirection vs. bidirectional flows >> >> >> > > >> >> >> > > >> >> >> > > >> >> >> > > IPFIX folks, >> >> >> > > >> >> >> > > As many of you recall, whether or not IPFIX should >> specify a >> >> >> > > unidirectional or bidirectional flow definition was a hot >> >> >> topic at >> >> >> > > the WG meeting. >> >> >> > > >> >> >> > > I think that specifying *something* gets us far along in >> >> >> > > communicating what a "flow" is. Personally, I >> would like us >> >> >> > > to specify one or the other and I am in favor a >> unidirection >> >> >> > > flow definition. >> >> >> > > >> >> >> > > Here are some thoughts about arguments made for >> >> >> bidirectional flows: >> >> >> > > >> >> >> > > A unidirectional flow definition means that the atomic >> >> >> flow report >> >> >> > > record is unidirection during the reporting/export >> phase. For >> >> >> > > instance, a single packet counter for a given >> >> >> > > protocol/srcaddr/srcport/dstaddr/dstport n-tuple. >> >> However, since >> >> >> > > the exporter is somewhat of a "black box", it need not be >> >> >> > > constrained to a unidirectional model internally. >> >> >> > > >> >> >> > > A unidirectional flow definition does not prevent a >> collector >> >> >> > > implementor from using a bi-directional notion of flows >> >> >> internally >> >> >> > > nor does it prevent it from being stored in a bidirectional >> >> >> > > way prior to export. Conversely if we specify a >> bidirectional >> >> >> > > definition, it does disqualify unidirectional >> implementations. >> >> >> > > >> >> >> > > For those that wish to perform measurements which >> may require >> >> >> > > bidirectional flow information (perhaps because start/end >> >> >> timestamp >> >> >> > > synchronization is required for flow pairs), the IPFIX >> >> data model >> >> >> > > and transport could define a mechanism by which the >> >> >> exporter could >> >> >> > > inform the collector about which unidirectional flow >> >> records are >> >> >> > > matched pairs. That is, the exporter could help the >> >> >> collector find >> >> >> > > the matched pairs of flow information, perhaps by >> >> >> exporting the flow >> >> >> > > records in pairs or by generating a numeric identifier >> >> that is an >> >> >> > > attribute of both flow records. Of course only >> bidirectional >> >> >> > > collectors would understand this convention, but a >> >> >> > > unidirection collector could ignore it and still >> make use of >> >> >> > > the exported flow information from an exporter that >> >> >> > > implemented this "bidirectional flow pairing" feature. >> >> >> > > >> >> >> > > Dave >> >> >> > > >> >> >> > > -- >> >> >> > > ":) You will make many changes before settling >> >> satisfactorily. :)" >> >> >> > > - the actual message (complete with smiley faces) that I >> >> >> received >> >> >> > > in a >> >> >> > > fortune cookie at dinner after the IPFIX WG meeting >> >> >> during IETF 52 >> >> >> > > in Salt Lake City >> >> >> > > >> >> >> > > -- >> >> >> > > Help mailto:majordomo@net.doit.wisc.edu and >> say "help" >> >> >> > > in message body >> >> >> > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say >> >> >> "unsubscribe >> >> >> > > ipfix" in message body >> >> >> > > Archive http://ipfix.doit.wisc.edu/archive/ >> >> >> > > >> >> >> > > >> >> >> > >> >> >> > >> >> >> > >> >> >> > -- >> >> >> > Help mailto:majordomo@net.doit.wisc.edu and say >> >> >> "help" in message body >> >> >> > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say >> >> "unsubscribe >> >> >> > ipfix" in message body >> >> >> > Archive http://ipfix.doit.wisc.edu/archive/ >> >> >> > >> >> >> >> >> >> >> >> >> >> >> > >> >> > >> >> > >> >> > -- >> >> > Help mailto:majordomo@net.doit.wisc.edu and say >> >> "help" in message body >> >> > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say >> "unsubscribe >> >> > ipfix" in message body >> >> > Archive http://ipfix.doit.wisc.edu/archive/ >> >> > >> >> >> >> >> >> >> >> -- >> >> Help mailto:majordomo@net.doit.wisc.edu and say "help" >> >> in message body >> >> Unsubscribe mailto:majordomo@net.doit.wisc.edu and say >> "unsubscribe >> >> ipfix" in message body >> >> Archive http://ipfix.doit.wisc.edu/archive/ >> >> >> >> >> > >> > >> > >> > -- >> > Help mailto:majordomo@net.doit.wisc.edu and say >> "help" in message body >> > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe >> > ipfix" in message body >> > Archive http://ipfix.doit.wisc.edu/archive/ >> > >> >> >> >> > > -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Tue Jan 8 11:41:36 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA28970 for ; Tue, 8 Jan 2002 11:41:36 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16NyzB-0001Rl-00 for ipfix-list@mil.doit.wisc.edu; Tue, 08 Jan 2002 10:20:25 -0600 Received: from host3.riverstonenet.com ([63.113.148.3] helo=exc-sc1.yagosys.com) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 16Nyz7-0001R0-00 for ipfix-arch@net.doit.wisc.edu; Tue, 08 Jan 2002 10:20:21 -0600 Received: from riverstonenet.com (134.141.180.100 [134.141.180.100]) by exc-sc1.yagosys.com with SMTP (Microsoft Exchange Internet Mail Service Version 5.5.2653.13) id X7WVYKTA; Tue, 8 Jan 2002 08:19:22 -0800 Message-ID: <3C3B1C0F.AEFD1C8D@riverstonenet.com> Date: Tue, 08 Jan 2002 11:19:27 -0500 From: calato@riverstonenet.com X-Mailer: Mozilla 4.75C-CCK-MCD NSCPCD475 [en] (X11; U; SunOS 5.6 sun4u) X-Accept-Language: en MIME-Version: 1.0 To: "'ipfix-arch@net.doit.wisc.edu'" Subject: Re: [ipfix-arch] configuration information. References: <4F973E944951D311B6660008C7917CF00837E17A@zsc4c008.us.nortel.com> <3C3A14F5.AD6A9AFC@cisco.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Ganesh Sadasivan wrote: > > Reinaldo, > I agree with your point that it is non-trivial to derive the complete > relationship > between the packets collected various templates. My question to the > group is that > whether IPFIX should spend time on architecting how this detailed > config > information is sent to the collector so that the collector gets this > information inband > or whether this can be dealt by the collector in an adhoc manner. > If we go by the former case to what level of details should the > collector be > made aware of about templates, packet selection and their > relationships. > > Thanks > Ganesh > > Reinaldo Penno wrote: > > > > > > > Hello, > > > > How a collector would know that a priori? I mean, I can have such > > general templates that it would be very difficult to account for > > duplicate measurements (even in non real-time). > > > > If you have very specific templates such as srcIP, destIP, etc, it's > > not to hard find records in common. But you can have things like: > > > > src_AnyIP Dest_Yahoo.com Proto_Any and > > src_AnyIP Dest_AS3462 Proto_HTTP. I assume here you mean that there are 2 counters and 2 "flows" reported. One has (to use cisco terms) a template of "dest-ip, byes, packets" and the other "dest-AS, proto, byes, packets". If the device uses a multi-match method then the device "knows" that a packet may be added to both buckets. It should tell the collector that these templates overlap. If on the other hand the device uses a first match method then the counts do not overlap. When the application is processing records reported via these templates it "knows" whether adding the byes/packets from these templates makes sense. > > > > How do you know a priori that the second rule will have records that > > are a subset of the first? Am I missing something? > > > > thanks, > > > > Reinaldo > > > > > > > -----Original Message----- > > > From: Ganesh Sadasivan [mailto:gsadasiv@cisco.com] > > > Sent: Monday, January 07, 2002 12:09 PM > > > To: ipfix-arch@net.doit.wisc.edu > > > Subject: [ipfix-arch] configuration information. > > > > > > > > > Hi, > > > One of the questions raised during the V9 presenatation if IETF > > was > > > "how a collector would know whether or not the same > > > packets or bytes were being reported in more than one flow > > record > > > (e.g. using a different template)." (quoting from the minutes). > > > > > This requires the collector to have configuration information > > > regading > > > the different templates, and the mode of selection of packets > > for > > > each > > > of these templates at each of the observation points. > > > This information can be either > > > 1. Communicated to the collector from the measuring device > > > inband as > > > a part > > > of the export protocol. > > > 2. The collector gets the required configuration out of band > > (.i.e > > > not thru > > > the export protocol. > > > > > > What are the thoughts of the group on this topic? > > > If we are going by 1., should it be a part of the arch- spec? > > One > > > think we > > > need to keep in mind is that things like , details on method of > > > > > packet > > > selection could vary widely with implementations. > > > > > > Thanks > > > Ganesh > > > > > > > > > > > > -- > > > Help mailto:majordomo@net.doit.wisc.edu and say "help" > > > in message body > > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > > "unsubscribe ipfix" in message body > > > Archive http://ipfix.doit.wisc.edu/archive/ > > > -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Tue Jan 8 11:45:02 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA29142 for ; Tue, 8 Jan 2002 11:45:02 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16Nyzh-0001SD-00 for ipfix-list@mil.doit.wisc.edu; Tue, 08 Jan 2002 10:20:57 -0600 Received: from relay1.pair.com ([209.68.1.20] helo=relay.pair.com) by mil.doit.wisc.edu with smtp (Exim 3.13 #1) id 16Nyzf-0001S8-00 for ipfix-req@net.doit.wisc.edu; Tue, 08 Jan 2002 10:20:55 -0600 Received: (qmail 30424 invoked from network); 8 Jan 2002 16:20:52 -0000 Received: from pool-162-83-252-213.ny5030.east.verizon.net (HELO sphynx) (162.83.252.213) by relay1.pair.com with SMTP; 8 Jan 2002 16:20:52 -0000 X-pair-Authenticated: 162.83.252.213 Reply-To: From: "Carter Bullard" To: "'Benoit Claise'" , Cc: , Subject: RE: [ipfix-req] unidirection vs. bidirectional flows Date: Tue, 8 Jan 2002 11:20:43 -0500 Organization: QoSient, LLC Message-ID: <5C8959A16A71B449AE793CF52FBBED66018FD1@ptah.newyork.qosient.com> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook, Build 10.0.3416 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 In-Reply-To: <200201081556.g08FuK613751@bru-cse-222.cisco.com> Importance: Normal Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Hey Benoit, You know you don't support netflow on the complete line of Cisco routers and switches. Carter Carter Bullard QoSient, LLC 300 E. 56th Street, Suite 18K New York, New York 10022 carter@qosient.com Phone +1 212 588-9133 Fax +1 212 588-9134 http://qosient.com > -----Original Message----- > From: majordomo listserver > [mailto:majordomo@mil.doit.wisc.edu] On Behalf Of Benoit Claise > Sent: Tuesday, January 08, 2002 10:56 AM > To: quittek@ccrle.nec.de; carter@qosient.com > Cc: plonka@doit.wisc.edu; ipfix-req@net.doit.wisc.edu > Subject: RE: [ipfix-req] unidirection vs. bidirectional flows > > > Hi Carter, > > > > Hey Juergen, > > You know, you have to buy special cards to get netflow > support in a > > Cisco router. I'm interested in how your going to make > > uni-directional flow record reporting cheap. > > This is not correct! This function is part of the software. > I know of only device which needs a special daughter card, > and I don't think this still applies. > > Regards, Benoit. > > > > > Carter > > > > Carter Bullard > > QoSient, LLC > > 300 E. 56th Street, Suite 18K > > New York, New York 10022 > > > > carter@qosient.com > > Phone +1 212 588-9133 > > Fax +1 212 588-9134 > > http://qosient.com > > > > > -----Original Message----- > > > From: Juergen Quittek [mailto:quittek@ccrle.nec.de] > > > Sent: Tuesday, January 08, 2002 8:23 AM > > > To: carter@qosient.com > > > Cc: plonka@doit.wisc.edu; ipfix-req@net.doit.wisc.edu > > > Subject: RE: [ipfix-req] unidirection vs. bidirectional flows > > > > > > > > > --On 08 January 2002 07:34 -0500 Carter Bullard wrote: > > > > > > > Hey Juergen, > > > > Just a note, the majority of switches and routers > > > > today (> 90%) cannot support uni-directional flow record > > > > > > What routers are you talking about? Isn't NetFlow > available for most > > > CisCo and Juniper boxes? > > > > > > > reporting. However, virtually all switches and routers (> 99%) > > > > support RMON-style interface statistics, which are L1/L2 > > > > bi-directional flow metrics. > > > > > > > > I believe that adoption of a uni-directional flow model excludes > > > > basically the entire population of legacy networking > devices that > > > > could easily report their existing bi-directional flow > > > metrics using > > > > IPFIX. > > > > > > > > Wouldn't this violate your fundamental constraint? > > > > > > Not at all. No one is trying to forbid existing technologies. > > > But this is about IPFIX requirements. I said I am not sure > > > whether we should specify uni- or bi-directional in the > > > requirements at all, but I am sure I do not want to have > > > "expensive" requirements. If you explain to me how to make > > > bi-directional flows cheap, I will find it easier considering > > > them as a potential requirement. > > > > > > Juergen > > > -- > > > Juergen Quittek quittek@ccrle.nec.de Tel: +49 > 6221 90511-15 > > > NEC Europe Ltd., Network Laboratories Fax: +49 > 6221 90511-55 > > > Adenauerplatz 6, 69115 Heidelberg, Germany > http://www.ccrle.nec.de > > > > > > > > > > > Carter > > > > > > > > Carter Bullard > > > > QoSient, LLC > > > > 300 E. 56th Street, Suite 18K > > > > New York, New York 10022 > > > > > > > > carter@qosient.com > > > > Phone +1 212 588-9133 > > > > Fax +1 212 588-9134 > > > > http://qosient.com > > > > > > > > > > > >> -----Original Message----- > > > >> From: majordomo listserver > [mailto:majordomo@mil.doit.wisc.edu] > > > >> On > > > >> Behalf Of Juergen Quittek > > > >> Sent: Tuesday, January 08, 2002 6:24 AM > > > >> To: carter@qosient.com; 'Vamsidhar Valluri' > > > >> Cc: plonka@doit.wisc.edu; ipfix-req@net.doit.wisc.edu > > > >> Subject: RE: [ipfix-req] unidirection vs. bidirectional flows > > > >> > > > >> > > > >> Hi Carter, > > > >> > > > >> One of the very important constraints we applied to each > > > requirements > > > >> so far, is that it must be possible to for routers as > well as for > > > >> probes to implement the respective requirement with reasonable > > > >> effort. > > > >> > > > >> (If anyone finds out that one or more of the requirements > > > appears to > > > >> be expensive, then please speak up!) > > > >> > > > >> Facing the hardware acrchitecture of a modern router, > > > >> I consider a bi-directional flow model to be too expensive > > > to be part > > > >> of the requirements (and consequently also too expensive > > > to be part > > > >> of the general architecture on the exporter side). > > > >> > > > >> The reason is that at line interface cards you typically > > > spent very > > > >> different effort on treatment on incoming and outgoing > > > packets. For > > > >> incoming packets you do a lot of > > > >> processing: checksums, TTL processing, routing based on cached > > > >> routing tables, ... Here you touch header fields > anyway and it is > > > >> relatively easy and relatively cheap to add metering > > > functions. For > > > >> outgoing packets the treatment is different. You just try > > > to get rid > > > >> of them as fast as possible (except from scheduling thme > > > first). Here > > > >> it would be rather expensive to investigate header > fields again > > > >> and it might also reduce your maximum thoughput. Therefore a > > > >> device just metering incoming packets should be able > to match the > > > >> requirements. > > > >> > > > >> Now, if we have a bi-directional flow model, you require such a > > > >> device to merge all its input measurements performed > at different > > > >> line cards by looking for measured uni-directional flows > > > belongig to > > > >> the same bi-directinal one. This task might be done at > a control > > > >> processor, but it creates additional cost and it might > > > slow down the > > > >> system. I believe that merging of these measurements is a > > > perfect job > > > >> for an analysis application and not for a meter. > > > >> > > > >> Definitely it is nothing we should ask every meter to > perform by > > > >> making it a MUST requirement! > > > >> > > > >> Consequently, my position is that IF we add a flow model to the > > > >> requirements, then it should be the uni-diractional one. > > > However, I > > > >> am not yet convinced, that we should specify the model at > > > all in the > > > >> requirements. > > > >> > > > >> Best wishes, > > > >> > > > >> Juergen > > > >> -- > > > >> Juergen Quittek quittek@ccrle.nec.de Tel: +49 > 6221 90511-15 > > > >> NEC Europe Ltd., Network Laboratories Fax: +49 > 6221 90511-55 > > > >> Adenauerplatz 6, 69115 Heidelberg, Germany > > > http://www.ccrle.nec.de > > > >> > > > >> > > > >> --On 07 January 2002 19:42 -0500 Carter Bullard > > > > > > >> wrote: > > > >> > > > >> > Hey Vamsi, > > > >> > Hmmm, is the interface a part of the flow > defintion? Not a > > > >> > problem. Lets look at how this may work. Suppose there > > > is a SRC > > > >> > interface and a DST interface, so that a flow is > simply all the > > > >> > packets moving from SRC -> DST. The return flow would > > > be all the > > > >> > packets moving from DST -> SRC. > > > >> > > > > >> > So the flow is reported as SRC <-> DST, with the > > > simple addition > > > >> > of the DST -> SRC metrics. Makes more sense than > independantly > > > >> > reporting two half-duplex flow records, which could > be skewed in > > > >> > time so badly, that the two records cannot be > correlated at all. > > > >> > > > > >> > > > > >> > Carter > > > >> > > > > >> > Carter Bullard > > > >> > QoSient, LLC > > > >> > 300 E. 56th Street, Suite 18K > > > >> > New York, New York 10022 > > > >> > > > > >> > carter@qosient.com > > > >> > Phone +1 212 588-9133 > > > >> > Fax +1 212 588-9134 > > > >> > http://qosient.com > > > >> > > > > >> > > > > >> >> -----Original Message----- > > > >> >> From: Vamsidhar Valluri [mailto:vvalluri@cisco.com] > > > >> >> Sent: Monday, January 07, 2002 6:58 PM > > > >> >> To: Carter Bullard > > > >> >> Cc: plonka@doit.wisc.edu; ipfix-req@net.doit.wisc.edu > > > >> >> Subject: RE: [ipfix-req] unidirection vs. > bidirectional flows > > > >> >> > > > >> >> > > > >> >> > > > >> >> > > > >> >> On Mon, 7 Jan 2002, Carter Bullard wrote: > > > >> >> > > > >> >> > Gentle people, > > > >> >> > There is absolutely no reason for IPFIX to limit > the type of > > > >> >> > flow data that it supports. The difference between > > > >> >> uni-directional > > > >> >> > and bi-directional flow records is simply the type of > > > >> >> metrics that are > > > >> >> > reported. The flow descriptors and keys are all the > > > >> same. Because > > > >> >> > IPFIX must support > > > >> >> ^^^^^ > > > >> >> > > > >> >> If "keys" you are refering to are the fields used > to identify > > > >> >> a > > > >> >> flow then "SRC interface" do not fit into the statement > > > "The flow > > > >> >> descriptors and keys are all same for both > unidirectional and > > > >> >> bi-directional flow records" mentioned above because > > > SRC interface > > > >> >> will make it unidirectional. > > > >> >> > > > >> >> -vamsi > > > >> >> > > > >> >> > > > >> >> > reporting metrics that each vendors will not > implement, such > > > >> >> as jitter > > > >> >> > and one-way delay, which are the few metrics that are > > > >> meaningful in > > > >> >> > uni-directional flow data models, supporting metrics > > > >> that report on > > > >> >> > return traffic should not be a hardship for IPFIX. > > > >> >> > > > > >> >> > But, I believe that if you are going to make a design > > > >> >> choice it should > > > >> >> > be motivated by design goals. What benefit do you gain > > > >> by limiting > > > >> >> > IPFIX to a simple uni-directional > > > >> >> > flow model? Is a uni-directional flow data > model the best > > > >> >> > data model for reporting all the metrics and network > > > >> behaviors that > > > >> >> > relate to network flows? Is it the best data model for > > > >> >> supporting all > > > >> >> > the existing commercial network flow data? > > > >> >> > > > > >> >> > But what about even the simplest of applications of flow > > > >> data? Is a > > > >> >> > uni-directional flow data model the best solution for > > > >> reporting the > > > >> >> > RTT of a ping volley? The empirical bulk transfer > > > >> >> properties of a TCP > > > >> >> > connection? How about TCP packet retransmission? > Existence > > > >> >> of routing > > > >> >> > failures in an OSPF routed network? Application > > > response time? > > > >> >> > Rerouting indications? Reachability fault detection? > > > >> >> Access policy > > > >> >> > validation? Connectivity status? > > > >> >> > > > > >> >> > Even FTP supports both ASCII and binary data transfer. > > > >> >> Handling both > > > >> >> > uni-directional data and bi-directional data in IPFIX > > > >> >> should be pretty > > > >> >> > easy stuff. > > > >> >> > > > > >> >> > > > > >> >> > Carter > > > >> >> > > > > >> >> > Carter Bullard > > > >> >> > QoSient, LLC > > > >> >> > 300 E. 56th Street, Suite 18K > > > >> >> > New York, New York 10022 > > > >> >> > > > > >> >> > carter@qosient.com > > > >> >> > Phone +1 212 588-9133 > > > >> >> > Fax +1 212 588-9134 > > > >> >> > http://qosient.com > > > >> >> > > > > >> >> > > > > >> >> > > > > >> >> > > > > >> >> > > -----Original Message----- > > > >> >> > > From: majordomo listserver > > > >> >> [mailto:majordomo@mil.doit.wisc.edu] On > > > >> >> > > Behalf Of Dave Plonka > > > >> >> > > Sent: Monday, January 07, 2002 4:52 PM > > > >> >> > > To: ipfix-req@net.doit.wisc.edu > > > >> >> > > Subject: [ipfix-req] unidirection vs. > bidirectional flows > > > >> >> > > > > > >> >> > > > > > >> >> > > > > > >> >> > > IPFIX folks, > > > >> >> > > > > > >> >> > > As many of you recall, whether or not IPFIX should > > > specify a > > > >> >> > > unidirectional or bidirectional flow definition > was a hot > > > >> >> topic at > > > >> >> > > the WG meeting. > > > >> >> > > > > > >> >> > > I think that specifying *something* gets us far along in > > > >> >> > > communicating what a "flow" is. Personally, I > > > would like us > > > >> >> > > to specify one or the other and I am in favor a > > > unidirection > > > >> >> > > flow definition. > > > >> >> > > > > > >> >> > > Here are some thoughts about arguments made for > > > >> >> bidirectional flows: > > > >> >> > > > > > >> >> > > A unidirectional flow definition means that the atomic > > > >> >> flow report > > > >> >> > > record is unidirection during the reporting/export > > > phase. For > > > >> >> > > instance, a single packet counter for a given > > > >> >> > > protocol/srcaddr/srcport/dstaddr/dstport n-tuple. > > > >> However, since > > > >> >> > > the exporter is somewhat of a "black box", it > need not be > > > >> >> > > constrained to a unidirectional model internally. > > > >> >> > > > > > >> >> > > A unidirectional flow definition does not prevent a > > > collector > > > >> >> > > implementor from using a bi-directional notion of flows > > > >> >> internally > > > >> >> > > nor does it prevent it from being stored in a > > > >> >> > > bidirectional > > > >> >> > > way prior to export. Conversely if we specify a > > > bidirectional > > > >> >> > > definition, it does disqualify unidirectional > > > implementations. > > > >> >> > > > > > >> >> > > For those that wish to perform measurements which > > > may require > > > >> >> > > bidirectional flow information (perhaps because > start/end > > > >> >> timestamp > > > >> >> > > synchronization is required for flow pairs), the IPFIX > > > >> data model > > > >> >> > > and transport could define a mechanism by which the > > > >> >> exporter could > > > >> >> > > inform the collector about which unidirectional flow > > > >> records are > > > >> >> > > matched pairs. That is, the exporter could help the > > > >> >> collector find > > > >> >> > > the matched pairs of flow information, perhaps by > > > >> >> exporting the flow > > > >> >> > > records in pairs or by generating a numeric identifier > > > >> that is an > > > >> >> > > attribute of both flow records. Of course only > > > bidirectional > > > >> >> > > collectors would understand this convention, but a > > > >> >> > > unidirection collector could ignore it and still > > > make use of > > > >> >> > > the exported flow information from an exporter that > > > >> >> > > implemented this "bidirectional flow pairing" feature. > > > >> >> > > > > > >> >> > > Dave > > > >> >> > > > > > >> >> > > -- > > > >> >> > > ":) You will make many changes before settling > > > >> satisfactorily. :)" > > > >> >> > > - the actual message (complete with smiley > faces) that I > > > >> >> received > > > >> >> > > in a > > > >> >> > > fortune cookie at dinner after the IPFIX WG meeting > > > >> >> during IETF 52 > > > >> >> > > in Salt Lake City > > > >> >> > > > > > >> >> > > -- > > > >> >> > > Help mailto:majordomo@net.doit.wisc.edu and > > > say "help" > > > >> >> > > in message body > > > >> >> > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > > >> >> "unsubscribe > > > >> >> > > ipfix" in message body > > > >> >> > > Archive http://ipfix.doit.wisc.edu/archive/ > > > >> >> > > > > > >> >> > > > > > >> >> > > > > >> >> > > > > >> >> > > > > >> >> > -- > > > >> >> > Help mailto:majordomo@net.doit.wisc.edu and say > > > >> >> "help" in message body > > > >> >> > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > > >> "unsubscribe > > > >> >> > ipfix" in message body > > > >> >> > Archive http://ipfix.doit.wisc.edu/archive/ > > > >> >> > > > > >> >> > > > >> >> > > > >> >> > > > >> > > > > >> > > > > >> > > > > >> > -- > > > >> > Help mailto:majordomo@net.doit.wisc.edu and say > > > >> "help" in message body > > > >> > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > > "unsubscribe > > > >> > ipfix" in message body > > > >> > Archive http://ipfix.doit.wisc.edu/archive/ > > > >> > > > > >> > > > >> > > > >> > > > >> -- > > > >> Help mailto:majordomo@net.doit.wisc.edu and say "help" > > > >> in message body > > > >> Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > > "unsubscribe > > > >> ipfix" in message body > > > >> Archive http://ipfix.doit.wisc.edu/archive/ > > > >> > > > >> > > > > > > > > > > > > > > > > -- > > > > Help mailto:majordomo@net.doit.wisc.edu and say > > > "help" in message body > > > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > > > "unsubscribe > > > > ipfix" in message body > > > > Archive http://ipfix.doit.wisc.edu/archive/ > > > > > > > > > > > > > > > > > > > > > > > > -- > > Help mailto:majordomo@net.doit.wisc.edu and say > "help" in message body > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe > > ipfix" in message body > > Archive http://ipfix.doit.wisc.edu/archive/ > > > > -- > Help mailto:majordomo@net.doit.wisc.edu and say "help" > in message body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ > > -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Tue Jan 8 11:48:47 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA29283 for ; Tue, 8 Jan 2002 11:48:47 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16Nz8w-0001dN-00 for ipfix-list@mil.doit.wisc.edu; Tue, 08 Jan 2002 10:30:30 -0600 Received: from host3.riverstonenet.com ([63.113.148.3] helo=exc-sc1.yagosys.com) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 16Nz8u-0001bu-00 for ipfix-req@net.doit.wisc.edu; Tue, 08 Jan 2002 10:30:28 -0600 Received: from riverstonenet.com (134.141.180.100 [134.141.180.100]) by exc-sc1.yagosys.com with SMTP (Microsoft Exchange Internet Mail Service Version 5.5.2653.13) id X7WVYKVM; Tue, 8 Jan 2002 08:29:29 -0800 Message-ID: <3C3B1E6E.1DF1350C@riverstonenet.com> Date: Tue, 08 Jan 2002 11:29:34 -0500 From: calato@riverstonenet.com X-Mailer: Mozilla 4.75C-CCK-MCD NSCPCD475 [en] (X11; U; SunOS 5.6 sun4u) X-Accept-Language: en MIME-Version: 1.0 To: "ipfix-req@net.doit.wisc.edu" Subject: Re: [ipfix-req] unidirection vs. bidirectional flows References: <5C8959A16A71B449AE793CF52FBBED66018FCC@ptah.newyork.qosient.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Carter, I'm a bit confused. Why doesn't the ability to provide bi-directional info, as Dave described, meet your needs? Those devices that can do bi-directional will report it that way, those that can't will report it as uni-directional. Some down stream application will have to do the work to turn 2 uni-directional flows into one bi-directional one, if that is what's needed. And if uni-directional is needed the app will split it apart. But if you require bi-directional reporting, many devices wont be able to do it. Which then eliminates any down stream analysis. Paul Carter Bullard wrote: > > Hey Vamsi, > Hmmm, is the interface a part of the flow defintion? > Not a problem. Lets look at how this may work. Suppose there > is a SRC interface and a DST interface, so that a flow is > simply all the packets moving from SRC -> DST. The return flow > would be all the packets moving from DST -> SRC. > > So the flow is reported as SRC <-> DST, with the simple > addition of the DST -> SRC metrics. Makes more sense than > independantly reporting two half-duplex flow records, which > could be skewed in time so badly, that the two records cannot > be correlated at all. > > Carter > > Carter Bullard > QoSient, LLC > 300 E. 56th Street, Suite 18K > New York, New York 10022 > > carter@qosient.com > Phone +1 212 588-9133 > Fax +1 212 588-9134 > http://qosient.com > > > -----Original Message----- > > From: Vamsidhar Valluri [mailto:vvalluri@cisco.com] > > Sent: Monday, January 07, 2002 6:58 PM > > To: Carter Bullard > > Cc: plonka@doit.wisc.edu; ipfix-req@net.doit.wisc.edu > > Subject: RE: [ipfix-req] unidirection vs. bidirectional flows > > > > > > > > > > On Mon, 7 Jan 2002, Carter Bullard wrote: > > > > > Gentle people, > > > There is absolutely no reason for IPFIX to limit the type > > > of flow data that it supports. The difference between > > uni-directional > > > and bi-directional flow records is simply the type of > > metrics that are > > > reported. The flow descriptors and keys are all the same. Because > > > IPFIX must support > > ^^^^^ > > > > If "keys" you are refering to are the fields used to identify > > a flow then "SRC interface" do not fit into the statement > > "The flow descriptors and keys are all same for both > > unidirectional and bi-directional flow records" mentioned > > above because SRC interface will make it unidirectional. > > > > -vamsi > > > > > > >reporting metrics that each vendors will not implement, such > > as jitter > > >and one-way delay, which are the few metrics that are meaningful in > > >uni-directional flow data models, supporting metrics that report on > > >return traffic should not be a hardship for IPFIX. > > > > > > But, I believe that if you are going to make a design > > choice it should > > > be motivated by design goals. What benefit do you gain by limiting > > > IPFIX to a simple uni-directional > > > flow model? Is a uni-directional flow data model the best > > > data model for reporting all the metrics and network behaviors that > > > relate to network flows? Is it the best data model for > > supporting all > > > the existing commercial network flow data? > > > > > > But what about even the simplest of applications of flow data? Is a > > > uni-directional flow data model the best solution for reporting the > > > RTT of a ping volley? The empirical bulk transfer > > properties of a TCP > > > connection? How about TCP packet retransmission? Existence > > of routing > > > failures in an OSPF routed network? Application response time? > > > Rerouting indications? Reachability fault detection? > > Access policy > > > validation? Connectivity status? > > > > > > Even FTP supports both ASCII and binary data transfer. > > Handling both > > > uni-directional data and bi-directional data in IPFIX > > should be pretty > > > easy stuff. > > > > > > > > > Carter > > > > > > Carter Bullard > > > QoSient, LLC > > > 300 E. 56th Street, Suite 18K > > > New York, New York 10022 > > > > > > carter@qosient.com > > > Phone +1 212 588-9133 > > > Fax +1 212 588-9134 > > > http://qosient.com > > > > > > > > > > > > > > > > -----Original Message----- > > > > From: majordomo listserver > > [mailto:majordomo@mil.doit.wisc.edu] On > > > > Behalf Of Dave Plonka > > > > Sent: Monday, January 07, 2002 4:52 PM > > > > To: ipfix-req@net.doit.wisc.edu > > > > Subject: [ipfix-req] unidirection vs. bidirectional flows > > > > > > > > > > > > > > > > IPFIX folks, > > > > > > > > As many of you recall, whether or not IPFIX should specify a > > > > unidirectional or bidirectional flow definition was a hot > > topic at > > > > the WG meeting. > > > > > > > > I think that specifying *something* gets us far along in > > > > communicating what a "flow" is. Personally, I would like us to > > > > specify one or the other and I am in favor a unidirection flow > > > > definition. > > > > > > > > Here are some thoughts about arguments made for > > bidirectional flows: > > > > > > > > A unidirectional flow definition means that the atomic > > flow report > > > > record is unidirection during the reporting/export phase. For > > > > instance, a single packet counter for a given > > > > protocol/srcaddr/srcport/dstaddr/dstport n-tuple. However, since > > > > the exporter is somewhat of a "black box", it need not be > > > > constrained to a unidirectional model internally. > > > > > > > > A unidirectional flow definition does not prevent a collector > > > > implementor from using a bi-directional notion of flows > > internally > > > > nor does it prevent it from being stored in a bidirectional way > > > > prior to export. Conversely if we specify a bidirectional > > > > definition, it does disqualify unidirectional implementations. > > > > > > > > For those that wish to perform measurements which may require > > > > bidirectional flow information (perhaps because start/end > > timestamp > > > > synchronization is required for flow pairs), the IPFIX data model > > > > and transport could define a mechanism by which the > > exporter could > > > > inform the collector about which unidirectional flow records are > > > > matched pairs. That is, the exporter could help the > > collector find > > > > the matched pairs of flow information, perhaps by > > exporting the flow > > > > records in pairs or by generating a numeric identifier that is an > > > > attribute of both flow records. Of course only bidirectional > > > > collectors would understand this convention, but a > > > > unidirection collector could ignore it and still make use of > > > > the exported flow information from an exporter that > > > > implemented this "bidirectional flow pairing" feature. > > > > > > > > Dave > > > > > > > > -- > > > > ":) You will make many changes before settling satisfactorily. :)" > > > > - the actual message (complete with smiley faces) that I > > received > > > > in a > > > > fortune cookie at dinner after the IPFIX WG meeting > > during IETF 52 > > > > in Salt Lake City > > > > > > > > -- > > > > Help mailto:majordomo@net.doit.wisc.edu and say "help" > > > > in message body > > > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > "unsubscribe > > > > ipfix" in message body > > > > Archive http://ipfix.doit.wisc.edu/archive/ > > > > > > > > > > > > > > > > > > > > -- > > > Help mailto:majordomo@net.doit.wisc.edu and say > > "help" in message body > > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe > > > ipfix" in message body > > > Archive http://ipfix.doit.wisc.edu/archive/ > > > > > > > > > > > -- > Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Tue Jan 8 11:53:20 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA29484 for ; Tue, 8 Jan 2002 11:53:20 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16NzBc-0001i8-00 for ipfix-list@mil.doit.wisc.edu; Tue, 08 Jan 2002 10:33:17 -0600 Received: from relay1.pair.com ([209.68.1.20] helo=relay.pair.com) by mil.doit.wisc.edu with smtp (Exim 3.13 #1) id 16NzBb-0001i0-00 for ipfix-req@net.doit.wisc.edu; Tue, 08 Jan 2002 10:33:15 -0600 Received: (qmail 32369 invoked from network); 8 Jan 2002 16:33:14 -0000 Received: from pool-162-83-252-213.ny5030.east.verizon.net (HELO sphynx) (162.83.252.213) by relay1.pair.com with SMTP; 8 Jan 2002 16:33:14 -0000 X-pair-Authenticated: 162.83.252.213 Reply-To: From: "Carter Bullard" To: "'Juergen Quittek'" Cc: , Subject: RE: [ipfix-req] unidirection vs. bidirectional flows Date: Tue, 8 Jan 2002 11:33:02 -0500 Organization: QoSient, LLC Message-ID: <5C8959A16A71B449AE793CF52FBBED66018FD2@ptah.newyork.qosient.com> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook, Build 10.0.3416 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 In-Reply-To: <1182459298.1010499773@[192.168.102.164]> Importance: Normal Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Hey Juergen, There are a significant number of Cisco routers and switches that do not support netflow. And Cisco/Riverstone do not constitute the complete set of router/switch vendors on the planet. My entire position is that IPFIX should not make any requirements/restrictions on the data model. Carter Carter Bullard QoSient, LLC 300 E. 56th Street, Suite 18K New York, New York 10022 carter@qosient.com Phone +1 212 588-9133 Fax +1 212 588-9134 http://qosient.com > -----Original Message----- > From: Juergen Quittek [mailto:quittek@ccrle.nec.de] > Sent: Tuesday, January 08, 2002 8:23 AM > To: carter@qosient.com > Cc: plonka@doit.wisc.edu; ipfix-req@net.doit.wisc.edu > Subject: RE: [ipfix-req] unidirection vs. bidirectional flows > > > --On 08 January 2002 07:34 -0500 Carter Bullard wrote: > > > Hey Juergen, > > Just a note, the majority of switches and routers > > today (> 90%) cannot support uni-directional flow record > > What routers are you talking about? Isn't NetFlow available > for most CisCo and Juniper boxes? > > > reporting. However, virtually all switches and routers > > (> 99%) support RMON-style interface statistics, which > > are L1/L2 bi-directional flow metrics. > > > > I believe that adoption of a uni-directional flow model excludes > > basically the entire population of legacy networking devices that > > could easily report their existing bi-directional flow > metrics using > > IPFIX. > > > > Wouldn't this violate your fundamental constraint? > > Not at all. No one is trying to forbid existing technologies. > But this is about IPFIX requirements. I said I am not sure > whether we should specify uni- or bi-directional in the > requirements at all, but I am sure I do not want to have > "expensive" requirements. If you explain to me how to make > bi-directional flows cheap, I will find it easier considering > them as a potential requirement. > > Juergen > -- > Juergen Quittek quittek@ccrle.nec.de Tel: +49 6221 90511-15 > NEC Europe Ltd., Network Laboratories Fax: +49 6221 90511-55 > Adenauerplatz 6, 69115 Heidelberg, Germany http://www.ccrle.nec.de > -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Tue Jan 8 11:58:22 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA29721 for ; Tue, 8 Jan 2002 11:58:21 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16NzJG-0001tZ-00 for ipfix-list@mil.doit.wisc.edu; Tue, 08 Jan 2002 10:41:10 -0600 Received: from relay1.pair.com ([209.68.1.20] helo=relay.pair.com) by mil.doit.wisc.edu with smtp (Exim 3.13 #1) id 16NzJD-0001tG-00 for ipfix-req@net.doit.wisc.edu; Tue, 08 Jan 2002 10:41:07 -0600 Received: (qmail 33570 invoked from network); 8 Jan 2002 16:41:05 -0000 Received: from pool-162-83-252-213.ny5030.east.verizon.net (HELO sphynx) (162.83.252.213) by relay1.pair.com with SMTP; 8 Jan 2002 16:41:05 -0000 X-pair-Authenticated: 162.83.252.213 Reply-To: From: "Carter Bullard" To: , Subject: RE: [ipfix-req] unidirection vs. bidirectional flows Date: Tue, 8 Jan 2002 11:40:53 -0500 Organization: QoSient, LLC Message-ID: <5C8959A16A71B449AE793CF52FBBED66018FD3@ptah.newyork.qosient.com> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook, Build 10.0.3416 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 In-Reply-To: <3C3B1E6E.1DF1350C@riverstonenet.com> Importance: Normal Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Hey Paul, So I have a flow monitor that generates RTT times for DNS transactions. How do I report RTT for a DNS transaction using IPFIX if it has the restriction that I must report my metrics using a uni-directional semantic? So how do I do it? Break the records in two, and somehow tag that the two records are unambiguously related, and where do I put the bi-directional semantic? In both of the records, in the first one, the last one, in a separate third record? Doesn't seem very efficient. Carter Carter Bullard QoSient, LLC 300 E. 56th Street, Suite 18K New York, New York 10022 carter@qosient.com Phone +1 212 588-9133 Fax +1 212 588-9134 http://qosient.com > -----Original Message----- > From: majordomo listserver > [mailto:majordomo@mil.doit.wisc.edu] On Behalf Of > calato@riverstonenet.com > Sent: Tuesday, January 08, 2002 11:30 AM > To: ipfix-req@net.doit.wisc.edu > Subject: Re: [ipfix-req] unidirection vs. bidirectional flows > > > > Carter, > > I'm a bit confused. Why doesn't the ability to provide bi-directional > info, as Dave described, meet your needs? Those devices that > can do bi-directional will report it that way, those that can't > will report it as uni-directional. Some down stream application > will have to do the work to turn 2 uni-directional flows into > one bi-directional one, if that is what's needed. And if > uni-directional > is needed the app will split it apart. > > But if you require bi-directional reporting, many devices wont > be able to do it. Which then eliminates any down stream analysis. > > Paul > > > Carter Bullard wrote: > > > > Hey Vamsi, > > Hmmm, is the interface a part of the flow defintion? > > Not a problem. Lets look at how this may work. Suppose there > > is a SRC interface and a DST interface, so that a flow is > > simply all the packets moving from SRC -> DST. The return flow > > would be all the packets moving from DST -> SRC. > > > > So the flow is reported as SRC <-> DST, with the simple > > addition of the DST -> SRC metrics. Makes more sense than > > independantly reporting two half-duplex flow records, which > > could be skewed in time so badly, that the two records cannot > > be correlated at all. > > > > Carter > > > > Carter Bullard > > QoSient, LLC > > 300 E. 56th Street, Suite 18K > > New York, New York 10022 > > > > carter@qosient.com > > Phone +1 212 588-9133 > > Fax +1 212 588-9134 > > http://qosient.com > > > > > -----Original Message----- > > > From: Vamsidhar Valluri [mailto:vvalluri@cisco.com] > > > Sent: Monday, January 07, 2002 6:58 PM > > > To: Carter Bullard > > > Cc: plonka@doit.wisc.edu; ipfix-req@net.doit.wisc.edu > > > Subject: RE: [ipfix-req] unidirection vs. bidirectional flows > > > > > > > > > > > > > > > On Mon, 7 Jan 2002, Carter Bullard wrote: > > > > > > > Gentle people, > > > > There is absolutely no reason for IPFIX to limit the type > > > > of flow data that it supports. The difference between > > > uni-directional > > > > and bi-directional flow records is simply the type of > > > metrics that are > > > > reported. The flow descriptors and keys are all the > same. Because > > > > IPFIX must support > > > ^^^^^ > > > > > > If "keys" you are refering to are the fields used to identify > > > a flow then "SRC interface" do not fit into the statement > > > "The flow descriptors and keys are all same for both > > > unidirectional and bi-directional flow records" mentioned > > > above because SRC interface will make it unidirectional. > > > > > > -vamsi > > > > > > > > > >reporting metrics that each vendors will not implement, such > > > as jitter > > > >and one-way delay, which are the few metrics that are > meaningful in > > > >uni-directional flow data models, supporting metrics > that report on > > > >return traffic should not be a hardship for IPFIX. > > > > > > > > But, I believe that if you are going to make a design > > > choice it should > > > > be motivated by design goals. What benefit do you gain > by limiting > > > > IPFIX to a simple uni-directional > > > > flow model? Is a uni-directional flow data model the best > > > > data model for reporting all the metrics and network > behaviors that > > > > relate to network flows? Is it the best data model for > > > supporting all > > > > the existing commercial network flow data? > > > > > > > > But what about even the simplest of applications of > flow data? Is a > > > > uni-directional flow data model the best solution for > reporting the > > > > RTT of a ping volley? The empirical bulk transfer > > > properties of a TCP > > > > connection? How about TCP packet retransmission? Existence > > > of routing > > > > failures in an OSPF routed network? Application response time? > > > > Rerouting indications? Reachability fault detection? > > > Access policy > > > > validation? Connectivity status? > > > > > > > > Even FTP supports both ASCII and binary data transfer. > > > Handling both > > > > uni-directional data and bi-directional data in IPFIX > > > should be pretty > > > > easy stuff. > > > > > > > > > > > > Carter > > > > > > > > Carter Bullard > > > > QoSient, LLC > > > > 300 E. 56th Street, Suite 18K > > > > New York, New York 10022 > > > > > > > > carter@qosient.com > > > > Phone +1 212 588-9133 > > > > Fax +1 212 588-9134 > > > > http://qosient.com > > > > > > > > > > > > > > > > > > > > > -----Original Message----- > > > > > From: majordomo listserver > > > [mailto:majordomo@mil.doit.wisc.edu] On > > > > > Behalf Of Dave Plonka > > > > > Sent: Monday, January 07, 2002 4:52 PM > > > > > To: ipfix-req@net.doit.wisc.edu > > > > > Subject: [ipfix-req] unidirection vs. bidirectional flows > > > > > > > > > > > > > > > > > > > > IPFIX folks, > > > > > > > > > > As many of you recall, whether or not IPFIX should specify a > > > > > unidirectional or bidirectional flow definition was a hot > > > topic at > > > > > the WG meeting. > > > > > > > > > > I think that specifying *something* gets us far along in > > > > > communicating what a "flow" is. Personally, I would > like us to > > > > > specify one or the other and I am in favor a unidirection flow > > > > > definition. > > > > > > > > > > Here are some thoughts about arguments made for > > > bidirectional flows: > > > > > > > > > > A unidirectional flow definition means that the atomic > > > flow report > > > > > record is unidirection during the reporting/export phase. For > > > > > instance, a single packet counter for a given > > > > > protocol/srcaddr/srcport/dstaddr/dstport n-tuple. > However, since > > > > > the exporter is somewhat of a "black box", it need not be > > > > > constrained to a unidirectional model internally. > > > > > > > > > > A unidirectional flow definition does not prevent a collector > > > > > implementor from using a bi-directional notion of flows > > > internally > > > > > nor does it prevent it from being stored in a > bidirectional way > > > > > prior to export. Conversely if we specify a bidirectional > > > > > definition, it does disqualify unidirectional implementations. > > > > > > > > > > For those that wish to perform measurements which may require > > > > > bidirectional flow information (perhaps because start/end > > > timestamp > > > > > synchronization is required for flow pairs), the > IPFIX data model > > > > > and transport could define a mechanism by which the > > > exporter could > > > > > inform the collector about which unidirectional flow > records are > > > > > matched pairs. That is, the exporter could help the > > > collector find > > > > > the matched pairs of flow information, perhaps by > > > exporting the flow > > > > > records in pairs or by generating a numeric > identifier that is an > > > > > attribute of both flow records. Of course only bidirectional > > > > > collectors would understand this convention, but a > > > > > unidirection collector could ignore it and still make use of > > > > > the exported flow information from an exporter that > > > > > implemented this "bidirectional flow pairing" feature. > > > > > > > > > > Dave > > > > > > > > > > -- > > > > > ":) You will make many changes before settling > satisfactorily. :)" > > > > > - the actual message (complete with smiley faces) that I > > > received > > > > > in a > > > > > fortune cookie at dinner after the IPFIX WG meeting > > > during IETF 52 > > > > > in Salt Lake City > > > > > > > > > > -- > > > > > Help mailto:majordomo@net.doit.wisc.edu and say "help" > > > > > in message body > > > > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > > "unsubscribe > > > > > ipfix" in message body > > > > > Archive http://ipfix.doit.wisc.edu/archive/ > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > Help mailto:majordomo@net.doit.wisc.edu and say > > > "help" in message body > > > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe > > > > ipfix" in message body > > > > Archive http://ipfix.doit.wisc.edu/archive/ > > > > > > > > > > > > > > > > > -- > > Help mailto:majordomo@net.doit.wisc.edu and say > "help" in message body > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > "unsubscribe ipfix" in message body > > Archive http://ipfix.doit.wisc.edu/archive/ > > -- > Help mailto:majordomo@net.doit.wisc.edu and say "help" > in message body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ > > -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Tue Jan 8 12:02:26 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA29868 for ; Tue, 8 Jan 2002 12:02:26 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16NzKZ-0001w9-00 for ipfix-list@mil.doit.wisc.edu; Tue, 08 Jan 2002 10:42:31 -0600 Received: from host3.riverstonenet.com ([63.113.148.3] helo=exc-sc1.yagosys.com) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 16NzKX-0001uP-00 for ipfix-req@net.doit.wisc.edu; Tue, 08 Jan 2002 10:42:29 -0600 Received: from riverstonenet.com (134.141.180.100 [134.141.180.100]) by exc-sc1.yagosys.com with SMTP (Microsoft Exchange Internet Mail Service Version 5.5.2653.13) id X7WVYKYC; Tue, 8 Jan 2002 08:41:29 -0800 Message-ID: <3C3B213E.DC622E62@riverstonenet.com> Date: Tue, 08 Jan 2002 11:41:34 -0500 From: calato@riverstonenet.com X-Mailer: Mozilla 4.75C-CCK-MCD NSCPCD475 [en] (X11; U; SunOS 5.6 sun4u) X-Accept-Language: en MIME-Version: 1.0 To: "ipfix-req@net.doit.wisc.edu" Subject: Re: [ipfix-req] unidirection vs. bidirectional flows References: <1182459298.1010499773@[192.168.102.164]> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Juergen Quittek wrote: > > --On 08 January 2002 07:34 -0500 Carter Bullard wrote: > > > Hey Juergen, > > Just a note, the majority of switches and routers > > today (> 90%) cannot support uni-directional flow record > > What routers are you talking about? Isn't NetFlow available > for most CisCo and Juniper boxes? > > > reporting. However, virtually all switches and routers > > (> 99%) support RMON-style interface statistics, which > > are L1/L2 bi-directional flow metrics. > > > > I believe that adoption of a uni-directional flow model > > excludes basically the entire population of legacy > > networking devices that could easily report their existing > > bi-directional flow metrics using IPFIX. Why wouldn't they report them as 2 uni-directionals? > > > > Wouldn't this violate your fundamental constraint? > > Not at all. No one is trying to forbid existing technologies. > But this is about IPFIX requirements. I said I am not sure > whether we should specify uni- or bi-directional in the > requirements at all, but I am sure I do not want to have > "expensive" requirements. If you explain to me how to make > bi-directional flows cheap, I will find it easier considering > them as a potential requirement. > > Juergen > -- > Juergen Quittek quittek@ccrle.nec.de Tel: +49 6221 90511-15 > NEC Europe Ltd., Network Laboratories Fax: +49 6221 90511-55 > Adenauerplatz 6, 69115 Heidelberg, Germany http://www.ccrle.nec.de > > > > > Carter > > > > Carter Bullard > > QoSient, LLC > > 300 E. 56th Street, Suite 18K > > New York, New York 10022 > > > > carter@qosient.com > > Phone +1 212 588-9133 > > Fax +1 212 588-9134 > > http://qosient.com > > > > > >> -----Original Message----- > >> From: majordomo listserver > >> [mailto:majordomo@mil.doit.wisc.edu] On Behalf Of Juergen Quittek > >> Sent: Tuesday, January 08, 2002 6:24 AM > >> To: carter@qosient.com; 'Vamsidhar Valluri' > >> Cc: plonka@doit.wisc.edu; ipfix-req@net.doit.wisc.edu > >> Subject: RE: [ipfix-req] unidirection vs. bidirectional flows > >> > >> > >> Hi Carter, > >> > >> One of the very important constraints we applied to each > >> requirements so far, is that it must be possible to for > >> routers as well as for probes to implement the respective > >> requirement with reasonable effort. > >> > >> (If anyone finds out that one or more of the requirements > >> appears to be expensive, then please speak up!) > >> > >> Facing the hardware acrchitecture of a modern router, > >> I consider a bi-directional flow model to be too expensive > >> to be part of the requirements (and consequently also > >> too expensive to be part of the general architecture on > >> the exporter side). > >> > >> The reason is that at line interface cards you typically > >> spent very different effort on treatment on incoming and > >> outgoing packets. For incoming packets you do a lot of > >> processing: checksums, TTL processing, routing based on > >> cached routing tables, ... Here you touch header fields > >> anyway and it is relatively easy and relatively cheap to > >> add metering functions. For outgoing packets the treatment > >> is different. You just try to get rid of them as fast as > >> possible (except from scheduling thme first). Here it would > >> be rather expensive to investigate header fields again and > >> it might also reduce your maximum thoughput. Therefore a > >> device just metering incoming packets should be able to > >> match the requirements. > >> > >> Now, if we have a bi-directional flow model, you require > >> such a device to merge all its input measurements performed > >> at different line cards by looking for measured uni-directional > >> flows belongig to the same bi-directinal one. This task > >> might be done at a control processor, but it creates > >> additional cost and it might slow down the system. I believe > >> that merging of these measurements is a perfect job for an > >> analysis application and not for a meter. > >> > >> Definitely it is nothing we should ask every meter to perform > >> by making it a MUST requirement! > >> > >> Consequently, my position is that IF we add a flow model to > >> the requirements, then it should be the uni-diractional one. > >> However, I am not yet convinced, that we should specify the > >> model at all in the requirements. > >> > >> Best wishes, > >> > >> Juergen > >> -- > >> Juergen Quittek quittek@ccrle.nec.de Tel: +49 6221 90511-15 > >> NEC Europe Ltd., Network Laboratories Fax: +49 6221 90511-55 > >> Adenauerplatz 6, 69115 Heidelberg, Germany http://www.ccrle.nec.de > >> > >> > >> --On 07 January 2002 19:42 -0500 Carter Bullard > >> wrote: > >> > >> > Hey Vamsi, > >> > Hmmm, is the interface a part of the flow defintion? > >> > Not a problem. Lets look at how this may work. Suppose there > >> > is a SRC interface and a DST interface, so that a flow is > >> > simply all the packets moving from SRC -> DST. The return flow > >> > would be all the packets moving from DST -> SRC. > >> > > >> > So the flow is reported as SRC <-> DST, with the simple > >> > addition of the DST -> SRC metrics. Makes more sense than > >> > independantly reporting two half-duplex flow records, which > >> > could be skewed in time so badly, that the two records cannot > >> > be correlated at all. > >> > > >> > > >> > Carter > >> > > >> > Carter Bullard > >> > QoSient, LLC > >> > 300 E. 56th Street, Suite 18K > >> > New York, New York 10022 > >> > > >> > carter@qosient.com > >> > Phone +1 212 588-9133 > >> > Fax +1 212 588-9134 > >> > http://qosient.com > >> > > >> > > >> >> -----Original Message----- > >> >> From: Vamsidhar Valluri [mailto:vvalluri@cisco.com] > >> >> Sent: Monday, January 07, 2002 6:58 PM > >> >> To: Carter Bullard > >> >> Cc: plonka@doit.wisc.edu; ipfix-req@net.doit.wisc.edu > >> >> Subject: RE: [ipfix-req] unidirection vs. bidirectional flows > >> >> > >> >> > >> >> > >> >> > >> >> On Mon, 7 Jan 2002, Carter Bullard wrote: > >> >> > >> >> > Gentle people, > >> >> > There is absolutely no reason for IPFIX to limit the type > >> >> > of flow data that it supports. The difference between > >> >> uni-directional > >> >> > and bi-directional flow records is simply the type of > >> >> metrics that are > >> >> > reported. The flow descriptors and keys are all the > >> same. Because > >> >> > IPFIX must support > >> >> ^^^^^ > >> >> > >> >> If "keys" you are refering to are the fields used to identify > >> >> a flow then "SRC interface" do not fit into the statement > >> >> "The flow descriptors and keys are all same for both > >> >> unidirectional and bi-directional flow records" mentioned > >> >> above because SRC interface will make it unidirectional. > >> >> > >> >> -vamsi > >> >> > >> >> > >> >> > reporting metrics that each vendors will not implement, such > >> >> as jitter > >> >> > and one-way delay, which are the few metrics that are > >> meaningful in > >> >> > uni-directional flow data models, supporting metrics > >> that report on > >> >> > return traffic should not be a hardship for IPFIX. > >> >> > > >> >> > But, I believe that if you are going to make a design > >> >> choice it should > >> >> > be motivated by design goals. What benefit do you gain > >> by limiting > >> >> > IPFIX to a simple uni-directional > >> >> > flow model? Is a uni-directional flow data model the best > >> >> > data model for reporting all the metrics and network > >> behaviors that > >> >> > relate to network flows? Is it the best data model for > >> >> supporting all > >> >> > the existing commercial network flow data? > >> >> > > >> >> > But what about even the simplest of applications of flow > >> data? Is a > >> >> > uni-directional flow data model the best solution for > >> reporting the > >> >> > RTT of a ping volley? The empirical bulk transfer > >> >> properties of a TCP > >> >> > connection? How about TCP packet retransmission? Existence > >> >> of routing > >> >> > failures in an OSPF routed network? Application response time? > >> >> > Rerouting indications? Reachability fault detection? > >> >> Access policy > >> >> > validation? Connectivity status? > >> >> > > >> >> > Even FTP supports both ASCII and binary data transfer. > >> >> Handling both > >> >> > uni-directional data and bi-directional data in IPFIX > >> >> should be pretty > >> >> > easy stuff. > >> >> > > >> >> > > >> >> > Carter > >> >> > > >> >> > Carter Bullard > >> >> > QoSient, LLC > >> >> > 300 E. 56th Street, Suite 18K > >> >> > New York, New York 10022 > >> >> > > >> >> > carter@qosient.com > >> >> > Phone +1 212 588-9133 > >> >> > Fax +1 212 588-9134 > >> >> > http://qosient.com > >> >> > > >> >> > > >> >> > > >> >> > > >> >> > > -----Original Message----- > >> >> > > From: majordomo listserver > >> >> [mailto:majordomo@mil.doit.wisc.edu] On > >> >> > > Behalf Of Dave Plonka > >> >> > > Sent: Monday, January 07, 2002 4:52 PM > >> >> > > To: ipfix-req@net.doit.wisc.edu > >> >> > > Subject: [ipfix-req] unidirection vs. bidirectional flows > >> >> > > > >> >> > > > >> >> > > > >> >> > > IPFIX folks, > >> >> > > > >> >> > > As many of you recall, whether or not IPFIX should specify a > >> >> > > unidirectional or bidirectional flow definition was a hot > >> >> topic at > >> >> > > the WG meeting. > >> >> > > > >> >> > > I think that specifying *something* gets us far along in > >> >> > > communicating what a "flow" is. Personally, I would like us to > >> >> > > specify one or the other and I am in favor a unidirection flow > >> >> > > definition. > >> >> > > > >> >> > > Here are some thoughts about arguments made for > >> >> bidirectional flows: > >> >> > > > >> >> > > A unidirectional flow definition means that the atomic > >> >> flow report > >> >> > > record is unidirection during the reporting/export phase. For > >> >> > > instance, a single packet counter for a given > >> >> > > protocol/srcaddr/srcport/dstaddr/dstport n-tuple. > >> However, since > >> >> > > the exporter is somewhat of a "black box", it need not be > >> >> > > constrained to a unidirectional model internally. > >> >> > > > >> >> > > A unidirectional flow definition does not prevent a collector > >> >> > > implementor from using a bi-directional notion of flows > >> >> internally > >> >> > > nor does it prevent it from being stored in a bidirectional way > >> >> > > prior to export. Conversely if we specify a bidirectional > >> >> > > definition, it does disqualify unidirectional implementations. > >> >> > > > >> >> > > For those that wish to perform measurements which may require > >> >> > > bidirectional flow information (perhaps because start/end > >> >> timestamp > >> >> > > synchronization is required for flow pairs), the IPFIX > >> data model > >> >> > > and transport could define a mechanism by which the > >> >> exporter could > >> >> > > inform the collector about which unidirectional flow > >> records are > >> >> > > matched pairs. That is, the exporter could help the > >> >> collector find > >> >> > > the matched pairs of flow information, perhaps by > >> >> exporting the flow > >> >> > > records in pairs or by generating a numeric identifier > >> that is an > >> >> > > attribute of both flow records. Of course only bidirectional > >> >> > > collectors would understand this convention, but a > >> >> > > unidirection collector could ignore it and still make use of > >> >> > > the exported flow information from an exporter that > >> >> > > implemented this "bidirectional flow pairing" feature. > >> >> > > > >> >> > > Dave > >> >> > > > >> >> > > -- > >> >> > > ":) You will make many changes before settling > >> satisfactorily. :)" > >> >> > > - the actual message (complete with smiley faces) that I > >> >> received > >> >> > > in a > >> >> > > fortune cookie at dinner after the IPFIX WG meeting > >> >> during IETF 52 > >> >> > > in Salt Lake City > >> >> > > > >> >> > > -- > >> >> > > Help mailto:majordomo@net.doit.wisc.edu and say "help" > >> >> > > in message body > >> >> > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > >> >> "unsubscribe > >> >> > > ipfix" in message body > >> >> > > Archive http://ipfix.doit.wisc.edu/archive/ > >> >> > > > >> >> > > > >> >> > > >> >> > > >> >> > > >> >> > -- > >> >> > Help mailto:majordomo@net.doit.wisc.edu and say > >> >> "help" in message body > >> >> > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > >> "unsubscribe > >> >> > ipfix" in message body > >> >> > Archive http://ipfix.doit.wisc.edu/archive/ > >> >> > > >> >> > >> >> > >> >> > >> > > >> > > >> > > >> > -- > >> > Help mailto:majordomo@net.doit.wisc.edu and say > >> "help" in message body > >> > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > >> > "unsubscribe ipfix" in message body > >> > Archive http://ipfix.doit.wisc.edu/archive/ > >> > > >> > >> > >> > >> -- > >> Help mailto:majordomo@net.doit.wisc.edu and say "help" > >> in message body > >> Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > >> "unsubscribe ipfix" in message body > >> Archive http://ipfix.doit.wisc.edu/archive/ > >> > >> > > > > > > > > -- > > Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > "unsubscribe ipfix" in message body > > Archive http://ipfix.doit.wisc.edu/archive/ > > > > -- > Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Tue Jan 8 12:10:46 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA00275 for ; Tue, 8 Jan 2002 12:10:46 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16NzTo-0002Ac-00 for ipfix-list@mil.doit.wisc.edu; Tue, 08 Jan 2002 10:52:04 -0600 Received: from host3.riverstonenet.com ([63.113.148.3] helo=exc-sc1.yagosys.com) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 16NzTn-00029Q-00 for ipfix-req@net.doit.wisc.edu; Tue, 08 Jan 2002 10:52:03 -0600 Received: from riverstonenet.com (134.141.180.100 [134.141.180.100]) by exc-sc1.yagosys.com with SMTP (Microsoft Exchange Internet Mail Service Version 5.5.2653.13) id X7WVYK5J; Tue, 8 Jan 2002 08:51:03 -0800 Message-ID: <3C3B237C.2F030A14@riverstonenet.com> Date: Tue, 08 Jan 2002 11:51:08 -0500 From: calato@riverstonenet.com X-Mailer: Mozilla 4.75C-CCK-MCD NSCPCD475 [en] (X11; U; SunOS 5.6 sun4u) X-Accept-Language: en MIME-Version: 1.0 To: carter@qosient.com CC: "ipfix-req@net.doit.wisc.edu" Subject: Re: [ipfix-req] unidirection vs. bidirectional flows References: <5C8959A16A71B449AE793CF52FBBED66018FD3@ptah.newyork.qosient.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit I'm not sure. Can you tell me the fields reported today? I'll see if I can make sense of it as 2 uni-directional flows. Paul Carter Bullard wrote: > > Hey Paul, > So I have a flow monitor that generates RTT times for DNS > transactions. How do I report RTT for a DNS transaction using > IPFIX if it has the restriction that I must report my metrics > using a uni-directional semantic? > > So how do I do it? Break the records in two, and somehow > tag that the two records are unambiguously related, and > where do I put the bi-directional semantic? In both of > the records, in the first one, the last one, in a separate > third record? > > Doesn't seem very efficient. > > Carter > > Carter Bullard > QoSient, LLC > 300 E. 56th Street, Suite 18K > New York, New York 10022 > > carter@qosient.com > Phone +1 212 588-9133 > Fax +1 212 588-9134 > http://qosient.com > > > -----Original Message----- > > From: majordomo listserver > > [mailto:majordomo@mil.doit.wisc.edu] On Behalf Of > > calato@riverstonenet.com > > Sent: Tuesday, January 08, 2002 11:30 AM > > To: ipfix-req@net.doit.wisc.edu > > Subject: Re: [ipfix-req] unidirection vs. bidirectional flows > > > > > > > > Carter, > > > > I'm a bit confused. Why doesn't the ability to provide bi-directional > > info, as Dave described, meet your needs? Those devices that > > can do bi-directional will report it that way, those that can't > > will report it as uni-directional. Some down stream application > > will have to do the work to turn 2 uni-directional flows into > > one bi-directional one, if that is what's needed. And if > > uni-directional > > is needed the app will split it apart. > > > > But if you require bi-directional reporting, many devices wont > > be able to do it. Which then eliminates any down stream analysis. > > > > Paul > > > > > > Carter Bullard wrote: > > > > > > Hey Vamsi, > > > Hmmm, is the interface a part of the flow defintion? > > > Not a problem. Lets look at how this may work. Suppose there > > > is a SRC interface and a DST interface, so that a flow is > > > simply all the packets moving from SRC -> DST. The return flow > > > would be all the packets moving from DST -> SRC. > > > > > > So the flow is reported as SRC <-> DST, with the simple > > > addition of the DST -> SRC metrics. Makes more sense than > > > independantly reporting two half-duplex flow records, which > > > could be skewed in time so badly, that the two records cannot > > > be correlated at all. > > > > > > Carter > > > > > > Carter Bullard > > > QoSient, LLC > > > 300 E. 56th Street, Suite 18K > > > New York, New York 10022 > > > > > > carter@qosient.com > > > Phone +1 212 588-9133 > > > Fax +1 212 588-9134 > > > http://qosient.com > > > > > > > -----Original Message----- > > > > From: Vamsidhar Valluri [mailto:vvalluri@cisco.com] > > > > Sent: Monday, January 07, 2002 6:58 PM > > > > To: Carter Bullard > > > > Cc: plonka@doit.wisc.edu; ipfix-req@net.doit.wisc.edu > > > > Subject: RE: [ipfix-req] unidirection vs. bidirectional flows > > > > > > > > > > > > > > > > > > > > On Mon, 7 Jan 2002, Carter Bullard wrote: > > > > > > > > > Gentle people, > > > > > There is absolutely no reason for IPFIX to limit the type > > > > > of flow data that it supports. The difference between > > > > uni-directional > > > > > and bi-directional flow records is simply the type of > > > > metrics that are > > > > > reported. The flow descriptors and keys are all the > > same. Because > > > > > IPFIX must support > > > > ^^^^^ > > > > > > > > If "keys" you are refering to are the fields used to identify > > > > a flow then "SRC interface" do not fit into the statement > > > > "The flow descriptors and keys are all same for both > > > > unidirectional and bi-directional flow records" mentioned > > > > above because SRC interface will make it unidirectional. > > > > > > > > -vamsi > > > > > > > > > > > > >reporting metrics that each vendors will not implement, such > > > > as jitter > > > > >and one-way delay, which are the few metrics that are > > meaningful in > > > > >uni-directional flow data models, supporting metrics > > that report on > > > > >return traffic should not be a hardship for IPFIX. > > > > > > > > > > But, I believe that if you are going to make a design > > > > choice it should > > > > > be motivated by design goals. What benefit do you gain > > by limiting > > > > > IPFIX to a simple uni-directional > > > > > flow model? Is a uni-directional flow data model the best > > > > > data model for reporting all the metrics and network > > behaviors that > > > > > relate to network flows? Is it the best data model for > > > > supporting all > > > > > the existing commercial network flow data? > > > > > > > > > > But what about even the simplest of applications of > > flow data? Is a > > > > > uni-directional flow data model the best solution for > > reporting the > > > > > RTT of a ping volley? The empirical bulk transfer > > > > properties of a TCP > > > > > connection? How about TCP packet retransmission? Existence > > > > of routing > > > > > failures in an OSPF routed network? Application response time? > > > > > Rerouting indications? Reachability fault detection? > > > > Access policy > > > > > validation? Connectivity status? > > > > > > > > > > Even FTP supports both ASCII and binary data transfer. > > > > Handling both > > > > > uni-directional data and bi-directional data in IPFIX > > > > should be pretty > > > > > easy stuff. > > > > > > > > > > > > > > > Carter > > > > > > > > > > Carter Bullard > > > > > QoSient, LLC > > > > > 300 E. 56th Street, Suite 18K > > > > > New York, New York 10022 > > > > > > > > > > carter@qosient.com > > > > > Phone +1 212 588-9133 > > > > > Fax +1 212 588-9134 > > > > > http://qosient.com > > > > > > > > > > > > > > > > > > > > > > > > > > -----Original Message----- > > > > > > From: majordomo listserver > > > > [mailto:majordomo@mil.doit.wisc.edu] On > > > > > > Behalf Of Dave Plonka > > > > > > Sent: Monday, January 07, 2002 4:52 PM > > > > > > To: ipfix-req@net.doit.wisc.edu > > > > > > Subject: [ipfix-req] unidirection vs. bidirectional flows > > > > > > > > > > > > > > > > > > > > > > > > IPFIX folks, > > > > > > > > > > > > As many of you recall, whether or not IPFIX should specify a > > > > > > unidirectional or bidirectional flow definition was a hot > > > > topic at > > > > > > the WG meeting. > > > > > > > > > > > > I think that specifying *something* gets us far along in > > > > > > communicating what a "flow" is. Personally, I would > > like us to > > > > > > specify one or the other and I am in favor a unidirection flow > > > > > > definition. > > > > > > > > > > > > Here are some thoughts about arguments made for > > > > bidirectional flows: > > > > > > > > > > > > A unidirectional flow definition means that the atomic > > > > flow report > > > > > > record is unidirection during the reporting/export phase. For > > > > > > instance, a single packet counter for a given > > > > > > protocol/srcaddr/srcport/dstaddr/dstport n-tuple. > > However, since > > > > > > the exporter is somewhat of a "black box", it need not be > > > > > > constrained to a unidirectional model internally. > > > > > > > > > > > > A unidirectional flow definition does not prevent a collector > > > > > > implementor from using a bi-directional notion of flows > > > > internally > > > > > > nor does it prevent it from being stored in a > > bidirectional way > > > > > > prior to export. Conversely if we specify a bidirectional > > > > > > definition, it does disqualify unidirectional implementations. > > > > > > > > > > > > For those that wish to perform measurements which may require > > > > > > bidirectional flow information (perhaps because start/end > > > > timestamp > > > > > > synchronization is required for flow pairs), the > > IPFIX data model > > > > > > and transport could define a mechanism by which the > > > > exporter could > > > > > > inform the collector about which unidirectional flow > > records are > > > > > > matched pairs. That is, the exporter could help the > > > > collector find > > > > > > the matched pairs of flow information, perhaps by > > > > exporting the flow > > > > > > records in pairs or by generating a numeric > > identifier that is an > > > > > > attribute of both flow records. Of course only bidirectional > > > > > > collectors would understand this convention, but a > > > > > > unidirection collector could ignore it and still make use of > > > > > > the exported flow information from an exporter that > > > > > > implemented this "bidirectional flow pairing" feature. > > > > > > > > > > > > Dave > > > > > > > > > > > > -- > > > > > > ":) You will make many changes before settling > > satisfactorily. :)" > > > > > > - the actual message (complete with smiley faces) that I > > > > received > > > > > > in a > > > > > > fortune cookie at dinner after the IPFIX WG meeting > > > > during IETF 52 > > > > > > in Salt Lake City > > > > > > > > > > > > -- > > > > > > Help mailto:majordomo@net.doit.wisc.edu and say "help" > > > > > > in message body > > > > > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > > > "unsubscribe > > > > > > ipfix" in message body > > > > > > Archive http://ipfix.doit.wisc.edu/archive/ > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > Help mailto:majordomo@net.doit.wisc.edu and say > > > > "help" in message body > > > > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > "unsubscribe > > > > > ipfix" in message body > > > > > Archive http://ipfix.doit.wisc.edu/archive/ > > > > > > > > > > > > > > > > > > > > > > > -- > > > Help mailto:majordomo@net.doit.wisc.edu and say > > "help" in message body > > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > > "unsubscribe ipfix" in message body > > > Archive http://ipfix.doit.wisc.edu/archive/ > > > > -- > > Help mailto:majordomo@net.doit.wisc.edu and say "help" > > in message body > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > "unsubscribe ipfix" in message body > > Archive http://ipfix.doit.wisc.edu/archive/ > > > > -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Tue Jan 8 12:18:41 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA00713 for ; Tue, 8 Jan 2002 12:18:40 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16NzdS-0002QA-00 for ipfix-list@mil.doit.wisc.edu; Tue, 08 Jan 2002 11:02:02 -0600 Received: from dplonka by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16NzdP-0002Ph-00 for ipfix-req@net.doit.wisc.edu; Tue, 08 Jan 2002 11:01:59 -0600 Date: Tue, 8 Jan 2002 11:01:58 -0600 From: Dave Plonka To: ipfix-req@net.doit.wisc.edu Subject: Re: [ipfix-req] unidirection vs. bidirectional flows Message-ID: <20020108110158.A8654@doit.wisc.edu> Reply-To: ipfix-req@net.doit.wisc.edu References: <200201081508.g08F8FI11407@bru-cse-222.cisco.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 1.0.1i In-Reply-To: <200201081508.g08F8FI11407@bru-cse-222.cisco.com>; from bclaise@cisco.com on Tue, Jan 08, 2002 at 04:08:15PM +0100 Organization: UW-Madison, DoIT, Network Services X-VMS-Error: %SYSTEM-W-OVRMAXARG, maximum parameter count exceeded X-Shakespearean-Insult: Thou goatish spur-galled wagtail Precedence: bulk Sender: majordomo listserver Benoit made this point that I was going to follow-up with, in a slightly more general way (below): On Tue, Jan 08, 2002 at 04:08:15PM +0100, Benoit Claise wrote: > If the observation point is interface 1 of line card 1 and the output > interface interface the interface 1 of line card 2, this is extremely > difficult (if not impossible) to correlate the flow going forth and the > flow going back. Because most of the processing for a "ipfix like" > protocol will be done on the incoming line card. > > As a conclusion, I think this only solution is that a flow is unidirectional. In other words, a given observation point within an exporter may only see a portion of the traffic for a given application-level stream. The result is that, in some circumstances, it can only report the one direction. Like in Benoit's example, asymmetric routing would further exacerbate this since each direction is observed by a different router/exporter. Now - on to the more general part... There are two operations that are sometimes performed during flow analysis: "flow pasting" and "flow matching" (my terms). "Flow pasting" is the sequencing and coalescing/concatenating of flow records into larger flow records. "Flow matching" is the identifying of corresponding flow records in each direction, that are ostensibly part of the same application-level stream. Usually the purpose of "flow pasting" and "flow matching" is to discover something about the underlying application-level stream. Note that this is not necessarily useful for some IPFIX applications (Usage-based Accounting, Traffic Profiling, Traffic Engineering), but is sometimes advantageous for others (Attack/Intrusion Detection, QoS Monitoring). For those applications that benefit from flow pasting and/or matching, these operations are still necessary _even_ if a bi-directional flow definition and export record is used. It is still necessary for the collector (or subsequent post-processor) to perform pasting and matching functions because sometimes the (bidirectional) flow records will be from multiple sources. From this I conclude that using a bidirectional flow export record does not simplify the implementation of the collector and others have said that it complicates the exporter implementation. I started off this thread by saying that a unidirectional flow definition (and export record) does not preclude someone from implementing an exporter which adds additional flow record attributes that facilitate "flow pasting" and "flow matching". Since IPFIX won't specify a file-format for flow records for persistant storage, implementors would be free to create their own bidirectional records, which they are free to refer to as "Foobar(tm) IP flows" rather than "IPFIX standard IP flows". Dave -- ":) You will make many changes before settling satisfactorily. :)" - the actual message (complete with smiley faces) that I received in a fortune cookie at dinner after the IPFIX WG meeting during IETF 52 in Salt Lake City -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Tue Jan 8 12:25:00 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA01006 for ; Tue, 8 Jan 2002 12:24:59 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16NzjH-0002YV-00 for ipfix-list@mil.doit.wisc.edu; Tue, 08 Jan 2002 11:08:03 -0600 Received: from dplonka by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16NzjE-0002YF-00 for ipfix-req@net.doit.wisc.edu; Tue, 08 Jan 2002 11:08:00 -0600 Date: Tue, 8 Jan 2002 11:08:00 -0600 From: Dave Plonka To: ipfix-req@net.doit.wisc.edu Subject: Re: [ipfix-req] unidirection vs. bidirectional flows Message-ID: <20020108110800.B8654@doit.wisc.edu> Reply-To: plonka@doit.wisc.edu References: <20020107155132.B19904@doit.wisc.edu> <5C8959A16A71B449AE793CF52FBBED66018FCB@ptah.newyork.qosient.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 1.0.1i In-Reply-To: <5C8959A16A71B449AE793CF52FBBED66018FCB@ptah.newyork.qosient.com>; from carter@qosient.com on Mon, Jan 07, 2002 at 06:31:08PM -0500 Organization: UW-Madison, DoIT, Network Services X-VMS-Error: %SYSTEM-W-OVRMAXARG, maximum parameter count exceeded X-Shakespearean-Insult: Thou goatish spur-galled wagtail Precedence: bulk Sender: majordomo listserver On Mon, Jan 07, 2002 at 06:31:08PM -0500, Carter Bullard wrote: > But, I believe that if you are going to make a design choice > it should be motivated by design goals. What benefit > do you gain by limiting IPFIX to a simple uni-directional > flow model? I'll try to explain why it may be important that the "unidirection vs. bidirectional" decision be made (in the requirements) and address how it is in accordance with our goals (which is a good point that Carter made, IMO): Here's the goals that a unidirectional flow defintion helps to address: 1) Define the notion of a "standard IP flow." [...] In the "real world", flows of water or lava have a single general direction which defines them as a flow. While it may be circuitous, material traveling in another direction is not part of the same flow. By this analogy and precedent, a unidirectional flow definition will be easier to understand by readers of our documents, and prospective implementors. Personally, I want one of the benefits of IPFIX to be able to use the term "IPFIX flow" or "standard IP flow" (in conversation) and know what it is that the speaker meant. The direction of traffic Internet traffic seems to me to be one of the most fundamental things we want to know. 2) Devise data encodings that support analysis of IPv4 and IPv6 unicast and multicast flows [...] Although this is an afterthought, it's interesting that we said "support analysis". Flow pasting and matching can be considered forms of analysis. So bidirectional flows could be considered the product, rather than the requisite input, to analysis. 3) Specify the transport mapping for carrying IP flow information, one which is amenable to router and instrumentation implementors, and to deployment. If the "standarde existing practice" fits, it probably fits here best. There are a number of hardware/ASIC based implementations that understand unidirectional flows. Only devices such as end hosts, stateful firewalls, and NAT routers, have all the information necessary to report accurate bidirectional flows. While there is still the nebulous(?) option of supporting both uni- and bi-directional flow defintions in IPFIX, I'd like to pare down the work we have ahead of us by making IPFIX as simple as possible, but no simpler. Unidirectional is simpler, and it is possible as existing implementations can attest. Bidirectional is likewise possible as there are also existing implementations, but it makes more work for the exporter. That work can be done by the collectors and in some cases must be done by the collector because the traffic in each direction will have been reported by discrete exporters. Dave -- ":) You will make many changes before settling satisfactorily. :)" - the actual message (complete with smiley faces) that I received in a fortune cookie at dinner after the IPFIX WG meeting during IETF 52 in Salt Lake City -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Tue Jan 8 12:31:11 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA01225 for ; Tue, 8 Jan 2002 12:31:11 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16Nzo7-0002gL-00 for ipfix-list@mil.doit.wisc.edu; Tue, 08 Jan 2002 11:13:03 -0600 Received: from zrc2s0jx.nortelnetworks.com ([47.103.122.112]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 16Nzo4-0002fE-00 for ipfix-arch@net.doit.wisc.edu; Tue, 08 Jan 2002 11:13:01 -0600 Received: from zsc4c000.us.nortel.com (zsc4c000.us.nortel.com [47.81.138.47]) by zrc2s0jx.nortelnetworks.com (Switch-2.2.0/Switch-2.2.0) with ESMTP id g08H8h011625; Tue, 8 Jan 2002 11:08:43 -0600 (CST) Received: by zsc4c000.us.nortel.com with Internet Mail Service (5.5.2653.19) id ; Tue, 8 Jan 2002 09:06:53 -0800 Message-ID: <4F973E944951D311B6660008C7917CF0083E088C@zsc4c008.us.nortel.com> From: "Reinaldo Penno" To: "'calato@riverstonenet.com'" , "'ipfix-arch@net.doit.wisc.edu'" Subject: RE: [ipfix-arch] configuration information. Date: Tue, 8 Jan 2002 09:06:52 -0800 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01C19866.DDFFF5B0" Precedence: bulk Sender: majordomo listserver This message is in MIME format. Since your mail reader does not understand this format, some or all of this message may not be legible. ------_=_NextPart_001_01C19866.DDFFF5B0 Content-Type: text/plain; charset="iso-8859-1" Hello Calato, ...... > > > > > > Hello, > > > > > > How a collector would know that a priori? I mean, I can have such > > > general templates that it would be very difficult to account for > > > duplicate measurements (even in non real-time). > > > > > > If you have very specific templates such as srcIP, > destIP, etc, it's > > > not to hard find records in common. But you can have things like: > > > > > > src_AnyIP Dest_Yahoo.com Proto_Any and > > > src_AnyIP Dest_AS3462 Proto_HTTP. > > I assume here you mean that there are 2 counters and > 2 "flows" reported. One has (to use cisco terms) a template > of "dest-ip, byes, packets" and the other "dest-AS, proto, > byes, packets". > > If the device uses a multi-match method then the device > "knows" that a packet may be added to both buckets. It > should tell the collector that these templates overlap. I agree. But it should say that not only these templates overlap but what 5-tuple? is overlapping. (also see below) > If on the other hand the device uses a first match method > then the counts do not overlap. Should we assume that you can have different templates per Card/Interface if the cards/interfaces have intelligence? In this case they could overlap. thanks, Reinaldo ------_=_NextPart_001_01C19866.DDFFF5B0 Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable RE: [ipfix-arch] configuration information.

Hello Calato,

<snip>...<snip>...

> > >
> > > Hello,
> > >
> > > How a collector would know that a = priori? I mean, I can have such
> > > general templates that it would be = very difficult to account for
> > > duplicate measurements (even in non = real-time).
> > >
> > > If you have very specific templates = such as srcIP,
> destIP, etc, it's
> > > not to hard find records in common. = But you can have things like:
> > >
> > > src_AnyIP   = Dest_Yahoo.com    Proto_Any   and
> > > src_AnyIP   = Dest_AS3462       Proto_HTTP.
>
>       I assume here = you mean that there are 2 counters and
>       2 = "flows" reported. One has (to use cisco terms) a = template
>       of = "dest-ip, byes, packets" and the other "dest-AS, = proto,
>       byes, = packets".
>
>       If the device = uses a multi-match method then the device
>       = "knows" that a packet may be added to both buckets. It =
>       should tell the = collector that these templates overlap.

I agree. But it should say that not only these = templates overlap but what 5-tuple? is overlapping. (also see = below) 

>       If on the other = hand the device uses a first match method
>       then the counts = do not overlap.


Should we assume that you can have different = templates per Card/Interface if the cards/interfaces have intelligence? = In this case they could overlap.

thanks,

Reinaldo

------_=_NextPart_001_01C19866.DDFFF5B0-- -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Tue Jan 8 13:03:16 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA02793 for ; Tue, 8 Jan 2002 13:03:16 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16O0IH-0003JO-00 for ipfix-list@mil.doit.wisc.edu; Tue, 08 Jan 2002 11:44:13 -0600 Received: from host3.riverstonenet.com ([63.113.148.3] helo=exc-sc1.yagosys.com) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 16O0IF-0003Ig-00 for ipfix-arch@net.doit.wisc.edu; Tue, 08 Jan 2002 11:44:12 -0600 Received: from riverstonenet.com (134.141.180.100 [134.141.180.100]) by exc-sc1.yagosys.com with SMTP (Microsoft Exchange Internet Mail Service Version 5.5.2653.13) id X7WVYLXG; Tue, 8 Jan 2002 09:43:13 -0800 Message-ID: <3C3B2FB6.3F1F0343@riverstonenet.com> Date: Tue, 08 Jan 2002 12:43:18 -0500 From: calato@riverstonenet.com X-Mailer: Mozilla 4.75C-CCK-MCD NSCPCD475 [en] (X11; U; SunOS 5.6 sun4u) X-Accept-Language: en MIME-Version: 1.0 To: Reinaldo Penno CC: arch Subject: Re: [ipfix-arch] configuration information. References: <4F973E944951D311B6660008C7917CF0083E088C@zsc4c008.us.nortel.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Reinaldo Penno wrote: > > Hello Calato, > > ...... > > > > > > > > > Hello, > > > > > > > > How a collector would know that a priori? I mean, I can have > such > > > > general templates that it would be very difficult to account for > > > > > duplicate measurements (even in non real-time). > > > > > > > > If you have very specific templates such as srcIP, > > destIP, etc, it's > > > > not to hard find records in common. But you can have things > like: > > > > > > > > src_AnyIP Dest_Yahoo.com Proto_Any and > > > > src_AnyIP Dest_AS3462 Proto_HTTP. > > > > I assume here you mean that there are 2 counters and > > 2 "flows" reported. One has (to use cisco terms) a template > > of "dest-ip, byes, packets" and the other "dest-AS, proto, > > byes, packets". > > > > If the device uses a multi-match method then the device > > "knows" that a packet may be added to both buckets. It > > should tell the collector that these templates overlap. > > I agree. But it should say that not only these templates overlap but > what 5-tuple? is overlapping. (also see below) > > > If on the other hand the device uses a first match method > > then the counts do not overlap. > > Should we assume that you can have different templates per > Card/Interface if the cards/interfaces have intelligence? In this case > they could overlap. Again, the device "knows" how it is counting. If the devices is able to track packets based on multiple keys then some thought needs to be given to whether or not those key sets overlap. For example, if one key set is "in-interface, src-ip, dst-ip" and the other is "in-interface, src-AS, dst-AS" then the associated counters do not overlap. If the application wants a total for src-AS1 then it can add the bytes from first ket set to the second. Overlaping counters are not something determined after the fact based on data seen, it is determined ahead of time for an entire class (or key set) of records. > > thanks, > > Reinaldo -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Tue Jan 8 13:51:45 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA04833 for ; Tue, 8 Jan 2002 13:51:44 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16O0yR-0004B2-00 for ipfix-list@mil.doit.wisc.edu; Tue, 08 Jan 2002 12:27:47 -0600 Received: from sj-msg-core-2.cisco.com ([171.69.24.11]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 16O0yQ-0004AN-00 for ipfix-req@net.doit.wisc.edu; Tue, 08 Jan 2002 12:27:46 -0600 Received: from postal.cisco.com (postal.cisco.com [171.71.160.26]) by sj-msg-core-2.cisco.com (8.11.3/8.9.1) with ESMTP id g08IRC401627; Tue, 8 Jan 2002 10:27:14 -0800 (PST) Received: from WILLW2K (dhcp-171-71-139-73.cisco.com [171.71.139.73]) by postal.cisco.com (Mirapoint) with SMTP id AAH66764; Tue, 8 Jan 2002 10:27:22 -0800 (PST) Reply-To: From: "Will Eatherton" To: , , Subject: RE: [ipfix-req] unidirection vs. bidirectional flows Date: Tue, 8 Jan 2002 10:26:05 -0800 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2910.0) In-Reply-To: <5C8959A16A71B449AE793CF52FBBED66018FD3@ptah.newyork.qosient.com> X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400 Importance: Normal Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Carter, Providing customized export of data of this type seems beyond the scope of first pass IPFIX goals to me (it is not flow export, strictly speaking). In the main stream router environment (i.e. cisco/juniper/etc..) this sort of functionality would be done by SW sitting behond or integrated into a collector. So this sort of summarization is one level above the stack from where I believe we want to be with IPFIX. I think this is a fundamental narrowing that we need to do in order to move forward. Will cisco > -----Original Message----- > From: majordomo listserver [mailto:majordomo@mil.doit.wisc.edu]On Behalf > Of Carter Bullard > Sent: Tuesday, January 08, 2002 8:41 AM > To: calato@riverstonenet.com; ipfix-req@net.doit.wisc.edu > Subject: RE: [ipfix-req] unidirection vs. bidirectional flows > > > Hey Paul, > So I have a flow monitor that generates RTT times for DNS > transactions. How do I report RTT for a DNS transaction using > IPFIX if it has the restriction that I must report my metrics > using a uni-directional semantic? > > So how do I do it? Break the records in two, and somehow > tag that the two records are unambiguously related, and > where do I put the bi-directional semantic? In both of > the records, in the first one, the last one, in a separate > third record? > > Doesn't seem very efficient. > > Carter > > Carter Bullard > QoSient, LLC > 300 E. 56th Street, Suite 18K > New York, New York 10022 > > carter@qosient.com > Phone +1 212 588-9133 > Fax +1 212 588-9134 > http://qosient.com > > > > > -----Original Message----- > > From: majordomo listserver > > [mailto:majordomo@mil.doit.wisc.edu] On Behalf Of > > calato@riverstonenet.com > > Sent: Tuesday, January 08, 2002 11:30 AM > > To: ipfix-req@net.doit.wisc.edu > > Subject: Re: [ipfix-req] unidirection vs. bidirectional flows > > > > > > > > Carter, > > > > I'm a bit confused. Why doesn't the ability to provide bi-directional > > info, as Dave described, meet your needs? Those devices that > > can do bi-directional will report it that way, those that can't > > will report it as uni-directional. Some down stream application > > will have to do the work to turn 2 uni-directional flows into > > one bi-directional one, if that is what's needed. And if > > uni-directional > > is needed the app will split it apart. > > > > But if you require bi-directional reporting, many devices wont > > be able to do it. Which then eliminates any down stream analysis. > > > > Paul > > > > > > Carter Bullard wrote: > > > > > > Hey Vamsi, > > > Hmmm, is the interface a part of the flow defintion? > > > Not a problem. Lets look at how this may work. Suppose there > > > is a SRC interface and a DST interface, so that a flow is > > > simply all the packets moving from SRC -> DST. The return flow > > > would be all the packets moving from DST -> SRC. > > > > > > So the flow is reported as SRC <-> DST, with the simple > > > addition of the DST -> SRC metrics. Makes more sense than > > > independantly reporting two half-duplex flow records, which > > > could be skewed in time so badly, that the two records cannot > > > be correlated at all. > > > > > > Carter > > > > > > Carter Bullard > > > QoSient, LLC > > > 300 E. 56th Street, Suite 18K > > > New York, New York 10022 > > > > > > carter@qosient.com > > > Phone +1 212 588-9133 > > > Fax +1 212 588-9134 > > > http://qosient.com > > > > > > > -----Original Message----- > > > > From: Vamsidhar Valluri [mailto:vvalluri@cisco.com] > > > > Sent: Monday, January 07, 2002 6:58 PM > > > > To: Carter Bullard > > > > Cc: plonka@doit.wisc.edu; ipfix-req@net.doit.wisc.edu > > > > Subject: RE: [ipfix-req] unidirection vs. bidirectional flows > > > > > > > > > > > > > > > > > > > > On Mon, 7 Jan 2002, Carter Bullard wrote: > > > > > > > > > Gentle people, > > > > > There is absolutely no reason for IPFIX to limit the type > > > > > of flow data that it supports. The difference between > > > > uni-directional > > > > > and bi-directional flow records is simply the type of > > > > metrics that are > > > > > reported. The flow descriptors and keys are all the > > same. Because > > > > > IPFIX must support > > > > ^^^^^ > > > > > > > > If "keys" you are refering to are the fields used to identify > > > > a flow then "SRC interface" do not fit into the statement > > > > "The flow descriptors and keys are all same for both > > > > unidirectional and bi-directional flow records" mentioned > > > > above because SRC interface will make it unidirectional. > > > > > > > > -vamsi > > > > > > > > > > > > >reporting metrics that each vendors will not implement, such > > > > as jitter > > > > >and one-way delay, which are the few metrics that are > > meaningful in > > > > >uni-directional flow data models, supporting metrics > > that report on > > > > >return traffic should not be a hardship for IPFIX. > > > > > > > > > > But, I believe that if you are going to make a design > > > > choice it should > > > > > be motivated by design goals. What benefit do you gain > > by limiting > > > > > IPFIX to a simple uni-directional > > > > > flow model? Is a uni-directional flow data model the best > > > > > data model for reporting all the metrics and network > > behaviors that > > > > > relate to network flows? Is it the best data model for > > > > supporting all > > > > > the existing commercial network flow data? > > > > > > > > > > But what about even the simplest of applications of > > flow data? Is a > > > > > uni-directional flow data model the best solution for > > reporting the > > > > > RTT of a ping volley? The empirical bulk transfer > > > > properties of a TCP > > > > > connection? How about TCP packet retransmission? Existence > > > > of routing > > > > > failures in an OSPF routed network? Application response time? > > > > > Rerouting indications? Reachability fault detection? > > > > Access policy > > > > > validation? Connectivity status? > > > > > > > > > > Even FTP supports both ASCII and binary data transfer. > > > > Handling both > > > > > uni-directional data and bi-directional data in IPFIX > > > > should be pretty > > > > > easy stuff. > > > > > > > > > > > > > > > Carter > > > > > > > > > > Carter Bullard > > > > > QoSient, LLC > > > > > 300 E. 56th Street, Suite 18K > > > > > New York, New York 10022 > > > > > > > > > > carter@qosient.com > > > > > Phone +1 212 588-9133 > > > > > Fax +1 212 588-9134 > > > > > http://qosient.com > > > > > > > > > > > > > > > > > > > > > > > > > > -----Original Message----- > > > > > > From: majordomo listserver > > > > [mailto:majordomo@mil.doit.wisc.edu] On > > > > > > Behalf Of Dave Plonka > > > > > > Sent: Monday, January 07, 2002 4:52 PM > > > > > > To: ipfix-req@net.doit.wisc.edu > > > > > > Subject: [ipfix-req] unidirection vs. bidirectional flows > > > > > > > > > > > > > > > > > > > > > > > > IPFIX folks, > > > > > > > > > > > > As many of you recall, whether or not IPFIX should specify a > > > > > > unidirectional or bidirectional flow definition was a hot > > > > topic at > > > > > > the WG meeting. > > > > > > > > > > > > I think that specifying *something* gets us far along in > > > > > > communicating what a "flow" is. Personally, I would > > like us to > > > > > > specify one or the other and I am in favor a unidirection flow > > > > > > definition. > > > > > > > > > > > > Here are some thoughts about arguments made for > > > > bidirectional flows: > > > > > > > > > > > > A unidirectional flow definition means that the atomic > > > > flow report > > > > > > record is unidirection during the reporting/export phase. For > > > > > > instance, a single packet counter for a given > > > > > > protocol/srcaddr/srcport/dstaddr/dstport n-tuple. > > However, since > > > > > > the exporter is somewhat of a "black box", it need not be > > > > > > constrained to a unidirectional model internally. > > > > > > > > > > > > A unidirectional flow definition does not prevent a collector > > > > > > implementor from using a bi-directional notion of flows > > > > internally > > > > > > nor does it prevent it from being stored in a > > bidirectional way > > > > > > prior to export. Conversely if we specify a bidirectional > > > > > > definition, it does disqualify unidirectional implementations. > > > > > > > > > > > > For those that wish to perform measurements which may require > > > > > > bidirectional flow information (perhaps because start/end > > > > timestamp > > > > > > synchronization is required for flow pairs), the > > IPFIX data model > > > > > > and transport could define a mechanism by which the > > > > exporter could > > > > > > inform the collector about which unidirectional flow > > records are > > > > > > matched pairs. That is, the exporter could help the > > > > collector find > > > > > > the matched pairs of flow information, perhaps by > > > > exporting the flow > > > > > > records in pairs or by generating a numeric > > identifier that is an > > > > > > attribute of both flow records. Of course only bidirectional > > > > > > collectors would understand this convention, but a > > > > > > unidirection collector could ignore it and still make use of > > > > > > the exported flow information from an exporter that > > > > > > implemented this "bidirectional flow pairing" feature. > > > > > > > > > > > > Dave > > > > > > > > > > > > -- > > > > > > ":) You will make many changes before settling > > satisfactorily. :)" > > > > > > - the actual message (complete with smiley faces) that I > > > > received > > > > > > in a > > > > > > fortune cookie at dinner after the IPFIX WG meeting > > > > during IETF 52 > > > > > > in Salt Lake City > > > > > > > > > > > > -- > > > > > > Help mailto:majordomo@net.doit.wisc.edu and say "help" > > > > > > in message body > > > > > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > > > "unsubscribe > > > > > > ipfix" in message body > > > > > > Archive http://ipfix.doit.wisc.edu/archive/ > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > Help mailto:majordomo@net.doit.wisc.edu and say > > > > "help" in message body > > > > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > "unsubscribe > > > > > ipfix" in message body > > > > > Archive http://ipfix.doit.wisc.edu/archive/ > > > > > > > > > > > > > > > > > > > > > > > -- > > > Help mailto:majordomo@net.doit.wisc.edu and say > > "help" in message body > > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > > "unsubscribe ipfix" in message body > > > Archive http://ipfix.doit.wisc.edu/archive/ > > > > -- > > Help mailto:majordomo@net.doit.wisc.edu and say "help" > > in message body > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > "unsubscribe ipfix" in message body > > Archive http://ipfix.doit.wisc.edu/archive/ > > > > > > > > -- > Help mailto:majordomo@net.doit.wisc.edu and say "help" in > message body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Tue Jan 8 14:18:08 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA06102 for ; Tue, 8 Jan 2002 14:18:08 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16O1U1-0004v0-00 for ipfix-list@mil.doit.wisc.edu; Tue, 08 Jan 2002 13:00:25 -0600 Received: from d2cspimg001.smartpipes.com ([63.89.185.24]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 16O1Ty-0004s1-00 for ipfix@net.doit.wisc.edu; Tue, 08 Jan 2002 13:00:22 -0600 Received: by D2CSPIMG001.smartpipes.com with Internet Mail Service (5.5.2653.19) id <44MDGJMW>; Tue, 8 Jan 2002 18:59:52 -0000 Message-ID: <4652644B98DFF34696801F8F3070D3FE01188578@D2CSPEXM001.smartpipes.com> From: "Wang, Cliff" To: "'calato@riverstonenet.com'" , ipfix@net.doit.wisc.edu Subject: RE: [ipfix] security requirement for IPFIX protocol Date: Tue, 8 Jan 2002 18:59:50 -0000 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain Precedence: bulk Sender: majordomo listserver -----Original Message----- From: calato@riverstonenet.com [mailto:calato@riverstonenet.com] Sent: Tuesday, January 08, 2002 10:48 AM To: ipfix@net.doit.wisc.edu Subject: Re: [ipfix] security requirement for IPFIX protocol [....deleted...] > > Also Section 8 discusses some security considerations. Client (probe) > to server (collection station) host level mutual authentication has > not been fully discussed. I think this should fit into the WG charter: > "Identify and address any security privacy concerns affecting flow > data. Determine technology for securing the flow information export > data, e.g. TLS. ". > > What in my mind is that in order to secure IPFIX data, you got to > provide both host security (where data is stored) and transport > security (where data is sent). However, we should only try to provide > requirement and use existing solutions rather than building solutions > from ground up. I don't think protection of data stored on the server or the client is in our charter. We only need to deal with transporting the data securely. > > > [cliff] I would suggest we clearly spell that out in the charter. On the other hand, I am still not clear whether authentication between the probe and the collection station is a MUST or SHOULD. My argument is that if you want the transport to be secure, then you probably also want the two ends of the transport to be authenticated to each other. Otherwise, from the security point of view, the extra effort of transport security can be potentially wasted. > > > > > > > -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Tue Jan 8 16:03:18 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA10428 for ; Tue, 8 Jan 2002 16:03:13 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16O32f-0006mu-00 for ipfix-list@mil.doit.wisc.edu; Tue, 08 Jan 2002 14:40:17 -0600 Received: from sj-msg-core-4.cisco.com ([171.71.163.10]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 16O32d-0006lw-00 for ipfix@net.doit.wisc.edu; Tue, 08 Jan 2002 14:40:15 -0600 Received: from mira-sjcd-1.cisco.com (mira-sjcd-1.cisco.com [171.69.43.44]) by sj-msg-core-4.cisco.com (8.11.3/8.9.1) with ESMTP id g08KdgJ20575; Tue, 8 Jan 2002 12:39:42 -0800 (PST) Received: from cisco.com (dhcp-171-71-137-45.cisco.com [171.71.137.45]) by mira-sjcd-1.cisco.com (Mirapoint) with ESMTP id AAU24194; Tue, 8 Jan 2002 12:39:49 -0800 (PST) Message-ID: <3C3B5B09.9FAC3B73@cisco.com> Date: Tue, 08 Jan 2002 12:48:09 -0800 From: Ganesh Sadasivan X-Mailer: Mozilla 4.76 [en]C-CCK-MCD (Windows NT 5.0; U) X-Accept-Language: en MIME-Version: 1.0 To: "K.C. Norseth" CC: Simon Leinen , ipfix@net.doit.wisc.edu Subject: Re: [ipfix] IPFIX Requirement presentation References: <20011214150236.A29404@doit.wisc.edu><3C1BDF61.B0BE8081@cisco.com> <001901c19805$a6ff6b60$850f880a@slc252750> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit What appears to be more appropriate to me here is to associate flows with an observation point rather than with an interface. The observation point could be one of a full system, a set of interfaces, a logical interface or even a sub-interface depending on how granular the measuring device can get upto. Ganesh "K.C. Norseth" wrote: > In the case of an etxternal probe where there is only one interface and the > probe is hooked to a mirrored port or bridge, the place where the probe is > connected, ther is no incomoing or outgoing interface. What would the > interface be that would satisfy MUST? > > K.C. > > ----- Original Message ----- > From: Simon Leinen > To: Ganesh Sadasivan > Cc: > Sent: Sunday, January 06, 2002 5:56 AM > Subject: Re: [ipfix] IPFIX Requirement presentation > > > >>>>> "gs" == Ganesh Sadasivan writes: > > > I want to bring up this point which I had raised on the > > > requirement spec. at the WG meeting. > > > It is from slide 5 on requirements where the flow definition has > > > been changed to include "incoming or outgoing interface" as a > > > MUST. > > > > The corresponding part of the requirements draft > > (ietf-ipfix-requirements-00) reads: > > > > 3.1. Interfaces > > > > The measuring device MUST be able to separate flows by the incoming > > interface or by the outgoing interface or by both of them. > > > > Note that this says "MUST be able to separate", not "MUST separate". > > Therefore I think your reasoning below doesn't apply: > > > > > There are cases like a SP is interested in getting all the > > > packest that go between 2 ASs. This does not have any strict > > > relevance to interface. An AS can be sourced by Multiple > > > interfaces in a router. Similar would be case if somebody > > > considers to define a flow based on other packet fields like > > > src/dst IP etc. In the general scheme of flow defintion ,I do not > > > see why there should be a restriction to include the interfaces > > > as a MUST. Claiming this as "most of the apps need the interfaces > > > today" is not a valid argument as this model is used to serve for > > > future apps. too. Therefore I request the WG to consider > > > removing this restriction. > > > > It is probably the case for *every* flow attribute that one can always > > come up with a sensible application that doesn't care about it. So it > > makes looks reasonable to make all flow attributes optional, in the > > sense that when an application isn't interested in them, the exporter > > doesn't have to send them in order to avoid waste of resources. This > > could be a requirement for the Flow Information Export protocol. > > > > But it also makes sense to make the *ability* of reporting certain > > attributes mandatory, so that applications can rely on them being > > available from all IPFIX compliant exporters. > > > > Personally I find the input and output interface sufficiently > > important to make their support mandatory. In particular the input > > interface is essential when one wants to do analysis of large-scale > > routing asymmetries. Input and output interfaces are also extremely > > useful when one collects flows at different points in the networks and > > wants to avoid counting the same flows twice. And the input interface > > cannot be derived from the source address and routing information, as > > can/must be done with e.g. the "source netmask", "upstream AS" or > > "source AS". > > > > Regards, > > -- > > Simon Leinen simon@babar.switch.ch > > SWITCH http://www.switch.ch/misc/leinen/ > > > > Computers hate being anthropomorphized. > > > > -- > > Help mailto:majordomo@net.doit.wisc.edu and say "help" in message > body > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > "unsubscribe ipfix" in message body > > Archive http://ipfix.doit.wisc.edu/archive/ > > -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Tue Jan 8 16:26:22 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA10836 for ; Tue, 8 Jan 2002 16:26:17 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16O3Sn-0007OX-00 for ipfix-list@mil.doit.wisc.edu; Tue, 08 Jan 2002 15:07:17 -0600 Received: from ctron-dnm.enterasys.com ([12.25.1.120] helo=ctron-dnm.ctron.com ident=firewall-user) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 16O3Sf-0007OP-00 for ipfix@net.doit.wisc.edu; Tue, 08 Jan 2002 15:07:09 -0600 Received: (from uucp@localhost) by ctron-dnm.ctron.com (8.8.7/8.8.7) id QAA17743; Tue, 8 Jan 2002 16:16:10 -0500 (EST) Received: from unknown(134.141.77.96) by ctron-dnm.ctron.com via smap (4.1) id xma017695; Tue, 8 Jan 02 16:15:07 -0500 Received: by cnc-exc1.enterasys.com with Internet Mail Service (5.5.2653.19) id ; Tue, 8 Jan 2002 16:05:28 -0500 Message-ID: <59358A738F45D51186A30008C74CE250DA0533@slc-exc1.ctron.com> From: "Norseth, KC" To: "'Ganesh Sadasivan'" , "K.C. Norseth" Cc: Simon Leinen , ipfix@net.doit.wisc.edu Subject: RE: [ipfix] IPFIX Requirement presentation Date: Tue, 8 Jan 2002 16:05:21 -0500 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01C19888.2F380820" Precedence: bulk Sender: majordomo listserver This message is in MIME format. Since your mail reader does not understand this format, some or all of this message may not be legible. ------_=_NextPart_001_01C19888.2F380820 Content-Type: text/plain You got it! |-----Original Message----- |From: Ganesh Sadasivan [mailto:gsadasiv@cisco.com] |Sent: Tuesday, January 08, 2002 1:48 PM |To: K.C. Norseth |Cc: Simon Leinen; ipfix@net.doit.wisc.edu |Subject: Re: [ipfix] IPFIX Requirement presentation | | |What appears to be more appropriate to me here is to associate |flows with an observation point rather than with an interface. |The observation point could be one of a full system, a set of |interfaces, a logical interface or even | |a sub-interface depending on how granular the measuring device |can get upto. Ganesh | |"K.C. Norseth" wrote: | |> In the case of an etxternal probe where there is only one interface |> and the probe is hooked to a mirrored port or bridge, the |place where |> the probe is connected, ther is no incomoing or outgoing interface. |> What would the interface be that would satisfy MUST? |> |> K.C. |> |> ----- Original Message ----- |> From: Simon Leinen |> To: Ganesh Sadasivan |> Cc: |> Sent: Sunday, January 06, 2002 5:56 AM |> Subject: Re: [ipfix] IPFIX Requirement presentation |> |> > >>>>> "gs" == Ganesh Sadasivan writes: |> > > I want to bring up this point which I had raised on the |> > > requirement spec. at the WG meeting. |> > > It is from slide 5 on requirements where the flow |definition has |> > > been changed to include "incoming or outgoing interface" as a |> > > MUST. |> > |> > The corresponding part of the requirements draft |> > (ietf-ipfix-requirements-00) reads: |> > |> > 3.1. Interfaces |> > |> > The measuring device MUST be able to separate flows |by the incoming |> > interface or by the outgoing interface or by both of them. |> > |> > Note that this says "MUST be able to separate", not "MUST |separate". |> > Therefore I think your reasoning below doesn't apply: |> > |> > > There are cases like a SP is interested in getting all the |> > > packest that go between 2 ASs. This does not have any strict |> > > relevance to interface. An AS can be sourced by Multiple |> > > interfaces in a router. Similar would be case if somebody |> > > considers to define a flow based on other packet fields like |> > > src/dst IP etc. In the general scheme of flow |defintion ,I do not |> > > see why there should be a restriction to include the |interfaces |> > > as a MUST. Claiming this as "most of the apps need |the interfaces |> > > today" is not a valid argument as this model is used |to serve for |> > > future apps. too. Therefore I request the WG to consider |> > > removing this restriction. |> > |> > It is probably the case for *every* flow attribute that one can |> > always come up with a sensible application that doesn't care about |> > it. So it makes looks reasonable to make all flow attributes |> > optional, in the sense that when an application isn't |interested in |> > them, the exporter doesn't have to send them in order to |avoid waste |> > of resources. This could be a requirement for the Flow |Information |> > Export protocol. |> > |> > But it also makes sense to make the *ability* of reporting certain |> > attributes mandatory, so that applications can rely on them being |> > available from all IPFIX compliant exporters. |> > |> > Personally I find the input and output interface sufficiently |> > important to make their support mandatory. In particular |the input |> > interface is essential when one wants to do analysis of |large-scale |> > routing asymmetries. Input and output interfaces are also |extremely |> > useful when one collects flows at different points in the networks |> > and wants to avoid counting the same flows twice. And the input |> > interface cannot be derived from the source address and routing |> > information, as can/must be done with e.g. the "source netmask", |> > "upstream AS" or "source AS". |> > |> > Regards, |> > -- |> > Simon Leinen simon@babar.switch.ch |> > SWITCH http://www.switch.ch/misc/leinen/ |> > |> > Computers hate being anthropomorphized. |> > |> > -- |> > Help mailto:majordomo@net.doit.wisc.edu and say |"help" in message |> body |> > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say |"unsubscribe |> > ipfix" in message body |> > Archive http://ipfix.doit.wisc.edu/archive/ |> > | | |-- |Help mailto:majordomo@net.doit.wisc.edu and say "help" |in message body |Unsubscribe mailto:majordomo@net.doit.wisc.edu and say |"unsubscribe ipfix" in message body |Archive http://ipfix.doit.wisc.edu/archive/ | ------_=_NextPart_001_01C19888.2F380820 Content-Type: text/html RE: [ipfix] IPFIX Requirement presentation

You got it!

|-----Original Message-----
|From: Ganesh Sadasivan [mailto:gsadasiv@cisco.com]
|Sent: Tuesday, January 08, 2002 1:48 PM
|To: K.C. Norseth
|Cc: Simon Leinen; ipfix@net.doit.wisc.edu
|Subject: Re: [ipfix] IPFIX Requirement presentation
|
|
|What appears to be more appropriate to me here is to associate
|flows with an observation point rather than with an interface.
|The observation point could be one of a full system, a set of
|interfaces, a logical interface or even
|
|a sub-interface depending on how granular the measuring device
|can get upto. Ganesh
|
|"K.C. Norseth" wrote:
|
|> In the case of an etxternal probe where there is only one interface
|> and the probe is hooked to a mirrored port or bridge, the
|place where
|> the probe is connected, ther is no incomoing or outgoing interface.
|> What would the interface be that would satisfy MUST?
|>
|> K.C.
|>
|> ----- Original Message -----
|> From: Simon Leinen <simon@limmat.switch.ch>
|> To: Ganesh Sadasivan <gsadasiv@cisco.com>
|> Cc: <ipfix@net.doit.wisc.edu>
|> Sent: Sunday, January 06, 2002 5:56 AM
|> Subject: Re: [ipfix] IPFIX Requirement presentation
|>
|> > >>>>> "gs" == Ganesh Sadasivan <gsadasiv@cisco.com> writes:
|> > >    I want to bring up this point which I had raised on the
|> > >    requirement spec. at the WG meeting.
|> > >    It is from slide 5 on requirements where the flow
|definition has
|> > >    been changed to include "incoming or outgoing interface" as a
|> > >    MUST.
|> >
|> > The corresponding part of the requirements draft
|> > (ietf-ipfix-requirements-00) reads:
|> >
|> >   3.1. Interfaces
|> >
|> >      The measuring device MUST be able to separate flows
|by the incoming
|> >      interface or by the outgoing interface or by both of them.
|> >
|> > Note that this says "MUST be able to separate", not "MUST
|separate".
|> > Therefore I think your reasoning below doesn't apply:
|> >
|> > >    There are cases like a SP is interested in getting all the
|> > >    packest that go between 2 ASs. This does not have any strict
|> > >    relevance to interface. An AS can be sourced by Multiple
|> > >    interfaces in a router. Similar would be case if somebody
|> > >    considers to define a flow based on other packet fields like
|> > >    src/dst IP etc. In the general scheme of flow
|defintion ,I do not
|> > >    see why there should be a restriction to include the
|interfaces
|> > >    as a MUST. Claiming this as "most of the apps need
|the interfaces
|> > >    today" is not a valid argument as this model is used
|to serve for
|> > >    future apps. too.  Therefore I request the WG to consider
|> > >    removing this restriction.
|> >
|> > It is probably the case for *every* flow attribute that one can
|> > always come up with a sensible application that doesn't care about
|> > it.  So it makes looks reasonable to make all flow attributes
|> > optional, in the sense that when an application isn't
|interested in
|> > them, the exporter doesn't have to send them in order to
|avoid waste
|> > of resources.  This could be a requirement for the Flow
|Information
|> > Export protocol.
|> >
|> > But it also makes sense to make the *ability* of reporting certain
|> > attributes mandatory, so that applications can rely on them being
|> > available from all IPFIX compliant exporters.
|> >
|> > Personally I find the input and output interface sufficiently
|> > important to make their support mandatory.  In particular
|the input
|> > interface is essential when one wants to do analysis of
|large-scale
|> > routing asymmetries.  Input and output interfaces are also
|extremely
|> > useful when one collects flows at different points in the networks
|> > and wants to avoid counting the same flows twice.  And the input
|> > interface cannot be derived from the source address and routing
|> > information, as can/must be done with e.g. the "source netmask",
|> > "upstream AS" or "source AS".
|> >
|> > Regards,
|> > --
|> > Simon Leinen        simon@babar.switch.ch
|> > SWITCH    http://www.switch.ch/misc/leinen/
|> >
|> >        Computers hate being anthropomorphized.
|> >
|> > --
|> > Help        mailto:majordomo@net.doit.wisc.edu and say
|"help" in message
|> body
|> > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say
|"unsubscribe
|> > ipfix" in message body
|> > Archive     http://ipfix.doit.wisc.edu/archive/
|> >
|
|
|--
|Help        mailto:majordomo@net.doit.wisc.edu and say "help"
|in message body
|Unsubscribe mailto:majordomo@net.doit.wisc.edu and say
|"unsubscribe ipfix" in message body
|Archive     http://ipfix.doit.wisc.edu/archive/
|

------_=_NextPart_001_01C19888.2F380820-- -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Tue Jan 8 17:26:07 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA12334 for ; Tue, 8 Jan 2002 17:26:07 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16O4R9-0000Wz-00 for ipfix-list@mil.doit.wisc.edu; Tue, 08 Jan 2002 16:09:39 -0600 Received: from relay1.pair.com ([209.68.1.20] helo=relay.pair.com) by mil.doit.wisc.edu with smtp (Exim 3.13 #1) id 16O4R6-0000WC-00 for ipfix-req@net.doit.wisc.edu; Tue, 08 Jan 2002 16:09:36 -0600 Received: (qmail 88905 invoked from network); 8 Jan 2002 22:09:33 -0000 Received: from pool-162-83-252-213.ny5030.east.verizon.net (HELO sphynx) (162.83.252.213) by relay1.pair.com with SMTP; 8 Jan 2002 22:09:33 -0000 X-pair-Authenticated: 162.83.252.213 Reply-To: From: "Carter Bullard" To: Subject: RE: [ipfix-req] unidirection vs. bidirectional flows Date: Tue, 8 Jan 2002 17:09:30 -0500 Organization: QoSient, LLC Message-ID: <5C8959A16A71B449AE793CF52FBBED6603BE1C@ptah.newyork.qosient.com> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook, Build 10.0.3416 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 Importance: Normal Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit [sorry for the delay on this, it bounced and this is a resend] Hey Vamsi, I'm one for allowing the most flexible flow descriptors possible. If a switch vendor wants to report on internal switch path performance, I want to support that. Whether that is uni-directional or bi-directional, I don't care. The vendor decided it wanted to generate the record, and IPFIX should support transporting it. If someone wants to do bi-directional MPLS flows, I won't understand why, but I'm not going to get in the way of them doing it. The idea is that if a vendor engineers its box to generate a particular type of flow data, IPFIX should be able to transport it, because if IPFIX is not the protocol of choice, it will fail as a standard. Carter Carter Bullard QoSient, LLC 300 E. 56th Street, Suite 18K New York, New York 10022 carter@qosient.com Phone +1 212 588-9133 Fax +1 212 588-9134 http://qosient.com =20 -----Original Message----- From: Vamsidhar Valluri [mailto:vvalluri@cisco.com]=20 Sent: Monday, January 07, 2002 7:52 PM To: Carter Bullard Cc: plonka@doit.wisc.edu; ipfix-req@net.doit.wisc.edu Subject: RE: [ipfix-req] unidirection vs. bidirectional flows On Mon, 7 Jan 2002, Carter Bullard wrote: > Hey Vamsi, > Hmmm, is the interface a part of the flow defintion? > Not a problem. Lets look at how this may work. Suppose there is a=20 > SRC interface and a DST interface, so that a flow is simply all the=20 > packets moving from SRC -> DST. The return flow would be all the=20 > packets moving from DST -> SRC. > > So the flow is reported as SRC <-> DST, with the simple addition of > the DST -> SRC metrics. Makes more sense than independantly reporting > two half-duplex flow records, which could be skewed in time so > badly,=20 that the two records cannot be correlated at all. Hi Carter, True, but there are certain high end platforms which will take considerable hit if destination interface has to become part of "keys". Other way of looking at it is to take away "interfaces" from keys so that bi-directional flows map to the same flow record but here we may lose interface information because birectional flows might take different interfaces at different point of time, if not then we are fine. -vamsi > > > Carter > > Carter Bullard > QoSient, LLC > 300 E. 56th Street, Suite 18K > New York, New York 10022 > > carter@qosient.com > Phone +1 212 588-9133 > Fax +1 212 588-9134 > http://qosient.com > > > > -----Original Message----- > > From: Vamsidhar Valluri [mailto:vvalluri@cisco.com] > > Sent: Monday, January 07, 2002 6:58 PM > > To: Carter Bullard > > Cc: plonka@doit.wisc.edu; ipfix-req@net.doit.wisc.edu > > Subject: RE: [ipfix-req] unidirection vs. bidirectional flows > > > > > > > > > > On Mon, 7 Jan 2002, Carter Bullard wrote: > > > > > Gentle people, > > > There is absolutely no reason for IPFIX to limit the type of > > > flow=20 data that it supports. The difference between > > uni-directional > > > and bi-directional flow records is simply the type of > > metrics that are > > > reported. The flow descriptors and keys are all the same. =20 > > > Because IPFIX must support > > ^^^^^ > > > > If "keys" you are refering to are the fields used to identify a flow > > then "SRC interface" do not fit into the statement "The flow=20 > > descriptors and keys are all same for both unidirectional and=20 > > bi-directional flow records" mentioned above because SRC > > interface=20 will make it unidirectional. > > > > -vamsi > > > > > > >reporting metrics that each vendors will not implement, such > > as jitter > > >and one-way delay, which are the few metrics that are > > >meaningful=20 in uni-directional flow data models, supporting > > >metrics that=20 report on return traffic should not be a hardship > > >for IPFIX. > > > > > > But, I believe that if you are going to make a design > > choice it should > > > be motivated by design goals. What benefit do you gain by=20 > > > limiting IPFIX to a simple uni-directional > > > flow model? Is a uni-directional flow data model the best > > > data model for reporting all the metrics and network behaviors=20 > > > that relate to network flows? Is it the best data model for > > supporting all > > > the existing commercial network flow data? > > > > > > But what about even the simplest of applications of flow data? > > > Is=20 a uni-directional flow data model the best solution for > > > reporting=20 the RTT of a ping volley? The empirical bulk > > > transfer > > properties of a TCP > > > connection? How about TCP packet retransmission? Existence > > of routing > > > failures in an OSPF routed network? Application response time?=20 > > > Rerouting indications? Reachability fault detection? > > Access policy > > > validation? Connectivity status? > > > > > > Even FTP supports both ASCII and binary data transfer. > > Handling both > > > uni-directional data and bi-directional data in IPFIX > > should be pretty > > > easy stuff. > > > > > > > > > Carter > > > > > > Carter Bullard > > > QoSient, LLC > > > 300 E. 56th Street, Suite 18K > > > New York, New York 10022 > > > > > > carter@qosient.com > > > Phone +1 212 588-9133 > > > Fax +1 212 588-9134 > > > http://qosient.com > > > > > > > > > > > > > > > > -----Original Message----- > > > > From: majordomo listserver > > [mailto:majordomo@mil.doit.wisc.edu] On > > > > Behalf Of Dave Plonka > > > > Sent: Monday, January 07, 2002 4:52 PM > > > > To: ipfix-req@net.doit.wisc.edu > > > > Subject: [ipfix-req] unidirection vs. bidirectional flows > > > > > > > > > > > > > > > > IPFIX folks, > > > > > > > > As many of you recall, whether or not IPFIX should specify a=20 > > > > unidirectional or bidirectional flow definition was a hot > > topic at > > > > the WG meeting. > > > > > > > > I think that specifying *something* gets us far along in=20 > > > > communicating what a "flow" is. Personally, I would like us > > > > to=20 specify one or the other and I am in favor a unidirection > > > > flow=20 definition. > > > > > > > > Here are some thoughts about arguments made for > > bidirectional flows: > > > > > > > > A unidirectional flow definition means that the atomic > > flow report > > > > record is unidirection during the reporting/export phase. > > > > For=20 instance, a single packet counter for a given=20 > > > > protocol/srcaddr/srcport/dstaddr/dstport n-tuple. However,=20 > > > > since the exporter is somewhat of a "black box", it need not > > > > be=20 constrained to a unidirectional model internally. > > > > > > > > A unidirectional flow definition does not prevent a collector=20 > > > > implementor from using a bi-directional notion of flows > > internally > > > > nor does it prevent it from being stored in a bidirectional > > > > way=20 prior to export. Conversely if we specify a > > > > bidirectional=20 definition, it does disqualify unidirectional > > > > implementations. > > > > > > > > For those that wish to perform measurements which may require=20 > > > > bidirectional flow information (perhaps because start/end > > timestamp > > > > synchronization is required for flow pairs), the IPFIX data=20 > > > > model and transport could define a mechanism by which the > > exporter could > > > > inform the collector about which unidirectional flow records are > > > > matched pairs. That is, the exporter could help the > > collector find > > > > the matched pairs of flow information, perhaps by > > exporting the flow > > > > records in pairs or by generating a numeric identifier that > > > > is=20 an attribute of both flow records. Of course only > > > > bidirectional > > > > collectors would understand this convention, but a > > > > unidirection=20 collector could ignore it and still make use of > > > > the exported=20 flow information from an exporter that > > > > implemented this=20 "bidirectional flow pairing" feature. > > > > > > > > Dave > > > > > > > > -- > > > > ":) You will make many changes before settling > > > > satisfactorily.=20 :)" > > > > - the actual message (complete with smiley faces) that I > > received > > > > in a > > > > fortune cookie at dinner after the IPFIX WG meeting > > during IETF 52 > > > > in Salt Lake City > > > > > > > > -- > > > > Help mailto:majordomo@net.doit.wisc.edu and say "help" > > > > in message body > > > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > "unsubscribe > > > > ipfix" in message body > > > > Archive http://ipfix.doit.wisc.edu/archive/ > > > > > > > > > > > > > > > > > > > > -- > > > Help mailto:majordomo@net.doit.wisc.edu and say > > "help" in message body > > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say=20 > > > "unsubscribe ipfix" in message body > > > Archive http://ipfix.doit.wisc.edu/archive/ > > > > > > > > > > > > ----- End forwarded message ----- -- "Give a man a fish, and he'll resent you for a day. Teach a man to fish, and he'll resent you all his life." - from the Tenets of Pragmatism (Pragmatic Society) -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Tue Jan 8 17:30:41 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA12430 for ; Tue, 8 Jan 2002 17:30:40 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16O4UV-0001J2-00 for ipfix-list@mil.doit.wisc.edu; Tue, 08 Jan 2002 16:13:07 -0600 Received: from relay1.pair.com ([209.68.1.20] helo=relay.pair.com) by mil.doit.wisc.edu with smtp (Exim 3.13 #1) id 16O4UT-0001IU-00 for ipfix-req@net.doit.wisc.edu; Tue, 08 Jan 2002 16:13:05 -0600 Received: (qmail 89525 invoked from network); 8 Jan 2002 22:13:02 -0000 Received: from pool-162-83-252-213.ny5030.east.verizon.net (HELO sphynx) (162.83.252.213) by relay1.pair.com with SMTP; 8 Jan 2002 22:13:02 -0000 X-pair-Authenticated: 162.83.252.213 Reply-To: From: "Carter Bullard" To: Cc: Subject: RE: [ipfix-req] unidirection vs. bidirectional flows Date: Tue, 8 Jan 2002 17:12:53 -0500 Organization: QoSient, LLC Message-ID: <5C8959A16A71B449AE793CF52FBBED66018FD4@ptah.newyork.qosient.com> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook, Build 10.0.3416 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 In-Reply-To: <3C3B237C.2F030A14@riverstonenet.com> Importance: Normal Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Hey Paul, The RTT metric in the example is Duration. If your uni-directional flow record's have start and stop times, then you could derive the Duration by subtracting the start times of one record from the stop times of the other record, but you need to have something that states that these records are related. You won't be able to infer this from the 5-tuple flow descriptor in either record, so you'll need some type of binding identifier. My problem with this is that you get 2 complete records for a single network event. They include redundant headers, redundant flow descriptors, two unused time-stamps, and we introduce new requirements to guarantee that the binding identifiers be unique. The added bandwidth requirements for transmitting the redundant information can get pretty big in high end monitors. A 5 tuple flow descriptor for IPv4 addresses is 16 bytes. For IPv6 its 38 bytes. The redundant timestamps are 16 bytes, and an IPFIX record header will probably be around 16-32 bytes. So the redundant information is around 64-128 bytes. That's pretty significant in my book. The XML schema for the data that Argus generates is at http://qosient.com/argus/Xml/ArgusRecord_xsd/ArgusRecord.htm Argus is a bi-directional flow monitor that generates very flexible flow records. Lots of optional fields. Now I am not recommending Argus's data strategy for IPFIX, I'm just pointing out that Argus like schemas can handle uni-directional and bi-directional reports at the same time, because it supports optional fields. Carter Carter Bullard QoSient, LLC 300 E. 56th Street, Suite 18K New York, New York 10022 carter@qosient.com Phone +1 212 588-9133 Fax +1 212 588-9134 http://qosient.com > -----Original Message----- > From: calato [mailto:calato] On Behalf Of calato@riverstonenet.com > Sent: Tuesday, January 08, 2002 11:51 AM > To: carter@qosient.com > Cc: ipfix-req@net.doit.wisc.edu > Subject: Re: [ipfix-req] unidirection vs. bidirectional flows > > > > I'm not sure. Can you tell me the fields reported today? > I'll see if I can make sense of it as 2 uni-directional > flows. > > Paul > > Carter Bullard wrote: > > > > Hey Paul, > > So I have a flow monitor that generates RTT times for DNS > > transactions. How do I report RTT for a DNS transaction > using IPFIX > > if it has the restriction that I must report my metrics using a > > uni-directional semantic? > > > > So how do I do it? Break the records in two, and somehow > > tag that the two records are unambiguously related, and > > where do I put the bi-directional semantic? In both of > > the records, in the first one, the last one, in a separate third > > record? > > > > Doesn't seem very efficient. > > > > Carter > > > > Carter Bullard > > QoSient, LLC > > 300 E. 56th Street, Suite 18K > > New York, New York 10022 > > > > carter@qosient.com > > Phone +1 212 588-9133 > > Fax +1 212 588-9134 > > http://qosient.com > > > > > -----Original Message----- > > > From: majordomo listserver > [mailto:majordomo@mil.doit.wisc.edu] On > > > Behalf Of calato@riverstonenet.com > > > Sent: Tuesday, January 08, 2002 11:30 AM > > > To: ipfix-req@net.doit.wisc.edu > > > Subject: Re: [ipfix-req] unidirection vs. bidirectional flows > > > > > > > > > > > > Carter, > > > > > > I'm a bit confused. Why doesn't the ability to provide > > > bi-directional info, as Dave described, meet your needs? Those > > > devices that can do bi-directional will report it that way, those > > > that can't will report it as uni-directional. Some down stream > > > application will have to do the work to turn 2 > uni-directional flows > > > into one bi-directional one, if that is what's needed. And if > > > uni-directional is needed the app will split it apart. > > > > > > But if you require bi-directional reporting, many devices wont be > > > able to do it. Which then eliminates any down stream analysis. > > > > > > Paul > > > > > > > > > Carter Bullard wrote: > > > > > > > > Hey Vamsi, > > > > Hmmm, is the interface a part of the flow defintion? Not a > > > > problem. Lets look at how this may work. Suppose > there is a SRC > > > > interface and a DST interface, so that a flow is simply all the > > > > packets moving from SRC -> DST. The return flow would > be all the > > > > packets moving from DST -> SRC. > > > > > > > > So the flow is reported as SRC <-> DST, with the simple > > > > addition of the DST -> SRC metrics. Makes more sense than > > > > independantly reporting two half-duplex flow records, > which could > > > > be skewed in time so badly, that the two records cannot be > > > > correlated at all. > > > > > > > > Carter > > > > > > > > Carter Bullard > > > > QoSient, LLC > > > > 300 E. 56th Street, Suite 18K > > > > New York, New York 10022 > > > > > > > > carter@qosient.com > > > > Phone +1 212 588-9133 > > > > Fax +1 212 588-9134 > > > > http://qosient.com > > > > > > > > > -----Original Message----- > > > > > From: Vamsidhar Valluri [mailto:vvalluri@cisco.com] > > > > > Sent: Monday, January 07, 2002 6:58 PM > > > > > To: Carter Bullard > > > > > Cc: plonka@doit.wisc.edu; ipfix-req@net.doit.wisc.edu > > > > > Subject: RE: [ipfix-req] unidirection vs. bidirectional flows > > > > > > > > > > > > > > > > > > > > > > > > > On Mon, 7 Jan 2002, Carter Bullard wrote: > > > > > > > > > > > Gentle people, > > > > > > There is absolutely no reason for IPFIX to limit > the type of > > > > > > flow data that it supports. The difference between > > > > > uni-directional > > > > > > and bi-directional flow records is simply the type of > > > > > metrics that are > > > > > > reported. The flow descriptors and keys are all the > > > same. Because > > > > > > IPFIX must support > > > > > ^^^^^ > > > > > > > > > > If "keys" you are refering to are the fields used to > identify a > > > > > flow then "SRC interface" do not fit into the statement "The > > > > > flow descriptors and keys are all same for both > unidirectional > > > > > and bi-directional flow records" mentioned above because SRC > > > > > interface will make it unidirectional. > > > > > > > > > > -vamsi > > > > > > > > > > > > > > > >reporting metrics that each vendors will not implement, such > > > > > as jitter > > > > > >and one-way delay, which are the few metrics that are > > > meaningful in > > > > > >uni-directional flow data models, supporting metrics > > > that report on > > > > > >return traffic should not be a hardship for IPFIX. > > > > > > > > > > > > But, I believe that if you are going to make a design > > > > > choice it should > > > > > > be motivated by design goals. What benefit do you gain > > > by limiting > > > > > > IPFIX to a simple uni-directional > > > > > > flow model? Is a uni-directional flow data model the best > > > > > > data model for reporting all the metrics and network > > > behaviors that > > > > > > relate to network flows? Is it the best data model for > > > > > supporting all > > > > > > the existing commercial network flow data? > > > > > > > > > > > > But what about even the simplest of applications of > > > flow data? Is a > > > > > > uni-directional flow data model the best solution for > > > reporting the > > > > > > RTT of a ping volley? The empirical bulk transfer > > > > > properties of a TCP > > > > > > connection? How about TCP packet retransmission? Existence > > > > > of routing > > > > > > failures in an OSPF routed network? Application > response time? > > > > > > Rerouting indications? Reachability fault detection? > > > > > Access policy > > > > > > validation? Connectivity status? > > > > > > > > > > > > Even FTP supports both ASCII and binary data transfer. > > > > > Handling both > > > > > > uni-directional data and bi-directional data in IPFIX > > > > > should be pretty > > > > > > easy stuff. > > > > > > > > > > > > > > > > > > Carter > > > > > > > > > > > > Carter Bullard > > > > > > QoSient, LLC > > > > > > 300 E. 56th Street, Suite 18K > > > > > > New York, New York 10022 > > > > > > > > > > > > carter@qosient.com > > > > > > Phone +1 212 588-9133 > > > > > > Fax +1 212 588-9134 > > > > > > http://qosient.com > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -----Original Message----- > > > > > > > From: majordomo listserver > > > > > [mailto:majordomo@mil.doit.wisc.edu] On > > > > > > > Behalf Of Dave Plonka > > > > > > > Sent: Monday, January 07, 2002 4:52 PM > > > > > > > To: ipfix-req@net.doit.wisc.edu > > > > > > > Subject: [ipfix-req] unidirection vs. bidirectional flows > > > > > > > > > > > > > > > > > > > > > > > > > > > > IPFIX folks, > > > > > > > > > > > > > > As many of you recall, whether or not IPFIX > should specify a > > > > > > > unidirectional or bidirectional flow definition was a hot > > > > > topic at > > > > > > > the WG meeting. > > > > > > > > > > > > > > I think that specifying *something* gets us far along in > > > > > > > communicating what a "flow" is. Personally, I would > > > like us to > > > > > > > specify one or the other and I am in favor a unidirection > > > > > > > flow definition. > > > > > > > > > > > > > > Here are some thoughts about arguments made for > > > > > bidirectional flows: > > > > > > > > > > > > > > A unidirectional flow definition means that the atomic > > > > > flow report > > > > > > > record is unidirection during the > reporting/export phase. > > > > > > > For instance, a single packet counter for a given > > > > > > > protocol/srcaddr/srcport/dstaddr/dstport n-tuple. > > > However, since > > > > > > > the exporter is somewhat of a "black box", it need not be > > > > > > > constrained to a unidirectional model internally. > > > > > > > > > > > > > > A unidirectional flow definition does not prevent a > > > > > > > collector implementor from using a bi-directional > notion of > > > > > > > flows > > > > > internally > > > > > > > nor does it prevent it from being stored in a > > > bidirectional way > > > > > > > prior to export. Conversely if we specify a > bidirectional > > > > > > > definition, it does disqualify unidirectional > > > > > > > implementations. > > > > > > > > > > > > > > For those that wish to perform measurements which may > > > > > > > require bidirectional flow information (perhaps because > > > > > > > start/end > > > > > timestamp > > > > > > > synchronization is required for flow pairs), the > > > IPFIX data model > > > > > > > and transport could define a mechanism by which the > > > > > exporter could > > > > > > > inform the collector about which unidirectional flow > > > records are > > > > > > > matched pairs. That is, the exporter could help the > > > > > collector find > > > > > > > the matched pairs of flow information, perhaps by > > > > > exporting the flow > > > > > > > records in pairs or by generating a numeric > > > identifier that is an > > > > > > > attribute of both flow records. Of course only > > > > > > > bidirectional collectors would understand this > convention, > > > > > > > but a unidirection collector could ignore it and > still make > > > > > > > use of the exported flow information from an > exporter that > > > > > > > implemented this "bidirectional flow pairing" feature. > > > > > > > > > > > > > > Dave > > > > > > > > > > > > > > -- > > > > > > > ":) You will make many changes before settling > > > satisfactorily. :)" > > > > > > > - the actual message (complete with smiley faces) that I > > > > > received > > > > > > > in a > > > > > > > fortune cookie at dinner after the IPFIX WG meeting > > > > > during IETF 52 > > > > > > > in Salt Lake City > > > > > > > > > > > > > > -- > > > > > > > Help mailto:majordomo@net.doit.wisc.edu > and say "help" > > > > > > > in message body > > > > > > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > > > > "unsubscribe > > > > > > > ipfix" in message body > > > > > > > Archive http://ipfix.doit.wisc.edu/archive/ > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > Help mailto:majordomo@net.doit.wisc.edu and say > > > > > "help" in message body > > > > > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > > "unsubscribe > > > > > > ipfix" in message body > > > > > > Archive http://ipfix.doit.wisc.edu/archive/ > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > Help mailto:majordomo@net.doit.wisc.edu and say > > > "help" in message body > > > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > > > "unsubscribe ipfix" in message body > > > > Archive http://ipfix.doit.wisc.edu/archive/ > > > > > > -- > > > Help mailto:majordomo@net.doit.wisc.edu and say "help" > > > in message body > > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe > > > ipfix" in message body > > > Archive http://ipfix.doit.wisc.edu/archive/ > > > > > > > > -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Wed Jan 9 10:39:15 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id KAA05652 for ; Wed, 9 Jan 2002 10:39:15 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16OKDS-0002JY-00 for ipfix-list@mil.doit.wisc.edu; Wed, 09 Jan 2002 09:00:34 -0600 Received: from yamato.ccrle.nec.de ([195.37.70.1]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 16OKDQ-0002HF-00 for ipfix-req@net.doit.wisc.edu; Wed, 09 Jan 2002 09:00:32 -0600 Received: from wallace.heidelberg.ccrle.nec.de (root@wallace [192.168.102.1]) by yamato.ccrle.nec.de (8.11.6/8.10.1) with ESMTP id g09F0OR15978; Wed, 9 Jan 2002 16:00:24 +0100 (CET) Received: from [192.168.102.164] (beta.heidelberg.ccrle.nec.de [192.168.102.164]) by wallace.heidelberg.ccrle.nec.de (8.9.3/8.9.3/SuSE Linux 8.9.3-0.1) with ESMTP id QAA10090; Wed, 9 Jan 2002 16:00:00 +0100 Date: Wed, 09 Jan 2002 16:01:04 +0100 From: Juergen Quittek To: carter@qosient.com cc: plonka@doit.wisc.edu, ipfix-req@net.doit.wisc.edu Subject: RE: [ipfix-req] unidirection vs. bidirectional flows Message-ID: <1274749929.1010592064@[192.168.102.164]> In-Reply-To: <5C8959A16A71B449AE793CF52FBBED66018FD2@ptah.newyork.qosient.com> X-Mailer: Mulberry/2.0.8 (Win32) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit --On 08 January 2002 11:33 -0500 Carter Bullard wrote: Hi Carter, > Hey Juergen, > There are a significant number of Cisco routers and switches > that do not support netflow. And Cisco/Riverstone do not > constitute the complete set of router/switch vendors on > the planet. I think I was talking about Juniper and Cisco, but this does not matter at all. This talking about 99% of all routers do this and that without any proove does lead us nowhere. > My entire position is that IPFIX should not make any > requirements/restrictions on the data model. I tend to agree when talking about the requirements, but when it comes to the architecture, it might get hard to avoid it completely. Juergen > Carter > > Carter Bullard > QoSient, LLC > 300 E. 56th Street, Suite 18K > New York, New York 10022 > > carter@qosient.com > Phone +1 212 588-9133 > Fax +1 212 588-9134 > http://qosient.com > >> -----Original Message----- >> From: Juergen Quittek [mailto:quittek@ccrle.nec.de] >> Sent: Tuesday, January 08, 2002 8:23 AM >> To: carter@qosient.com >> Cc: plonka@doit.wisc.edu; ipfix-req@net.doit.wisc.edu >> Subject: RE: [ipfix-req] unidirection vs. bidirectional flows >> >> >> --On 08 January 2002 07:34 -0500 Carter Bullard wrote: >> >> > Hey Juergen, >> > Just a note, the majority of switches and routers >> > today (> 90%) cannot support uni-directional flow record >> >> What routers are you talking about? Isn't NetFlow available >> for most CisCo and Juniper boxes? >> >> > reporting. However, virtually all switches and routers >> > (> 99%) support RMON-style interface statistics, which >> > are L1/L2 bi-directional flow metrics. >> > >> > I believe that adoption of a uni-directional flow model excludes >> > basically the entire population of legacy networking devices that >> > could easily report their existing bi-directional flow >> metrics using >> > IPFIX. >> > >> > Wouldn't this violate your fundamental constraint? >> >> Not at all. No one is trying to forbid existing technologies. >> But this is about IPFIX requirements. I said I am not sure >> whether we should specify uni- or bi-directional in the >> requirements at all, but I am sure I do not want to have >> "expensive" requirements. If you explain to me how to make >> bi-directional flows cheap, I will find it easier considering >> them as a potential requirement. >> >> Juergen >> -- >> Juergen Quittek quittek@ccrle.nec.de Tel: +49 6221 90511-15 >> NEC Europe Ltd., Network Laboratories Fax: +49 6221 90511-55 >> Adenauerplatz 6, 69115 Heidelberg, Germany http://www.ccrle.nec.de >> > > -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Wed Jan 9 10:46:32 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id KAA05773 for ; Wed, 9 Jan 2002 10:46:32 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16OKL4-0002U8-00 for ipfix-list@mil.doit.wisc.edu; Wed, 09 Jan 2002 09:08:26 -0600 Received: from yamato.ccrle.nec.de ([195.37.70.1]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 16OKL1-0002TZ-00 for ipfix-req@net.doit.wisc.edu; Wed, 09 Jan 2002 09:08:23 -0600 Received: from wallace.heidelberg.ccrle.nec.de (root@wallace [192.168.102.1]) by yamato.ccrle.nec.de (8.11.6/8.10.1) with ESMTP id g09F8ER16019; Wed, 9 Jan 2002 16:08:14 +0100 (CET) Received: from [192.168.102.164] (beta.heidelberg.ccrle.nec.de [192.168.102.164]) by wallace.heidelberg.ccrle.nec.de (8.9.3/8.9.3/SuSE Linux 8.9.3-0.1) with ESMTP id QAA10204; Wed, 9 Jan 2002 16:07:49 +0100 Date: Wed, 09 Jan 2002 16:08:55 +0100 From: Juergen Quittek To: carter@qosient.com, calato@riverstonenet.com cc: ipfix-req@net.doit.wisc.edu Subject: RE: [ipfix-req] unidirection vs. bidirectional flows Message-ID: <1275220606.1010592535@[192.168.102.164]> In-Reply-To: <5C8959A16A71B449AE793CF52FBBED66018FD4@ptah.newyork.qosient.com> X-Mailer: Mulberry/2.0.8 (Win32) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit --On 08 January 2002 17:12 -0500 Carter Bullard wrote: > Hey Paul, > The RTT metric in the example is Duration. If your > uni-directional flow record's have start and stop times, > then you could derive the Duration by subtracting > the start times of one record from the stop times of the > other record, but you need to have something that states > that these records are related. You won't be able to infer > this from the 5-tuple flow descriptor in either record, > so you'll need some type of binding identifier. Exactly. Now who's work should it be to find the binding? The exporter? Why not the application analyzing the data at the collector. It is much better suited for this job and can do it online or offline depending of capabilities and requirements for this analysis. But I would not ask the exporter to do this job. > My problem with this is that you get 2 complete records > for a single network event. They include redundant headers, > redundant flow descriptors, two unused time-stamps, and we > introduce new requirements to guarantee that the binding > identifiers be unique. The added bandwidth requirements > for transmitting the redundant information can get pretty > big in high end monitors. A 5 tuple flow descriptor for > IPv4 addresses is 16 bytes. For IPv6 its 38 bytes. The > redundant timestamps are 16 bytes, and an IPFIX record > header will probably be around 16-32 bytes. So the > redundant information is around 64-128 bytes. That's pretty > significant in my book. You are right. This dumb approach wastes resources. But that's why we require flexible flow records for IPFIX. If this above is your only measurement, then reduce the flow record to just two tomestamps and a rule (set) identifier and you waste much less bandwidth. Juergen -- Juergen Quittek quittek@ccrle.nec.de Tel: +49 6221 90511-15 NEC Europe Ltd., Network Laboratories Fax: +49 6221 90511-55 Adenauerplatz 6, 69115 Heidelberg, Germany http://www.ccrle.nec.de > The XML schema for the data that Argus generates is at > http://qosient.com/argus/Xml/ArgusRecord_xsd/ArgusRecord.htm > Argus is a bi-directional flow monitor that generates very > flexible flow records. Lots of optional fields. Now I am > not recommending Argus's data strategy for IPFIX, I'm just > pointing out that Argus like schemas can handle uni-directional > and bi-directional reports at the same time, because it > supports optional fields. > > > Carter > > Carter Bullard > QoSient, LLC > 300 E. 56th Street, Suite 18K > New York, New York 10022 > > carter@qosient.com > Phone +1 212 588-9133 > Fax +1 212 588-9134 > http://qosient.com > > > > >> -----Original Message----- >> From: calato [mailto:calato] On Behalf Of calato@riverstonenet.com >> Sent: Tuesday, January 08, 2002 11:51 AM >> To: carter@qosient.com >> Cc: ipfix-req@net.doit.wisc.edu >> Subject: Re: [ipfix-req] unidirection vs. bidirectional flows >> >> >> >> I'm not sure. Can you tell me the fields reported today? >> I'll see if I can make sense of it as 2 uni-directional >> flows. >> >> Paul >> >> Carter Bullard wrote: >> > >> > Hey Paul, >> > So I have a flow monitor that generates RTT times for DNS >> > transactions. How do I report RTT for a DNS transaction >> using IPFIX >> > if it has the restriction that I must report my metrics using a >> > uni-directional semantic? >> > >> > So how do I do it? Break the records in two, and somehow >> > tag that the two records are unambiguously related, and >> > where do I put the bi-directional semantic? In both of >> > the records, in the first one, the last one, in a separate third >> > record? >> > >> > Doesn't seem very efficient. >> > >> > Carter >> > >> > Carter Bullard >> > QoSient, LLC >> > 300 E. 56th Street, Suite 18K >> > New York, New York 10022 >> > >> > carter@qosient.com >> > Phone +1 212 588-9133 >> > Fax +1 212 588-9134 >> > http://qosient.com >> > >> > > -----Original Message----- >> > > From: majordomo listserver >> [mailto:majordomo@mil.doit.wisc.edu] On >> > > Behalf Of calato@riverstonenet.com >> > > Sent: Tuesday, January 08, 2002 11:30 AM >> > > To: ipfix-req@net.doit.wisc.edu >> > > Subject: Re: [ipfix-req] unidirection vs. bidirectional flows >> > > >> > > >> > > >> > > Carter, >> > > >> > > I'm a bit confused. Why doesn't the ability to provide >> > > bi-directional info, as Dave described, meet your needs? Those >> > > devices that can do bi-directional will report it that way, those >> > > that can't will report it as uni-directional. Some down stream >> > > application will have to do the work to turn 2 >> uni-directional flows >> > > into one bi-directional one, if that is what's needed. And if >> > > uni-directional is needed the app will split it apart. >> > > >> > > But if you require bi-directional reporting, many devices wont be >> > > able to do it. Which then eliminates any down stream analysis. >> > > >> > > Paul >> > > >> > > >> > > Carter Bullard wrote: >> > > > >> > > > Hey Vamsi, >> > > > Hmmm, is the interface a part of the flow defintion? Not a >> > > > problem. Lets look at how this may work. Suppose >> there is a SRC >> > > > interface and a DST interface, so that a flow is simply all the >> > > > packets moving from SRC -> DST. The return flow would >> be all the >> > > > packets moving from DST -> SRC. >> > > > >> > > > So the flow is reported as SRC <-> DST, with the simple >> > > > addition of the DST -> SRC metrics. Makes more sense than >> > > > independantly reporting two half-duplex flow records, >> which could >> > > > be skewed in time so badly, that the two records cannot be >> > > > correlated at all. >> > > > >> > > > Carter >> > > > >> > > > Carter Bullard >> > > > QoSient, LLC >> > > > 300 E. 56th Street, Suite 18K >> > > > New York, New York 10022 >> > > > >> > > > carter@qosient.com >> > > > Phone +1 212 588-9133 >> > > > Fax +1 212 588-9134 >> > > > http://qosient.com >> > > > >> > > > > -----Original Message----- >> > > > > From: Vamsidhar Valluri [mailto:vvalluri@cisco.com] >> > > > > Sent: Monday, January 07, 2002 6:58 PM >> > > > > To: Carter Bullard >> > > > > Cc: plonka@doit.wisc.edu; ipfix-req@net.doit.wisc.edu >> > > > > Subject: RE: [ipfix-req] unidirection vs. bidirectional flows >> > > > > >> > > > > >> > > > > >> > > > > >> > > > > On Mon, 7 Jan 2002, Carter Bullard wrote: >> > > > > >> > > > > > Gentle people, >> > > > > > There is absolutely no reason for IPFIX to limit >> the type of >> > > > > > flow data that it supports. The difference between >> > > > > uni-directional >> > > > > > and bi-directional flow records is simply the type of >> > > > > metrics that are >> > > > > > reported. The flow descriptors and keys are all the >> > > same. Because >> > > > > > IPFIX must support >> > > > > ^^^^^ >> > > > > >> > > > > If "keys" you are refering to are the fields used to >> identify a >> > > > > flow then "SRC interface" do not fit into the statement "The >> > > > > flow descriptors and keys are all same for both >> unidirectional >> > > > > and bi-directional flow records" mentioned above because SRC >> > > > > interface will make it unidirectional. >> > > > > >> > > > > -vamsi >> > > > > >> > > > > >> > > > > > reporting metrics that each vendors will not implement, such >> > > > > as jitter >> > > > > > and one-way delay, which are the few metrics that are >> > > meaningful in >> > > > > > uni-directional flow data models, supporting metrics >> > > that report on >> > > > > > return traffic should not be a hardship for IPFIX. >> > > > > > >> > > > > > But, I believe that if you are going to make a design >> > > > > choice it should >> > > > > > be motivated by design goals. What benefit do you gain >> > > by limiting >> > > > > > IPFIX to a simple uni-directional >> > > > > > flow model? Is a uni-directional flow data model the best >> > > > > > data model for reporting all the metrics and network >> > > behaviors that >> > > > > > relate to network flows? Is it the best data model for >> > > > > supporting all >> > > > > > the existing commercial network flow data? >> > > > > > >> > > > > > But what about even the simplest of applications of >> > > flow data? Is a >> > > > > > uni-directional flow data model the best solution for >> > > reporting the >> > > > > > RTT of a ping volley? The empirical bulk transfer >> > > > > properties of a TCP >> > > > > > connection? How about TCP packet retransmission? Existence >> > > > > of routing >> > > > > > failures in an OSPF routed network? Application >> response time? >> > > > > > Rerouting indications? Reachability fault detection? >> > > > > Access policy >> > > > > > validation? Connectivity status? >> > > > > > >> > > > > > Even FTP supports both ASCII and binary data transfer. >> > > > > Handling both >> > > > > > uni-directional data and bi-directional data in IPFIX >> > > > > should be pretty >> > > > > > easy stuff. >> > > > > > >> > > > > > >> > > > > > Carter >> > > > > > >> > > > > > Carter Bullard >> > > > > > QoSient, LLC >> > > > > > 300 E. 56th Street, Suite 18K >> > > > > > New York, New York 10022 >> > > > > > >> > > > > > carter@qosient.com >> > > > > > Phone +1 212 588-9133 >> > > > > > Fax +1 212 588-9134 >> > > > > > http://qosient.com >> > > > > > >> > > > > > >> > > > > > >> > > > > > >> > > > > > > -----Original Message----- >> > > > > > > From: majordomo listserver >> > > > > [mailto:majordomo@mil.doit.wisc.edu] On >> > > > > > > Behalf Of Dave Plonka >> > > > > > > Sent: Monday, January 07, 2002 4:52 PM >> > > > > > > To: ipfix-req@net.doit.wisc.edu >> > > > > > > Subject: [ipfix-req] unidirection vs. bidirectional flows >> > > > > > > >> > > > > > > >> > > > > > > >> > > > > > > IPFIX folks, >> > > > > > > >> > > > > > > As many of you recall, whether or not IPFIX >> should specify a >> > > > > > > unidirectional or bidirectional flow definition was a hot >> > > > > topic at >> > > > > > > the WG meeting. >> > > > > > > >> > > > > > > I think that specifying *something* gets us far along in >> > > > > > > communicating what a "flow" is. Personally, I would >> > > like us to >> > > > > > > specify one or the other and I am in favor a unidirection >> > > > > > > flow definition. >> > > > > > > >> > > > > > > Here are some thoughts about arguments made for >> > > > > bidirectional flows: >> > > > > > > >> > > > > > > A unidirectional flow definition means that the atomic >> > > > > flow report >> > > > > > > record is unidirection during the >> reporting/export phase. >> > > > > > > For instance, a single packet counter for a given >> > > > > > > protocol/srcaddr/srcport/dstaddr/dstport n-tuple. >> > > However, since >> > > > > > > the exporter is somewhat of a "black box", it need not be >> > > > > > > constrained to a unidirectional model internally. >> > > > > > > >> > > > > > > A unidirectional flow definition does not prevent a >> > > > > > > collector implementor from using a bi-directional >> notion of >> > > > > > > flows >> > > > > internally >> > > > > > > nor does it prevent it from being stored in a >> > > bidirectional way >> > > > > > > prior to export. Conversely if we specify a >> bidirectional >> > > > > > > definition, it does disqualify unidirectional >> > > > > > > implementations. >> > > > > > > >> > > > > > > For those that wish to perform measurements which may >> > > > > > > require bidirectional flow information (perhaps because >> > > > > > > start/end >> > > > > timestamp >> > > > > > > synchronization is required for flow pairs), the >> > > IPFIX data model >> > > > > > > and transport could define a mechanism by which the >> > > > > exporter could >> > > > > > > inform the collector about which unidirectional flow >> > > records are >> > > > > > > matched pairs. That is, the exporter could help the >> > > > > collector find >> > > > > > > the matched pairs of flow information, perhaps by >> > > > > exporting the flow >> > > > > > > records in pairs or by generating a numeric >> > > identifier that is an >> > > > > > > attribute of both flow records. Of course only >> > > > > > > bidirectional collectors would understand this >> convention, >> > > > > > > but a unidirection collector could ignore it and >> still make >> > > > > > > use of the exported flow information from an >> exporter that >> > > > > > > implemented this "bidirectional flow pairing" feature. >> > > > > > > >> > > > > > > Dave >> > > > > > > >> > > > > > > -- >> > > > > > > ":) You will make many changes before settling >> > > satisfactorily. :)" >> > > > > > > - the actual message (complete with smiley faces) that I >> > > > > received >> > > > > > > in a >> > > > > > > fortune cookie at dinner after the IPFIX WG meeting >> > > > > during IETF 52 >> > > > > > > in Salt Lake City >> > > > > > > >> > > > > > > -- >> > > > > > > Help mailto:majordomo@net.doit.wisc.edu >> and say "help" >> > > > > > > in message body >> > > > > > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say >> > > > > "unsubscribe >> > > > > > > ipfix" in message body >> > > > > > > Archive http://ipfix.doit.wisc.edu/archive/ >> > > > > > > >> > > > > > > >> > > > > > >> > > > > > >> > > > > > >> > > > > > -- >> > > > > > Help mailto:majordomo@net.doit.wisc.edu and say >> > > > > "help" in message body >> > > > > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say >> > > "unsubscribe >> > > > > > ipfix" in message body >> > > > > > Archive http://ipfix.doit.wisc.edu/archive/ >> > > > > > >> > > > > >> > > > > >> > > > > >> > > > >> > > > -- >> > > > Help mailto:majordomo@net.doit.wisc.edu and say >> > > "help" in message body >> > > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say >> > > > "unsubscribe ipfix" in message body >> > > > Archive http://ipfix.doit.wisc.edu/archive/ >> > > >> > > -- >> > > Help mailto:majordomo@net.doit.wisc.edu and say "help" >> > > in message body >> > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say >> "unsubscribe >> > > ipfix" in message body >> > > Archive http://ipfix.doit.wisc.edu/archive/ >> > > >> > > >> >> > > > > -- > Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ > -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Wed Jan 9 11:06:23 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA06165 for ; Wed, 9 Jan 2002 11:06:22 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16OL14-0003Uj-00 for ipfix-list@mil.doit.wisc.edu; Wed, 09 Jan 2002 09:51:50 -0600 Received: from dplonka by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16OL13-0003Ud-00 for ipfix-req@net.doit.wisc.edu; Wed, 09 Jan 2002 09:51:49 -0600 Date: Wed, 9 Jan 2002 09:51:48 -0600 From: Dave Plonka To: ipfix-req@net.doit.wisc.edu Subject: Re: [ipfix-req] unidirection vs. bidirectional flows Message-ID: <20020109095148.A9422@doit.wisc.edu> Reply-To: ipfix-req@net.doit.wisc.edu References: <5C8959A16A71B449AE793CF52FBBED66018FD4@ptah.newyork.qosient.com> <1275220606.1010592535@[192.168.102.164]> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 1.0.1i In-Reply-To: <1275220606.1010592535@[192.168.102.164]>; from quittek@ccrle.nec.de on Wed, Jan 09, 2002 at 04:08:55PM +0100 Organization: UW-Madison, DoIT, Network Services X-VMS-Error: %SYSTEM-F-INTOVF, arithmetic trap, integer overflow at PC=00000000, PS=0000047A X-Shakespearean-Insult: Thou errant earth-vexing pignut Precedence: bulk Sender: majordomo listserver On Wed, Jan 09, 2002 at 04:08:55PM +0100, Juergen Quittek wrote: > --On 08 January 2002 17:12 -0500 Carter Bullard wrote: > > but you need to have something that states > > that these records are related. You won't be able to infer > > this from the 5-tuple flow descriptor in either record, > > so you'll need some type of binding identifier. > > Exactly. Now who's work should it be to find the binding? > The exporter? Why not the application analyzing the data > at the collector. It is much better suited for this job and > can do it online or offline depending of capabilities and > requirements for this analysis. But I would not ask the > exporter to do this job. (But what if the exporter _volunteers_ to do that job? ;^) That binding is what I called "flow matching" in an earlier post: http://ipfix.doit.wisc.edu/archive/0356.html I take it that Carter's point is that some exporters already know that binding (before they export) - i.e. it will know which flows match. This is what I was trying to accommodate, in my first posting: http://ipfix.doit.wisc.edu/archive/0330.html in this thread when I said: [...] the IPFIX data model and transport could define a mechanism by which the exporter could inform the collector about which unidirectional flow records are matched pairs. and Paul reiterated here: http://ipfix.doit.wisc.edu/archive/0349.html I've been giving this some more thought, and would like to have some email-friendly format for us to run through examples. How about this: I'll show individual unidirectional flows as a single line in pseudo-syntax sort of like that produced by the argus ra(1) command or my "flowdumper -s" utility with NetFlow or LFAP, i.e.: +----------------------------------------------------------------------+ | FLOW: src_addr:src_port -> dst_addr:dstport PROTO ... | +----------------------------------------------------------------------+ Consider this example of a SSH session: Current Unidirectional exporters will report it like this (essentially two records, minimum): +----------------------------------------------------------------------+ | FLOW: 10.42.69.10:1024 -> 10.10.10.69:22 TCP | +----------------------------------------------------------------------+ | FLOW: 10.10.10.69:22 -> 10.42.69.10:1024 TCP | +----------------------------------------------------------------------+ If the data model and architecture is sufficiently feature-rich, a bidirectional exporter could export those flows, perhaps, as two flow records wrapped as one "Flow Information Record" or whatever you'd like to call it. E.g.: +----------------------------------------------------------------------+ | FIR: (unicast, "normal" TCP session) | |+--------------------------------------------------------------------+| ||FLOW: 10.42.69.10:1024 -> 10.10.10.69:22 TCP || |+--------------------------------------------------------------------+| ||FLOW: 10.10.10.69:22 -> 10.42.69.10:1024 TCP || |+--------------------------------------------------------------------+| +----------------------------------------------------------------------+ Note that the two unidirectional flow records are wrapped in what I called a FIR, or "Flow Information Record". (This ASCII art is just meant to be a logical representation, I'm not suggestion a typedef struct, TLV, or anything specific.) The advantage of this would be that the embedded flow records could be understood by either an unidirectional or bidirectionally aware collecter. The encapsulating "FIR" (that binds the two together) could essentially turn into a "no-op" for collectors that don't need to understand any application-level relationship between those flows, for the given application. I've worked through other examples for ping, traceroute, multicast w/multiple egress ifIndex numbers, etc. but I'll hold off until I see if this is understandable. All in all, I like the clarity of unidirectional flow definition, and also only requiring unidirectional support as the common-denominator, but I do agree that and the adopting of a unidirectional flow definition (if we do so) should not make more work for the collector when the exporter could pass that info along. Dave -- plonka@doit.wisc.edu http://net.doit.wisc.edu/~plonka ARS:N9HZF Madison, WI -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Wed Jan 9 12:01:15 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA07318 for ; Wed, 9 Jan 2002 12:01:15 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16OLoh-0004eE-00 for ipfix-list@mil.doit.wisc.edu; Wed, 09 Jan 2002 10:43:07 -0600 Received: from mailhub.fokus.gmd.de ([193.174.154.14]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 16OLod-0004dU-00 for ipfix-req@net.doit.wisc.edu; Wed, 09 Jan 2002 10:43:03 -0600 Received: from fokus.fhg.de (dhcp187 [195.37.78.187]) by mailhub.fokus.gmd.de (8.11.6/8.11.6) with ESMTP id g09Ggvh12483; Wed, 9 Jan 2002 17:42:57 +0100 (MET) Message-ID: <3C3C7245.3A9926C0@fokus.fhg.de> Date: Wed, 09 Jan 2002 17:39:33 +0100 From: Sebastian Zander Organization: FhI FOKUS X-Mailer: Mozilla 4.75 [de] (Windows NT 5.0; U) X-Accept-Language: de MIME-Version: 1.0 To: plonka@doit.wisc.edu CC: ipfix-req@net.doit.wisc.edu Subject: Re: [ipfix-req] suggestion for requirements doc abstract References: <20020107151835.A19904@doit.wisc.edu> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Hi Dave, Dave Plonka schrieb: > > IPFIX folks, > > The term "passive measurement" is commonly used by the measurement > community and I have been thinking about how to include it in our > requirements document to help clarify which systems are bound to these > requirements. > > I suggest we reword the requirements abstract (see > "http://www.ietf.org/internet-drafts/draft-ietf-ipfix-reqs-00.txt") > to this: > > Abstract > > This memo defines requirements for network elements that export > Internet Protocol flow information within a passive measurement > framework. These network elements are devices such as routers, > dedicated traffic measurement probes, and middleboxes. > I find your wording confusing as it sounds that IPFIX is limited to be used in a passive measurement framework. Obviously IPFIX boxes do passive measurement but I could use them in an active measurement framework as well combining some sort of traffic generators with IPFIX probes and coordinating both. I guess your intention was to say that IPFIX export information is collected by passive measurement so what about "...network elements that export Internet Protocol flow information collected by passive measurement." ? Cheers, Sebastian -- Sebastian Zander E-mail: zander@fokus.fhg.de FhI FOKUS / Global Networking (GloNe) Tel: +49-30-3463-7287 Kaiserin-Augusta-Allee 31 Fax: +49-30-3463-8287 D-10589 Berlin, Germany www.fokus.fhg.de/usr/sebastian.zander -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Wed Jan 9 12:34:46 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA07860 for ; Wed, 9 Jan 2002 12:34:46 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16OMDd-0005M6-00 for ipfix-list@mil.doit.wisc.edu; Wed, 09 Jan 2002 11:08:53 -0600 Received: from mailhub.fokus.gmd.de ([193.174.154.14]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 16OMDb-0005M1-00 for ipfix-req@net.doit.wisc.edu; Wed, 09 Jan 2002 11:08:51 -0600 Received: from fokus.fhg.de (dhcp187 [195.37.78.187]) by mailhub.fokus.gmd.de (8.11.6/8.11.6) with ESMTP id g09H8oh18017 for ; Wed, 9 Jan 2002 18:08:50 +0100 (MET) Message-ID: <3C3C7855.3EA1620D@fokus.fhg.de> Date: Wed, 09 Jan 2002 18:05:25 +0100 From: Sebastian Zander Organization: FhI FOKUS X-Mailer: Mozilla 4.75 [de] (Windows NT 5.0; U) X-Accept-Language: de MIME-Version: 1.0 To: ipfix-req@net.doit.wisc.edu Subject: Re: [ipfix-req] unidirection vs. bidirectional flows References: <5C8959A16A71B449AE793CF52FBBED66018FD4@ptah.newyork.qosient.com> <1275220606.1010592535@[192.168.102.164]> <20020109095148.A9422@doit.wisc.edu> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit I am trying to summarize the discussion from the requirements viewpoint: Do we need to add a requirement on the flow model to the requirements draft? If yes then looking at the discussion it seems that there is rough consensus on: MUST support uni-directional, additionally MAY/SHOULD(?) support bi-directional if the exporter supports flow matching. Correct? Cheers, Sebastian -- Sebastian Zander E-mail: zander@fokus.fhg.de FhI FOKUS / Global Networking (GloNe) Tel: +49-30-3463-7287 Kaiserin-Augusta-Allee 31 Fax: +49-30-3463-8287 D-10589 Berlin, Germany www.fokus.fhg.de/usr/sebastian.zander -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Wed Jan 9 13:56:46 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA11397 for ; Wed, 9 Jan 2002 13:56:46 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16ONSP-00074C-00 for ipfix-list@mil.doit.wisc.edu; Wed, 09 Jan 2002 12:28:13 -0600 Received: from relay1.pair.com ([209.68.1.20] helo=relay.pair.com) by mil.doit.wisc.edu with smtp (Exim 3.13 #1) id 16ONSM-000743-00 for ipfix-req@net.doit.wisc.edu; Wed, 09 Jan 2002 12:28:10 -0600 Received: (qmail 11790 invoked from network); 9 Jan 2002 18:28:07 -0000 Received: from pool-162-83-252-213.ny5030.east.verizon.net (HELO sphynx) (162.83.252.213) by relay1.pair.com with SMTP; 9 Jan 2002 18:28:07 -0000 X-pair-Authenticated: 162.83.252.213 Reply-To: From: "Carter Bullard" To: "'Juergen Quittek'" , Cc: Subject: RE: [ipfix-req] unidirection vs. bidirectional flows Date: Wed, 9 Jan 2002 13:27:57 -0500 Organization: QoSient, LLC Message-ID: <5C8959A16A71B449AE793CF52FBBED66018FD9@ptah.newyork.qosient.com> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook, Build 10.0.3416 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 In-Reply-To: <1275220606.1010592535@[192.168.102.164]> Importance: Normal Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Hey Juergen, Hmmmm, if you have such flexible support, why not support adding the return metrics as optional data? Why generate all this work? Carter Carter Bullard QoSient, LLC 300 E. 56th Street, Suite 18K New York, New York 10022 carter@qosient.com Phone +1 212 588-9133 Fax +1 212 588-9134 http://qosient.com > -----Original Message----- > From: Juergen Quittek [mailto:quittek@ccrle.nec.de] > Sent: Wednesday, January 09, 2002 10:09 AM > To: carter@qosient.com; calato@riverstonenet.com > Cc: ipfix-req@net.doit.wisc.edu > Subject: RE: [ipfix-req] unidirection vs. bidirectional flows > > > --On 08 January 2002 17:12 -0500 Carter Bullard > wrote: > > > Hey Paul, > > The RTT metric in the example is Duration. If your > uni-directional > > flow record's have start and stop times, then you could derive the > > Duration by subtracting the start times of one record from the stop > > times of the other record, but you need to have something > that states > > that these records are related. You won't be able to infer > > this from the 5-tuple flow descriptor in either record, > > so you'll need some type of binding identifier. > > Exactly. Now who's work should it be to find the binding? > The exporter? Why not the application analyzing the data > at the collector. It is much better suited for this job and > can do it online or offline depending of capabilities and > requirements for this analysis. But I would not ask the > exporter to do this job. > > > My problem with this is that you get 2 complete records for a > > single network event. They include redundant headers, > redundant flow > > descriptors, two unused time-stamps, and we introduce new > requirements > > to guarantee that the binding identifiers be unique. The added > > bandwidth requirements for transmitting the redundant > information can > > get pretty big in high end monitors. A 5 tuple flow descriptor for > > IPv4 addresses is 16 bytes. For IPv6 its 38 bytes. The > > redundant timestamps are 16 bytes, and an IPFIX record > > header will probably be around 16-32 bytes. So the > > redundant information is around 64-128 bytes. That's pretty > > significant in my book. > > You are right. This dumb approach wastes resources. But > that's why we require flexible flow records for IPFIX. If > this above is your only measurement, then reduce the flow > record to just two tomestamps and a rule (set) identifier and > you waste much less bandwidth. > > Juergen > -- > Juergen Quittek quittek@ccrle.nec.de Tel: +49 6221 90511-15 > NEC Europe Ltd., Network Laboratories Fax: +49 6221 90511-55 > Adenauerplatz 6, 69115 Heidelberg, Germany http://www.ccrle.nec.de > > > > The XML schema for the data that Argus generates is at > > http://qosient.com/argus/Xml/ArgusRecord_xsd/ArgusRecord.htm > > Argus is a bi-directional flow monitor that generates very flexible > > flow records. Lots of optional fields. Now I am not recommending > > Argus's data strategy for IPFIX, I'm just pointing out that > Argus like > > schemas can handle uni-directional and bi-directional > reports at the > > same time, because it supports optional fields. > > > > > > Carter > > > > Carter Bullard > > QoSient, LLC > > 300 E. 56th Street, Suite 18K > > New York, New York 10022 > > > > carter@qosient.com > > Phone +1 212 588-9133 > > Fax +1 212 588-9134 > > http://qosient.com > > > > > > > > > >> -----Original Message----- > >> From: calato [mailto:calato] On Behalf Of calato@riverstonenet.com > >> Sent: Tuesday, January 08, 2002 11:51 AM > >> To: carter@qosient.com > >> Cc: ipfix-req@net.doit.wisc.edu > >> Subject: Re: [ipfix-req] unidirection vs. bidirectional flows > >> > >> > >> > >> I'm not sure. Can you tell me the fields reported today? > I'll see if > >> I can make sense of it as 2 uni-directional flows. > >> > >> Paul > >> > >> Carter Bullard wrote: > >> > > >> > Hey Paul, > >> > So I have a flow monitor that generates RTT times for DNS > >> > transactions. How do I report RTT for a DNS transaction > >> using IPFIX > >> > if it has the restriction that I must report my metrics using a > >> > uni-directional semantic? > >> > > >> > So how do I do it? Break the records in two, and > somehow tag that > >> > the two records are unambiguously related, and where do > I put the > >> > bi-directional semantic? In both of the records, in the > first one, > >> > the last one, in a separate third record? > >> > > >> > Doesn't seem very efficient. > >> > > >> > Carter > >> > > >> > Carter Bullard > >> > QoSient, LLC > >> > 300 E. 56th Street, Suite 18K > >> > New York, New York 10022 > >> > > >> > carter@qosient.com > >> > Phone +1 212 588-9133 > >> > Fax +1 212 588-9134 > >> > http://qosient.com > >> > > >> > > -----Original Message----- > >> > > From: majordomo listserver > >> [mailto:majordomo@mil.doit.wisc.edu] On > >> > > Behalf Of calato@riverstonenet.com > >> > > Sent: Tuesday, January 08, 2002 11:30 AM > >> > > To: ipfix-req@net.doit.wisc.edu > >> > > Subject: Re: [ipfix-req] unidirection vs. bidirectional flows > >> > > > >> > > > >> > > > >> > > Carter, > >> > > > >> > > I'm a bit confused. Why doesn't the ability to provide > >> > > bi-directional info, as Dave described, meet your needs? Those > >> > > devices that can do bi-directional will report it that > way, those > >> > > that can't will report it as uni-directional. Some down stream > >> > > application will have to do the work to turn 2 > >> uni-directional flows > >> > > into one bi-directional one, if that is what's needed. And if > >> > > uni-directional is needed the app will split it apart. > >> > > > >> > > But if you require bi-directional reporting, many > devices wont be > >> > > able to do it. Which then eliminates any down stream analysis. > >> > > > >> > > Paul > >> > > > >> > > > >> > > Carter Bullard wrote: > >> > > > > >> > > > Hey Vamsi, > >> > > > Hmmm, is the interface a part of the flow > defintion? Not a > >> > > > problem. Lets look at how this may work. Suppose > >> there is a SRC > >> > > > interface and a DST interface, so that a flow is > simply all the > >> > > > packets moving from SRC -> DST. The return flow would > >> be all the > >> > > > packets moving from DST -> SRC. > >> > > > > >> > > > So the flow is reported as SRC <-> DST, with the simple > >> > > > addition of the DST -> SRC metrics. Makes more sense than > >> > > > independantly reporting two half-duplex flow records, > >> which could > >> > > > be skewed in time so badly, that the two records cannot be > >> > > > correlated at all. > >> > > > > >> > > > Carter > >> > > > > >> > > > Carter Bullard > >> > > > QoSient, LLC > >> > > > 300 E. 56th Street, Suite 18K > >> > > > New York, New York 10022 > >> > > > > >> > > > carter@qosient.com > >> > > > Phone +1 212 588-9133 > >> > > > Fax +1 212 588-9134 > >> > > > http://qosient.com > >> > > > > >> > > > > -----Original Message----- > >> > > > > From: Vamsidhar Valluri [mailto:vvalluri@cisco.com] > >> > > > > Sent: Monday, January 07, 2002 6:58 PM > >> > > > > To: Carter Bullard > >> > > > > Cc: plonka@doit.wisc.edu; ipfix-req@net.doit.wisc.edu > >> > > > > Subject: RE: [ipfix-req] unidirection vs. > bidirectional flows > >> > > > > > >> > > > > > >> > > > > > >> > > > > > >> > > > > On Mon, 7 Jan 2002, Carter Bullard wrote: > >> > > > > > >> > > > > > Gentle people, > >> > > > > > There is absolutely no reason for IPFIX to limit > >> the type of > >> > > > > > flow data that it supports. The difference between > >> > > > > uni-directional > >> > > > > > and bi-directional flow records is simply the type of > >> > > > > metrics that are > >> > > > > > reported. The flow descriptors and keys are all the > >> > > same. Because > >> > > > > > IPFIX must support > >> > > > > ^^^^^ > >> > > > > > >> > > > > If "keys" you are refering to are the fields used to > >> identify a > >> > > > > flow then "SRC interface" do not fit into the > statement "The > >> > > > > flow descriptors and keys are all same for both > >> unidirectional > >> > > > > and bi-directional flow records" mentioned above > because SRC > >> > > > > interface will make it unidirectional. > >> > > > > > >> > > > > -vamsi > >> > > > > > >> > > > > > >> > > > > > reporting metrics that each vendors will not implement, > >> > > > > > such > >> > > > > as jitter > >> > > > > > and one-way delay, which are the few metrics that are > >> > > meaningful in > >> > > > > > uni-directional flow data models, supporting metrics > >> > > that report on > >> > > > > > return traffic should not be a hardship for IPFIX. > >> > > > > > > >> > > > > > But, I believe that if you are going to make a design > >> > > > > choice it should > >> > > > > > be motivated by design goals. What benefit do you gain > >> > > by limiting > >> > > > > > IPFIX to a simple uni-directional > >> > > > > > flow model? Is a uni-directional flow data > model the best > >> > > > > > data model for reporting all the metrics and network > >> > > behaviors that > >> > > > > > relate to network flows? Is it the best data model for > >> > > > > supporting all > >> > > > > > the existing commercial network flow data? > >> > > > > > > >> > > > > > But what about even the simplest of applications of > >> > > flow data? Is a > >> > > > > > uni-directional flow data model the best solution for > >> > > reporting the > >> > > > > > RTT of a ping volley? The empirical bulk transfer > >> > > > > properties of a TCP > >> > > > > > connection? How about TCP packet > retransmission? Existence > >> > > > > of routing > >> > > > > > failures in an OSPF routed network? Application > >> response time? > >> > > > > > Rerouting indications? Reachability fault detection? > >> > > > > Access policy > >> > > > > > validation? Connectivity status? > >> > > > > > > >> > > > > > Even FTP supports both ASCII and binary data transfer. > >> > > > > Handling both > >> > > > > > uni-directional data and bi-directional data in IPFIX > >> > > > > should be pretty > >> > > > > > easy stuff. > >> > > > > > > >> > > > > > > >> > > > > > Carter > >> > > > > > > >> > > > > > Carter Bullard > >> > > > > > QoSient, LLC > >> > > > > > 300 E. 56th Street, Suite 18K > >> > > > > > New York, New York 10022 > >> > > > > > > >> > > > > > carter@qosient.com > >> > > > > > Phone +1 212 588-9133 > >> > > > > > Fax +1 212 588-9134 > >> > > > > > http://qosient.com > >> > > > > > > >> > > > > > > >> > > > > > > >> > > > > > > >> > > > > > > -----Original Message----- > >> > > > > > > From: majordomo listserver > >> > > > > [mailto:majordomo@mil.doit.wisc.edu] On > >> > > > > > > Behalf Of Dave Plonka > >> > > > > > > Sent: Monday, January 07, 2002 4:52 PM > >> > > > > > > To: ipfix-req@net.doit.wisc.edu > >> > > > > > > Subject: [ipfix-req] unidirection vs. > bidirectional flows > >> > > > > > > > >> > > > > > > > >> > > > > > > > >> > > > > > > IPFIX folks, > >> > > > > > > > >> > > > > > > As many of you recall, whether or not IPFIX > >> should specify a > >> > > > > > > unidirectional or bidirectional flow > definition was a hot > >> > > > > topic at > >> > > > > > > the WG meeting. > >> > > > > > > > >> > > > > > > I think that specifying *something* gets us > far along in > >> > > > > > > communicating what a "flow" is. Personally, I would > >> > > like us to > >> > > > > > > specify one or the other and I am in favor a > unidirection > >> > > > > > > flow definition. > >> > > > > > > > >> > > > > > > Here are some thoughts about arguments made for > >> > > > > bidirectional flows: > >> > > > > > > > >> > > > > > > A unidirectional flow definition means that the atomic > >> > > > > flow report > >> > > > > > > record is unidirection during the > >> reporting/export phase. > >> > > > > > > For instance, a single packet counter for a given > >> > > > > > > protocol/srcaddr/srcport/dstaddr/dstport n-tuple. > >> > > However, since > >> > > > > > > the exporter is somewhat of a "black box", it > need not be > >> > > > > > > constrained to a unidirectional model internally. > >> > > > > > > > >> > > > > > > A unidirectional flow definition does not prevent a > >> > > > > > > collector implementor from using a bi-directional > >> notion of > >> > > > > > > flows > >> > > > > internally > >> > > > > > > nor does it prevent it from being stored in a > >> > > bidirectional way > >> > > > > > > prior to export. Conversely if we specify a > >> bidirectional > >> > > > > > > definition, it does disqualify unidirectional > >> > > > > > > implementations. > >> > > > > > > > >> > > > > > > For those that wish to perform measurements which may > >> > > > > > > require bidirectional flow information > (perhaps because > >> > > > > > > start/end > >> > > > > timestamp > >> > > > > > > synchronization is required for flow pairs), the > >> > > IPFIX data model > >> > > > > > > and transport could define a mechanism by which the > >> > > > > exporter could > >> > > > > > > inform the collector about which unidirectional flow > >> > > records are > >> > > > > > > matched pairs. That is, the exporter could help the > >> > > > > collector find > >> > > > > > > the matched pairs of flow information, perhaps by > >> > > > > exporting the flow > >> > > > > > > records in pairs or by generating a numeric > >> > > identifier that is an > >> > > > > > > attribute of both flow records. Of course only > >> > > > > > > bidirectional collectors would understand this > >> convention, > >> > > > > > > but a unidirection collector could ignore it and > >> still make > >> > > > > > > use of the exported flow information from an > >> exporter that > >> > > > > > > implemented this "bidirectional flow pairing" feature. > >> > > > > > > > >> > > > > > > Dave > >> > > > > > > > >> > > > > > > -- > >> > > > > > > ":) You will make many changes before settling > >> > > satisfactorily. :)" > >> > > > > > > - the actual message (complete with smiley > faces) that I > >> > > > > received > >> > > > > > > in a > >> > > > > > > fortune cookie at dinner after the IPFIX WG meeting > >> > > > > during IETF 52 > >> > > > > > > in Salt Lake City > >> > > > > > > > >> > > > > > > -- > >> > > > > > > Help mailto:majordomo@net.doit.wisc.edu > >> and say "help" > >> > > > > > > in message body > >> > > > > > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > >> > > > > "unsubscribe > >> > > > > > > ipfix" in message body > >> > > > > > > Archive http://ipfix.doit.wisc.edu/archive/ > >> > > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > > >> > > > > > > >> > > > > > -- > >> > > > > > Help mailto:majordomo@net.doit.wisc.edu and say > >> > > > > "help" in message body > >> > > > > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > >> > > "unsubscribe > >> > > > > > ipfix" in message body > >> > > > > > Archive http://ipfix.doit.wisc.edu/archive/ > >> > > > > > > >> > > > > > >> > > > > > >> > > > > > >> > > > > >> > > > -- > >> > > > Help mailto:majordomo@net.doit.wisc.edu and say > >> > > "help" in message body > >> > > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > >> > > > "unsubscribe ipfix" in message body > >> > > > Archive http://ipfix.doit.wisc.edu/archive/ > >> > > > >> > > -- > >> > > Help mailto:majordomo@net.doit.wisc.edu and say "help" > >> > > in message body > >> > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > >> "unsubscribe > >> > > ipfix" in message body > >> > > Archive http://ipfix.doit.wisc.edu/archive/ > >> > > > >> > > > >> > >> > > > > > > > > -- > > Help mailto:majordomo@net.doit.wisc.edu and say > "help" in message body > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe > > ipfix" in message body > > Archive http://ipfix.doit.wisc.edu/archive/ > > > > > > -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Wed Jan 9 17:36:28 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA18378 for ; Wed, 9 Jan 2002 17:36:28 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16OQtD-00045Y-00 for ipfix-list@mil.doit.wisc.edu; Wed, 09 Jan 2002 16:08:08 -0600 Received: from mercury.sun.com ([192.9.25.1]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 16OQtB-00045R-00 for ipfix-req@net.doit.wisc.edu; Wed, 09 Jan 2002 16:08:05 -0600 Received: from sunmail1.Sun.COM ([129.145.1.2]) by mercury.Sun.COM (8.9.3+Sun/8.9.3) with ESMTP id OAA24439 for ; Wed, 9 Jan 2002 14:08:03 -0800 (PST) Received: from jurassic.eng.sun.com (jurassic.Eng.Sun.COM [129.146.85.31]) by sunmail1.Sun.COM (8.9.3+Sun/8.9.3/ENSMAIL,v2.1p1-Sun.COM.mod.2) with ESMTP id OAA26911 for ; Wed, 9 Jan 2002 14:09:08 -0800 (PST) Received: from sun.com (inox.Eng.Sun.COM [129.146.85.133]) by jurassic.eng.sun.com (8.12.2.Beta4+Sun/8.12.2.Beta4) with ESMTP id g09M81Ul264127 for ; Wed, 9 Jan 2002 14:08:01 -0800 (PST) Message-ID: <3C3CBF47.92AA9926@sun.com> Date: Wed, 09 Jan 2002 14:08:07 -0800 From: Jc Martin Organization: Sun Microsystems X-Mailer: Mozilla 4.78 [en] (X11; U; SunOS 5.9 sun4u) X-Accept-Language: en MIME-Version: 1.0 To: ipfix-req@net.doit.wisc.edu Subject: Re: [ipfix-req] unidirection vs. bidirectional flows References: <5C8959A16A71B449AE793CF52FBBED66018FD4@ptah.newyork.qosient.com> <1275220606.1010592535@[192.168.102.164]> <20020109095148.A9422@doit.wisc.edu> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Dave [snip] > > If the data model and architecture is sufficiently feature-rich, a > bidirectional exporter could export those flows, perhaps, as two flow > records wrapped as one "Flow Information Record" or whatever you'd like > to call it. E.g.: > > +----------------------------------------------------------------------+ > | FIR: (unicast, "normal" TCP session) | > |+--------------------------------------------------------------------+| > ||FLOW: 10.42.69.10:1024 -> 10.10.10.69:22 TCP || > |+--------------------------------------------------------------------+| > ||FLOW: 10.10.10.69:22 -> 10.42.69.10:1024 TCP || > |+--------------------------------------------------------------------+| > +----------------------------------------------------------------------+ > > Note that the two unidirectional flow records are wrapped in what I > called a FIR, or "Flow Information Record". > > (This ASCII art is just meant to be a logical representation, I'm not > suggestion a typedef struct, TLV, or anything specific.) > This suggestion could be seen has adding application level semantic to the flow exported. In some ways, this seems to extend the grouping that a collector can do based on exported attributes (like the 5 tuple), with information that is known only by the exporter. I think that this is valuable, but should not be restricted in grouping together flows that are "pairs", but could be used to group flows that are part of a transaction, or "dialogue" involving multiple flows. JC -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Wed Jan 9 17:46:55 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA18970 for ; Wed, 9 Jan 2002 17:46:55 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16ORAh-0004Su-00 for ipfix-list@mil.doit.wisc.edu; Wed, 09 Jan 2002 16:26:11 -0600 Received: from sj-msg-core-1.cisco.com ([171.71.163.11]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 16ORAf-0004Ry-00 for ipfix-arch@net.doit.wisc.edu; Wed, 09 Jan 2002 16:26:09 -0600 Received: from mira-sjcd-1.cisco.com (mira-sjcd-1.cisco.com [171.69.43.44]) by sj-msg-core-1.cisco.com (8.11.3/8.9.1) with ESMTP id g09MPcF02803; Wed, 9 Jan 2002 14:25:38 -0800 (PST) Received: from cisco.com (dhcp-171-71-137-45.cisco.com [171.71.137.45]) by mira-sjcd-1.cisco.com (Mirapoint) with ESMTP id AAU61927; Wed, 9 Jan 2002 14:25:44 -0800 (PST) Message-ID: <3C3CC55D.DF16DE29@cisco.com> Date: Wed, 09 Jan 2002 14:34:05 -0800 From: Ganesh Sadasivan X-Mailer: Mozilla 4.76 [en]C-CCK-MCD (Windows NT 5.0; U) X-Accept-Language: en MIME-Version: 1.0 To: calato@riverstonenet.com CC: Reinaldo Penno , arch Subject: Re: [ipfix-arch] configuration information. References: <4F973E944951D311B6660008C7917CF0083E088C@zsc4c008.us.nortel.com> <3C3B2FB6.3F1F0343@riverstonenet.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit calato@riverstonenet.com wrote: > > > > > > If you have very specific templates such as srcIP, > > > destIP, etc, it's > > > > > not to hard find records in common. But you can have things > > like: > > > > > > > > > > src_AnyIP Dest_Yahoo.com Proto_Any and > > > > > src_AnyIP Dest_AS3462 Proto_HTTP. > > > > > > I assume here you mean that there are 2 counters and > > > 2 "flows" reported. One has (to use cisco terms) a template > > > of "dest-ip, byes, packets" and the other "dest-AS, proto, > > > byes, packets". > > > > > > If the device uses a multi-match method then the device > > > "knows" that a packet may be added to both buckets. It > > > should tell the collector that these templates overlap. > > > > I agree. But it should say that not only these templates overlap but > > what 5-tuple? is overlapping. (also see below) > > > > > If on the other hand the device uses a first match method > > > then the counts do not overlap. > > > > Should we assume that you can have different templates per > > Card/Interface if the cards/interfaces have intelligence? In this case > > they could overlap. > > Again, the device "knows" how it is counting. If the > devices is able to track packets based on multiple > keys then some thought needs to be given to whether > or not those key sets overlap. For example, if one key > set is "in-interface, src-ip, dst-ip" and the other is > "in-interface, src-AS, dst-AS" then the associated > counters do not overlap. If the application wants > a total for src-AS1 then it can add the bytes from > first ket set to the second. > > Overlaping counters are not something determined > after the fact based on data seen, it is determined > ahead of time for an entire class (or key set) of > records. > This is really not true. In the above example there are multiple ways of measuring: 1. All packets are tracked based on both the key sets. There is 100% duplication. 2. Packets are tracked based on the first matching key set. A single src-ip can have multiple src-ASs. Assuming the ASs for src-IP1 are src-AS1 and src-AS2, and say only src-AS1 is setup to match and the keyset {in-interface, src-AS, dst-AS} gets matched first, part of the packets that come on src-IP1 gets matched to the first key set and the other part gets matched to the {in-interface, src-ip, dst-ip} key set. The collector can get the src-ASs per src-IP information through an external BGP daemon or some other means. Even if the collector has knowledge of how many origin ASs are there for src-IP1, it still cannot find out how many packets passed through src-IP1. This is a fairly simple example. Looks to me that the complexity on the collector side so that it can automatically infer the relationships between key sets is quite high. Thanks Ganesh -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Wed Jan 9 19:03:32 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id TAA21366 for ; Wed, 9 Jan 2002 19:03:32 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16OS9a-0005nR-00 for ipfix-list@mil.doit.wisc.edu; Wed, 09 Jan 2002 17:29:06 -0600 Received: from dplonka by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16OS9Y-0005nK-00 for ipfix-req@net.doit.wisc.edu; Wed, 09 Jan 2002 17:29:04 -0600 Date: Wed, 9 Jan 2002 17:29:04 -0600 From: Dave Plonka To: ipfix-req@net.doit.wisc.edu Subject: [ipfix-req] common attributes amonst uni-dir flows (was "Re: unidirection vs. bidirectional flows") Message-ID: <20020109172904.E3019@doit.wisc.edu> Reply-To: ipfix-req@net.doit.wisc.edu References: <5C8959A16A71B449AE793CF52FBBED66018FD4@ptah.newyork.qosient.com> <1275220606.1010592535@[192.168.102.164]> <20020109095148.A9422@doit.wisc.edu> <3C3CBF47.92AA9926@sun.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 1.0.1i In-Reply-To: <3C3CBF47.92AA9926@sun.com>; from jean-christophe.martin@sun.com on Wed, Jan 09, 2002 at 02:08:07PM -0800 Organization: UW-Madison, DoIT, Network Services X-VMS-Error: %SYSTEM-W-NOPSWAPM, operation requires PSWAPM privilege X-Shakespearean-Insult: Thou mangled tickle-brained clotpole Precedence: bulk Sender: majordomo listserver On Wed, Jan 09, 2002 at 02:08:07PM -0800, Jc Martin wrote: > Dave > [snip] > > > > If the data model and architecture is sufficiently feature-rich, a > > bidirectional exporter could export those flows, perhaps, as two flow > > records wrapped as one "Flow Information Record" or whatever you'd like > > to call it. E.g.: > > > > +----------------------------------------------------------------------+ > > | FIR: (unicast, "normal" TCP session) | > > |+--------------------------------------------------------------------+| > > ||FLOW: 10.42.69.10:1024 -> 10.10.10.69:22 TCP || > > |+--------------------------------------------------------------------+| > > ||FLOW: 10.10.10.69:22 -> 10.42.69.10:1024 TCP || > > |+--------------------------------------------------------------------+| > > +----------------------------------------------------------------------+ > > > > Note that the two unidirectional flow records are wrapped in what I > > called a FIR, or "Flow Information Record". > > > > (This ASCII art is just meant to be a logical representation, I'm not > > suggestion a typedef struct, TLV, or anything specific.) > > > > This suggestion could be seen has adding application level semantic to > the flow exported. > In some ways, this seems to extend the grouping that a collector can do > based on exported attributes (like the 5 tuple), with information that > is known only by the exporter. > I think that this is valuable, but should not be restricted in grouping > together flows that are "pairs", but could be used to group flows that > are part of a transaction, or "dialogue" involving multiple flows. I agree. And now I feel we might be getting somewhere, at least in discussing some things that hadn't occurred to me before. As you say perhaps more than two flows share a common attribute. For instance, a sequence of 20 packets (about half in each direction) that were the result of running traceroute(1) could be grouped together by a very intelligent exporter. Back to "bidirectionality". That two flows were part of the same "session" is but one _common_attribute_ that a given set of flows might share, and even there are even multiple kinds (or confidence levels) of bidirectionality: 1) "simple" 5-tuple match 2) state network/transport-level match 3) application-level "sane transaction" match Using simple 5-tuple rules, two unidirectional UDP flows might be mistakenly matched to each other because they happened to be DoS floods between two hosts using inverse port numbers. (e.g. foo_host:3000 -> bar_host:53, bar_host:53 -> foo_host:3000). However, a more intelligent exporter (that understood DNS question & answer payloads), might realize that these two flows were not a true match at the application level, perhaps because the packet payload consisted of all zeroes (i.e. was not a valid DNS transaction.) That bidirectionality might just be consider one of many possible common attributes amongst sets of unidirectional flows lends support to the argument for the "standard IP flow" being defined as unidirectional. Still, we don't lose any reporting capabilities for this, since IPFIX exporters MAY be able to export additional information to IPFIX collectors that are prepared to handle such information. That additional information that will convey that a set of two or more flows may share a common attribute such as "bidirectionality", a single multicast source, etc. Dave -- ":) You will make many changes before settling satisfactorily. :)" - the actual message (complete with smiley faces) that I received in a fortune cookie at dinner after the IPFIX WG meeting during IETF 52 in Salt Lake City -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Fri Jan 11 09:01:58 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id JAA15071 for ; Fri, 11 Jan 2002 09:01:58 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16P1si-0003na-00 for ipfix-list@mil.doit.wisc.edu; Fri, 11 Jan 2002 07:38:04 -0600 Received: from weird-brew.cisco.com ([144.254.15.118] helo=strange-brew.cisco.com) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 16P1se-0003mw-00 for ipfix-req@net.doit.wisc.edu; Fri, 11 Jan 2002 07:38:00 -0600 Received: from cisco.com (dhcp-peg3-vl30-144-254-7-111.cisco.com [144.254.7.111]) by strange-brew.cisco.com (8.8.8+Sun/8.8.8) with ESMTP id OAA29035; Fri, 11 Jan 2002 14:37:22 +0100 (MET) Message-ID: <3C3EEA95.62E3D592@cisco.com> Date: Fri, 11 Jan 2002 14:37:26 +0100 From: Benoit Claise X-Mailer: Mozilla 4.76 [en]C-CCK-MCD (Windows NT 5.0; U) X-Accept-Language: en MIME-Version: 1.0 To: Sebastian Zander CC: ipfix-req@net.doit.wisc.edu Subject: Re: [ipfix-req] unidirection vs. bidirectional flows References: <5C8959A16A71B449AE793CF52FBBED66018FD4@ptah.newyork.qosient.com> <1275220606.1010592535@[192.168.102.164]> <20020109095148.A9422@doit.wisc.edu> <3C3C7855.3EA1620D@fokus.fhg.de> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit All, > I am trying to summarize the discussion from the requirements viewpoint: > > Do we need to add a requirement on the flow model to the requirements draft? > > If yes then looking at the discussion it seems that there is rough consensus > on: MUST support uni-directional, additionally MAY/SHOULD(?) support > bi-directional if the exporter supports flow matching. I'm in favor of not mentioning anything about bidirectional flows and to have a flexible export mechanism (as Dave proposed). This flexible export format must be extensible so that, if Carter has got in the exporter an Identifier to link several flows, this Identifier could be sent part of the export. From http://www.ietf.org/internet-drafts/draft-ietf-ipfix-reqs-00.txt: The data model MUST be extensible for future attributes to be added. Even if a set of attributes is fixed in the flow record, the data model MUST provide a way of extending the record by configuration or for certain implementations. But actually, I'm not sure there is a big difference between having nothing specified about bidirectional and having a MAY... Regards, Benoit > > > Correct? > > Cheers, > > Sebastian > > -- > Sebastian Zander E-mail: zander@fokus.fhg.de > FhI FOKUS / Global Networking (GloNe) Tel: +49-30-3463-7287 > Kaiserin-Augusta-Allee 31 Fax: +49-30-3463-8287 > D-10589 Berlin, Germany www.fokus.fhg.de/usr/sebastian.zander > > -- > Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Fri Jan 11 11:37:13 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA20715 for ; Fri, 11 Jan 2002 11:37:13 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16P4P1-000774-00 for ipfix-list@mil.doit.wisc.edu; Fri, 11 Jan 2002 10:19:35 -0600 Received: from host3.riverstonenet.com ([63.113.148.3] helo=exc-sc1.yagosys.com) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 16P4Oz-00076C-00 for ipfix-arch@net.doit.wisc.edu; Fri, 11 Jan 2002 10:19:33 -0600 Received: from riverstonenet.com (134.141.180.91 [134.141.180.91]) by exc-sc1.yagosys.com with SMTP (Microsoft Exchange Internet Mail Service Version 5.5.2653.13) id X7WVZ8JH; Fri, 11 Jan 2002 08:18:32 -0800 Message-ID: <3C3F105D.8D980DAE@riverstonenet.com> Date: Fri, 11 Jan 2002 11:18:37 -0500 From: calato@riverstonenet.com X-Mailer: Mozilla 4.75C-CCK-MCD NSCPCD475 [en] (X11; U; SunOS 5.6 sun4u) X-Accept-Language: en MIME-Version: 1.0 To: Ganesh Sadasivan , arch Subject: Re: [ipfix-arch] configuration information. References: <4F973E944951D311B6660008C7917CF0083E088C@zsc4c008.us.nortel.com> <3C3B2FB6.3F1F0343@riverstonenet.com> <3C3CC55D.DF16DE29@cisco.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Ganesh Sadasivan wrote: > > > calato@riverstonenet.com wrote: > > > > > > > > > If you have very specific templates such as srcIP, > > > > destIP, etc, it's > > > > > > not to hard find records in common. But you can have things > > > like: > > > > > > > > > > > > src_AnyIP Dest_Yahoo.com Proto_Any and > > > > > > src_AnyIP Dest_AS3462 Proto_HTTP. > > > > > > > > I assume here you mean that there are 2 counters and > > > > 2 "flows" reported. One has (to use cisco terms) a template > > > > of "dest-ip, byes, packets" and the other "dest-AS, proto, > > > > byes, packets". > > > > > > > > If the device uses a multi-match method then the device > > > > "knows" that a packet may be added to both buckets. It > > > > should tell the collector that these templates overlap. > > > > > > I agree. But it should say that not only these templates overlap but > > > what 5-tuple? is overlapping. (also see below) > > > > > > > If on the other hand the device uses a first match method > > > > then the counts do not overlap. > > > > > > Should we assume that you can have different templates per > > > Card/Interface if the cards/interfaces have intelligence? In this case > > > they could overlap. > > > > Again, the device "knows" how it is counting. If the > > devices is able to track packets based on multiple > > keys then some thought needs to be given to whether > > or not those key sets overlap. For example, if one key > > set is "in-interface, src-ip, dst-ip" and the other is > > "in-interface, src-AS, dst-AS" then the associated > > counters do not overlap. If the application wants > > a total for src-AS1 then it can add the bytes from > > first ket set to the second. > > > > Overlaping counters are not something determined > > after the fact based on data seen, it is determined > > ahead of time for an entire class (or key set) of > > records. > > > > This is really not true. In the above example there are multiple > ways of measuring: > 1. All packets are tracked based on both the key sets. > There is 100% duplication. > 2. Packets are tracked based on the first matching key set. > A single src-ip can have multiple src-ASs. Assuming the ASs > for src-IP1 are src-AS1 and src-AS2, and say only > src-AS1 is setup to match and the keyset {in-interface, src-AS, dst-AS} > gets matched first, part of the packets that come on src-IP1 > gets matched to the first key set and the other part gets matched > to the {in-interface, src-ip, dst-ip} key set. The collector can > get the src-ASs per src-IP information through an external BGP > daemon or some other means. > Even if the collector has knowledge of how many origin ASs are there > for src-IP1, it still cannot find out how many packets passed through > src-IP1. I don't follow you here. If the src IP matches the first rule then I stop. I don't care what the AS is because I don't look further. The point is that only one counter bucket is ever updated. Thus I know I have no duplication of counts. What am I missing? > > This is a fairly simple example. Looks to me that the complexity on the > collector side so that it can automatically infer the relationships between > key sets is quite high. > > Thanks > Ganesh -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Fri Jan 11 12:14:24 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA22010 for ; Fri, 11 Jan 2002 12:14:24 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16P4wy-000023-00 for ipfix-list@mil.doit.wisc.edu; Fri, 11 Jan 2002 10:54:40 -0600 Received: from host3.riverstonenet.com ([63.113.148.3] helo=exc-sc1.yagosys.com) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 16P4wv-00001V-00 for ipfix-req@net.doit.wisc.edu; Fri, 11 Jan 2002 10:54:37 -0600 Received: from riverstonenet.com (134.141.180.91 [134.141.180.91]) by exc-sc1.yagosys.com with SMTP (Microsoft Exchange Internet Mail Service Version 5.5.2653.13) id X7WVZ9AT; Fri, 11 Jan 2002 08:53:37 -0800 Message-ID: <3C3F1895.D89D4BF9@riverstonenet.com> Date: Fri, 11 Jan 2002 11:53:41 -0500 From: calato@riverstonenet.com X-Mailer: Mozilla 4.75C-CCK-MCD NSCPCD475 [en] (X11; U; SunOS 5.6 sun4u) X-Accept-Language: en MIME-Version: 1.0 To: ipfix-req@net.doit.wisc.edu Subject: Re: [ipfix-req] unidirection vs. bidirectional flows References: <5C8959A16A71B449AE793CF52FBBED66018FD4@ptah.newyork.qosient.com> <1275220606.1010592535@[192.168.102.164]> <20020109095148.A9422@doit.wisc.edu> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit I think one of Carter's points though was efficiency. So, to use your structure, instead of reporting this... +----------------------------------------------------------------------+ | FIR: (unicast, "normal" TCP session) | |+--------------------------------------------------------------------+| ||FLOW: 10.42.69.10:1024 -> 10.10.10.69:22 TCP || |+--------------------------------------------------------------------+| ||FLOW: 10.10.10.69:22 -> 10.42.69.10:1024 TCP || |+--------------------------------------------------------------------+| +----------------------------------------------------------------------+ You would be more efficient by reporting somehthing like this. +--------------------------------------------------------------------+| |FLOW: 10.42.69.10:1024 <-> 10.10.10.69:22 TCP | +--------------------------------------------------------------------+| In either case the collector needs to know how to deal with the information. In the first case, if the collector wants bi-directional it can combine the 2 records. If it wants uni-directional ignore the FIR part. In the seconds case, if it wants bi-directional, just forward on. If it wants uni-directional, create 2 records and pass them along. You made this comment... > The advantage of this [the FIR] would be that the embedded flow > records could be understood by either an unidirectional or > bidirectionally aware collecter. The encapsulating "FIR" (that > binds the two together) could essentially turn into a "no-op" for > collectors that don't need to understand any application-level > relationship between those flows, for the given application. While it may make things slightly simpler in the collector the bandwidth cost is significant when you think of millions of flows per day. In the second case above. the cost of turning one record into 2 at the collector, would be less than the bandwidth cost of the frist case, IMHO. Paul Dave Plonka wrote: > > On Wed, Jan 09, 2002 at 04:08:55PM +0100, Juergen Quittek wrote: > > --On 08 January 2002 17:12 -0500 Carter Bullard wrote: > > > > but you need to have something that states > > > that these records are related. You won't be able to infer > > > this from the 5-tuple flow descriptor in either record, > > > so you'll need some type of binding identifier. > > > > Exactly. Now who's work should it be to find the binding? > > The exporter? Why not the application analyzing the data > > at the collector. It is much better suited for this job and > > can do it online or offline depending of capabilities and > > requirements for this analysis. But I would not ask the > > exporter to do this job. > > (But what if the exporter _volunteers_ to do that job? ;^) > > That binding is what I called "flow matching" in an earlier post: > > http://ipfix.doit.wisc.edu/archive/0356.html > > I take it that Carter's point is that some exporters already know that > binding (before they export) - i.e. it will know which flows match. > > This is what I was trying to accommodate, in my first posting: > > http://ipfix.doit.wisc.edu/archive/0330.html > > in this thread when I said: > > [...] the IPFIX data model and transport could define a mechanism by > which the exporter could inform the collector about which > unidirectional flow records are matched pairs. > > and Paul reiterated here: > > http://ipfix.doit.wisc.edu/archive/0349.html > > I've been giving this some more thought, and would like to have some > email-friendly format for us to run through examples. How about this: > I'll show individual unidirectional flows as a single line in > pseudo-syntax sort of like that produced by the argus ra(1) command or > my "flowdumper -s" utility with NetFlow or LFAP, i.e.: > > +----------------------------------------------------------------------+ > | FLOW: src_addr:src_port -> dst_addr:dstport PROTO ... | > +----------------------------------------------------------------------+ > > Consider this example of a SSH session: > > Current Unidirectional exporters will report it like this (essentially > two records, minimum): > > +----------------------------------------------------------------------+ > | FLOW: 10.42.69.10:1024 -> 10.10.10.69:22 TCP | > +----------------------------------------------------------------------+ > | FLOW: 10.10.10.69:22 -> 10.42.69.10:1024 TCP | > +----------------------------------------------------------------------+ > > If the data model and architecture is sufficiently feature-rich, a > bidirectional exporter could export those flows, perhaps, as two flow > records wrapped as one "Flow Information Record" or whatever you'd like > to call it. E.g.: > > +----------------------------------------------------------------------+ > | FIR: (unicast, "normal" TCP session) | > |+--------------------------------------------------------------------+| > ||FLOW: 10.42.69.10:1024 -> 10.10.10.69:22 TCP || > |+--------------------------------------------------------------------+| > ||FLOW: 10.10.10.69:22 -> 10.42.69.10:1024 TCP || > |+--------------------------------------------------------------------+| > +----------------------------------------------------------------------+ > > Note that the two unidirectional flow records are wrapped in what I > called a FIR, or "Flow Information Record". > > (This ASCII art is just meant to be a logical representation, I'm not > suggestion a typedef struct, TLV, or anything specific.) > > The advantage of this would be that the embedded flow records could be > understood by either an unidirectional or bidirectionally aware > collecter. The encapsulating "FIR" (that binds the two together) could > essentially turn into a "no-op" for collectors that don't need to > understand any application-level relationship between those flows, for > the given application. > > I've worked through other examples for ping, traceroute, multicast > w/multiple egress ifIndex numbers, etc. but I'll hold off until I see > if this is understandable. > > All in all, I like the clarity of unidirectional flow definition, and > also only requiring unidirectional support as the common-denominator, > but I do agree that and the adopting of a unidirectional flow > definition (if we do so) should not make more work for the collector > when the exporter could pass that info along. > > Dave > > -- > plonka@doit.wisc.edu http://net.doit.wisc.edu/~plonka ARS:N9HZF Madison, WI > > -- > Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Fri Jan 11 12:53:35 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA23390 for ; Fri, 11 Jan 2002 12:53:35 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16P5cH-0000yF-00 for ipfix-list@mil.doit.wisc.edu; Fri, 11 Jan 2002 11:37:21 -0600 Received: from sj-msg-core-4.cisco.com ([171.71.163.10]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 16P5cD-0000wx-00 for ipfix-arch@net.doit.wisc.edu; Fri, 11 Jan 2002 11:37:17 -0600 Received: from mira-sjcd-1.cisco.com (mira-sjcd-1.cisco.com [171.69.43.44]) by sj-msg-core-4.cisco.com (8.11.3/8.9.1) with ESMTP id g0BHakJ06373; Fri, 11 Jan 2002 09:36:46 -0800 (PST) Received: from cisco.com (dhcp-171-71-137-45.cisco.com [171.71.137.45]) by mira-sjcd-1.cisco.com (Mirapoint) with ESMTP id AAV17238; Fri, 11 Jan 2002 09:36:53 -0800 (PST) Message-ID: <3C3F24AB.1C2B4180@cisco.com> Date: Fri, 11 Jan 2002 09:45:15 -0800 From: Ganesh Sadasivan X-Mailer: Mozilla 4.76 [en]C-CCK-MCD (Windows NT 5.0; U) X-Accept-Language: en MIME-Version: 1.0 To: calato@riverstonenet.com CC: arch Subject: Re: [ipfix-arch] configuration information. References: <4F973E944951D311B6660008C7917CF0083E088C@zsc4c008.us.nortel.com> <3C3B2FB6.3F1F0343@riverstonenet.com> <3C3CC55D.DF16DE29@cisco.com> <3C3F105D.8D980DAE@riverstonenet.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Caloto, calato@riverstonenet.com wrote: > Ganesh Sadasivan wrote: > > > > > > calato@riverstonenet.com wrote: > > > > > > > > > > > > If you have very specific templates such as srcIP, > > > > > destIP, etc, it's > > > > > > > not to hard find records in common. But you can have things > > > > like: > > > > > > > > > > > > > > src_AnyIP Dest_Yahoo.com Proto_Any and > > > > > > > src_AnyIP Dest_AS3462 Proto_HTTP. > > > > > > > > > > I assume here you mean that there are 2 counters and > > > > > 2 "flows" reported. One has (to use cisco terms) a template > > > > > of "dest-ip, byes, packets" and the other "dest-AS, proto, > > > > > byes, packets". > > > > > > > > > > If the device uses a multi-match method then the device > > > > > "knows" that a packet may be added to both buckets. It > > > > > should tell the collector that these templates overlap. > > > > > > > > I agree. But it should say that not only these templates overlap but > > > > what 5-tuple? is overlapping. (also see below) > > > > > > > > > If on the other hand the device uses a first match method > > > > > then the counts do not overlap. > > > > > > > > Should we assume that you can have different templates per > > > > Card/Interface if the cards/interfaces have intelligence? In this case > > > > they could overlap. > > > > > > Again, the device "knows" how it is counting. If the > > > devices is able to track packets based on multiple > > > keys then some thought needs to be given to whether > > > or not those key sets overlap. For example, if one key > > > set is "in-interface, src-ip, dst-ip" and the other is > > > "in-interface, src-AS, dst-AS" then the associated > > > counters do not overlap. If the application wants > > > a total for src-AS1 then it can add the bytes from > > > first ket set to the second. > > > > > > Overlaping counters are not something determined > > > after the fact based on data seen, it is determined > > > ahead of time for an entire class (or key set) of > > > records. > > > > > > > This is really not true. In the above example there are multiple > > ways of measuring: > > 1. All packets are tracked based on both the key sets. > > There is 100% duplication. > > 2. Packets are tracked based on the first matching key set. > > A single src-ip can have multiple src-ASs. Assuming the ASs > > for src-IP1 are src-AS1 and src-AS2, and say only > > src-AS1 is setup to match and the keyset {in-interface, src-AS, dst-AS} > > gets matched first, part of the packets that come on src-IP1 > > gets matched to the first key set and the other part gets matched > > to the {in-interface, src-ip, dst-ip} key set. The collector can > > get the src-ASs per src-IP information through an external BGP > > daemon or some other means. > > Even if the collector has knowledge of how many origin ASs are there > > for src-IP1, it still cannot find out how many packets passed through > > src-IP1. > > I don't follow you here. If the src IP matches the > first rule then I stop. I don't care what the AS is > because I don't look further. The point is that > only one counter bucket is ever updated. Thus I know > I have no duplication of counts. You are correct if the src IP is matched first. If the src AS is matched first ,and only one of the src AS corresponding to src IP is setup to match (not because of mis configuration, but because of the dynamic nature of AS assignment to the IP), then the counts for the src IP gets split across 2 different key sets. There will be no easy way to determine the actual number of packets that came on this src IP. Ganesh > > What am I missing? > > > > > This is a fairly simple example. Looks to me that the complexity on the > > collector side so that it can automatically infer the relationships between > > key sets is quite high. > > > > Thanks > > Ganesh > > -- > Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Fri Jan 11 13:10:28 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA24021 for ; Fri, 11 Jan 2002 13:10:28 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16P5rt-0001EN-00 for ipfix-list@mil.doit.wisc.edu; Fri, 11 Jan 2002 11:53:29 -0600 Received: from host3.riverstonenet.com ([63.113.148.3] helo=exc-sc1.yagosys.com) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 16P5rr-0001Dg-00 for ipfix-req@net.doit.wisc.edu; Fri, 11 Jan 2002 11:53:27 -0600 Received: from riverstonenet.com (134.141.180.91 [134.141.180.91]) by exc-sc1.yagosys.com with SMTP (Microsoft Exchange Internet Mail Service Version 5.5.2653.13) id X7WVZ965; Fri, 11 Jan 2002 09:52:28 -0800 Message-ID: <3C3F2660.F03D3D19@riverstonenet.com> Date: Fri, 11 Jan 2002 12:52:32 -0500 From: calato@riverstonenet.com X-Mailer: Mozilla 4.75C-CCK-MCD NSCPCD475 [en] (X11; U; SunOS 5.6 sun4u) X-Accept-Language: en MIME-Version: 1.0 To: Sebastian Zander CC: req Subject: Re: [ipfix-req] unidirection vs. bidirectional flows References: <5C8959A16A71B449AE793CF52FBBED66018FD4@ptah.newyork.qosient.com> <1275220606.1010592535@[192.168.102.164]> <20020109095148.A9422@doit.wisc.edu> <3C3C7855.3EA1620D@fokus.fhg.de> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Sebastian Zander wrote: > > I am trying to summarize the discussion from the requirements viewpoint: > > Do we need to add a requirement on the flow model to the requirements draft? > > If yes then looking at the discussion it seems that there is rough consensus > on: MUST support uni-directional, additionally MAY/SHOULD(?) support > bi-directional if the exporter supports flow matching. I think it is a MUST on uni-directional. I think bi-directional is a data efficiency question. If I as the exporter "know" these 2 flows are bi-directional, then I want to do 2 things 1) save space by reporting information once and 2) indicate the relationship between these uni-directional flows. There may be lots of relationships we want be able to represent. Such as, these flows are part of the same multi-cast. I don't see why we need to call out bi-directional as a special case in the requirements. As for the data model, there are lots of ways to save space and we should look at that as a topic of its own. As for the relationship issue, again, we need to look at that on its own. Some relationships we will want in the IPFIX data and some we will want to calculate down stream. Paul > > Correct? > > Cheers, > > Sebastian > > -- > Sebastian Zander E-mail: zander@fokus.fhg.de > FhI FOKUS / Global Networking (GloNe) Tel: +49-30-3463-7287 > Kaiserin-Augusta-Allee 31 Fax: +49-30-3463-8287 > D-10589 Berlin, Germany www.fokus.fhg.de/usr/sebastian.zander > > -- > Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Fri Jan 11 13:55:54 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA25543 for ; Fri, 11 Jan 2002 13:55:53 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16P6Yo-0002BW-00 for ipfix-list@mil.doit.wisc.edu; Fri, 11 Jan 2002 12:37:50 -0600 Received: from host3.riverstonenet.com ([63.113.148.3] helo=exc-sc1.yagosys.com) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 16P6Yl-0002B2-00 for ipfix-arch@net.doit.wisc.edu; Fri, 11 Jan 2002 12:37:48 -0600 Received: from riverstonenet.com (134.141.180.91 [134.141.180.91]) by exc-sc1.yagosys.com with SMTP (Microsoft Exchange Internet Mail Service Version 5.5.2653.13) id X7WVZ03H; Fri, 11 Jan 2002 10:36:47 -0800 Message-ID: <3C3F30C2.BA24EBEA@riverstonenet.com> Date: Fri, 11 Jan 2002 13:36:50 -0500 From: calato@riverstonenet.com X-Mailer: Mozilla 4.75C-CCK-MCD NSCPCD475 [en] (X11; U; SunOS 5.6 sun4u) X-Accept-Language: en MIME-Version: 1.0 To: Ganesh Sadasivan CC: arch Subject: Re: [ipfix-arch] configuration information. References: <4F973E944951D311B6660008C7917CF0083E088C@zsc4c008.us.nortel.com> <3C3B2FB6.3F1F0343@riverstonenet.com> <3C3CC55D.DF16DE29@cisco.com> <3C3F105D.8D980DAE@riverstonenet.com> <3C3F24AB.1C2B4180@cisco.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Ganesh Sadasivan wrote: > > Caloto, > > calato@riverstonenet.com wrote: > > > Ganesh Sadasivan wrote: > > > > > > > > > calato@riverstonenet.com wrote: > > > > > > > > > > > > > > > If you have very specific templates such as srcIP, > > > > > > destIP, etc, it's > > > > > > > > not to hard find records in common. But you can have things > > > > > like: > > > > > > > > > > > > > > > > src_AnyIP Dest_Yahoo.com Proto_Any and > > > > > > > > src_AnyIP Dest_AS3462 Proto_HTTP. > > > > > > > > > > > > I assume here you mean that there are 2 counters and > > > > > > 2 "flows" reported. One has (to use cisco terms) a template > > > > > > of "dest-ip, byes, packets" and the other "dest-AS, proto, > > > > > > byes, packets". > > > > > > > > > > > > If the device uses a multi-match method then the device > > > > > > "knows" that a packet may be added to both buckets. It > > > > > > should tell the collector that these templates overlap. > > > > > > > > > > I agree. But it should say that not only these templates overlap but > > > > > what 5-tuple? is overlapping. (also see below) > > > > > > > > > > > If on the other hand the device uses a first match method > > > > > > then the counts do not overlap. > > > > > > > > > > Should we assume that you can have different templates per > > > > > Card/Interface if the cards/interfaces have intelligence? In this case > > > > > they could overlap. > > > > > > > > Again, the device "knows" how it is counting. If the > > > > devices is able to track packets based on multiple > > > > keys then some thought needs to be given to whether > > > > or not those key sets overlap. For example, if one key > > > > set is "in-interface, src-ip, dst-ip" and the other is > > > > "in-interface, src-AS, dst-AS" then the associated > > > > counters do not overlap. If the application wants > > > > a total for src-AS1 then it can add the bytes from > > > > first ket set to the second. > > > > > > > > Overlaping counters are not something determined > > > > after the fact based on data seen, it is determined > > > > ahead of time for an entire class (or key set) of > > > > records. > > > > > > > > > > This is really not true. In the above example there are multiple > > > ways of measuring: > > > 1. All packets are tracked based on both the key sets. > > > There is 100% duplication. > > > 2. Packets are tracked based on the first matching key set. > > > A single src-ip can have multiple src-ASs. Assuming the ASs > > > for src-IP1 are src-AS1 and src-AS2, and say only > > > src-AS1 is setup to match and the keyset {in-interface, src-AS, dst-AS} > > > gets matched first, part of the packets that come on src-IP1 > > > gets matched to the first key set and the other part gets matched > > > to the {in-interface, src-ip, dst-ip} key set. The collector can > > > get the src-ASs per src-IP information through an external BGP > > > daemon or some other means. > > > Even if the collector has knowledge of how many origin ASs are there > > > for src-IP1, it still cannot find out how many packets passed through > > > src-IP1. > > > > I don't follow you here. If the src IP matches the > > first rule then I stop. I don't care what the AS is > > because I don't look further. The point is that > > only one counter bucket is ever updated. Thus I know > > I have no duplication of counts. > > You are correct if the src IP is matched first. If the src AS is matched > first ,and only one of the src AS corresponding to src IP is setup to > match (not because of mis configuration, but because of the dynamic > nature of AS assignment to the IP), then the > counts for the src IP gets split across 2 different key sets. There will be > no easy way to determine the actual number of packets that came on > this src IP. If the goal was to know how much data src-IP1 generated then I would suggest that this is a miss configuration. The src-IP1 rule should have been first. You can't go from a higher level aggregation like AS to a lower level one like IP. But there is still no question on overlap. There is no overlap because only one bucket is updated for a packet. My original point was the collector will need some overlap information in order to make sense of the data. And I think your point was that it is too hard to know what buckets overlap. I still don't quite follow why that is. In a first match scenario, there is no overlap. I'm free to add the counts together if my collector analysis shows them to be related. Without overlap info, I cannot do that. We might need a phone call to get to the bottom. Feel free to call me at 603-738-4206. I'm on the east coast in the US. > > Ganesh > > > > > What am I missing? > > > > > > > > This is a fairly simple example. Looks to me that the complexity on the > > > collector side so that it can automatically infer the relationships between > > > key sets is quite high. > > > > > > Thanks > > > Ganesh > > > > -- > > Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > "unsubscribe ipfix" in message body > > Archive http://ipfix.doit.wisc.edu/archive/ -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Fri Jan 11 15:08:08 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA27836 for ; Fri, 11 Jan 2002 15:08:07 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16P7id-0003dS-00 for ipfix-list@mil.doit.wisc.edu; Fri, 11 Jan 2002 13:52:03 -0600 Received: from patan.sun.com ([192.18.98.43]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 16P7ia-0003cy-00 for ipfix-req@net.doit.wisc.edu; Fri, 11 Jan 2002 13:52:00 -0600 Received: from sunmail1.Sun.COM ([129.145.1.2]) by patan.sun.com (8.9.3+Sun/8.9.3) with ESMTP id MAA27718; Fri, 11 Jan 2002 12:51:56 -0700 (MST) Received: from jurassic.eng.sun.com (jurassic.Eng.Sun.COM [129.146.82.166]) by sunmail1.Sun.COM (8.9.3+Sun/8.9.3/ENSMAIL,v2.1p1-Sun.COM.mod.2) with ESMTP id LAA05606; Fri, 11 Jan 2002 11:53:03 -0800 (PST) Received: from sun.com (inox.Eng.Sun.COM [129.146.85.133]) by jurassic.eng.sun.com (8.12.2.Beta5+Sun/8.12.2.Beta5) with ESMTP id g0BJprS2638640; Fri, 11 Jan 2002 11:51:53 -0800 (PST) Message-ID: <3C3F4260.98A81734@sun.com> Date: Fri, 11 Jan 2002 11:52:00 -0800 From: Jc Martin Organization: Sun Microsystems X-Mailer: Mozilla 4.78 [en] (X11; U; SunOS 5.9 sun4u) X-Accept-Language: en MIME-Version: 1.0 To: calato@riverstonenet.com CC: Sebastian Zander , req Subject: Re: [ipfix-req] unidirection vs. bidirectional flows References: <5C8959A16A71B449AE793CF52FBBED66018FD4@ptah.newyork.qosient.com> <1275220606.1010592535@[192.168.102.164]> <20020109095148.A9422@doit.wisc.edu> <3C3C7855.3EA1620D@fokus.fhg.de> <3C3F2660.F03D3D19@riverstonenet.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit calato@riverstonenet.com wrote: > > Sebastian Zander wrote: > > > > I am trying to summarize the discussion from the requirements viewpoint: > > > > Do we need to add a requirement on the flow model to the requirements draft? > > > > If yes then looking at the discussion it seems that there is rough consensus > > on: MUST support uni-directional, additionally MAY/SHOULD(?) support > > bi-directional if the exporter supports flow matching. > > I think it is a MUST on uni-directional. I think > bi-directional is a data efficiency question. > > If I as the exporter "know" these 2 flows > are bi-directional, then I want to do 2 things > 1) save space by reporting information once and > 2) indicate the relationship between these > uni-directional flows. > > There may be lots of relationships we want be able > to represent. Such as, these flows are part of the > same multi-cast. I don't see why we need to call > out bi-directional as a special case in the > requirements. > > As for the data model, there are lots of ways to > save space and we should look at that as a topic > of its own. > > As for the relationship issue, again, we need to look > at that on its own. Some relationships we will want > in the IPFIX data and some we will want to calculate > down stream. I agree. What is mandatory is the support of uni-directional. Then, we should mention that the exporter MAY group unidirectional flows to represent higher semantic, such as bi-directional flows, or transactions. The way this grouping is done is going to be specified in the data model. Ideally, it would be hierarchical to accomodate multiple levels of grouping, for example : transaction -> bi-directional -> unidirectional This grouping can be done as simply as adding attributes that are processed by the collectors to reconstruct higher level flows. JC. -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Fri Jan 11 16:08:01 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA29658 for ; Fri, 11 Jan 2002 16:08:01 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16P8fE-0004uL-00 for ipfix-list@mil.doit.wisc.edu; Fri, 11 Jan 2002 14:52:36 -0600 Received: from mailhub.lawrence.edu ([143.44.65.14]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 16P8fC-0004uE-00 for ipfix-req@net.doit.wisc.edu; Fri, 11 Jan 2002 14:52:34 -0600 Received: from lawrence.edu ([143.44.97.14]) by lawrence.edu (PMDF V6.0-025 #44893) with ESMTP id <0GPS00B87L64Q8@lawrence.edu> for ipfix-req@net.doit.wisc.edu; Fri, 11 Jan 2002 15:03:40 -0600 (CST) Date: Fri, 11 Jan 2002 14:52:33 -0600 From: Robert Lowe Subject: Re: [ipfix-req] unidirection vs. bidirectional flows To: calato@riverstonenet.com Cc: ipfix-req@net.doit.wisc.edu Message-id: <3C3F5091.E77F3B6B@lawrence.edu> Organization: Lawrence University MIME-version: 1.0 X-Mailer: Mozilla 4.74 [en] (Windows NT 5.0; U) Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7bit X-Accept-Language: en References: <5C8959A16A71B449AE793CF52FBBED66018FD4@ptah.newyork.qosient.com> <1275220606.1010592535@[192.168.102.164]> <20020109095148.A9422@doit.wisc.edu> <3C3F1895.D89D4BF9@riverstonenet.com> Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit calato@riverstonenet.com wrote: > > I think one of Carter's points though was efficiency. > So, to use your structure, instead of reporting this... > > +----------------------------------------------------------------------+ > | FIR: (unicast, "normal" TCP session) | > |+--------------------------------------------------------------------+| > ||FLOW: 10.42.69.10:1024 -> 10.10.10.69:22 TCP || > |+--------------------------------------------------------------------+| > ||FLOW: 10.10.10.69:22 -> 10.42.69.10:1024 TCP || > |+--------------------------------------------------------------------+| > +----------------------------------------------------------------------+ > > You would be more efficient by reporting somehthing like this. > > +--------------------------------------------------------------------+| > |FLOW: 10.42.69.10:1024 <-> 10.10.10.69:22 TCP | > +--------------------------------------------------------------------+| > > In either case the collector needs to know how to deal > with the information. In the first case, if the collector > wants bi-directional it can combine the 2 records. If it > wants uni-directional ignore the FIR part. A couple of simple-minded questions as a lurker here... If you do report a bi-directional flow, do you turn reported elements, e.g. bytes or packets, into tuples within the flow record, one for from-src and the other for from-dest? Or do you always assume an aggregate? Assuming an aggregate is simpler, but not ideal. > In the seconds case, if it wants bi-directional, just forward > on. If it wants uni-directional, create 2 records and pass them > along. > > You made this comment... > > > The advantage of this [the FIR] would be that the embedded flow > > records could be understood by either an unidirectional or > > bidirectionally aware collecter. The encapsulating "FIR" (that > > binds the two together) could essentially turn into a "no-op" for > > collectors that don't need to understand any application-level > > relationship between those flows, for the given application. > > While it may make things slightly simpler in the collector the > bandwidth cost is significant when you think of millions of > flows per day. In the second case above. the cost of turning > one record into 2 at the collector, would be less than the > bandwidth cost of the frist case, IMHO. Given that the reporting method will have a bandwidth impact, does an exporter export anything about the traffic it generates? Or does it store aggregate statistics that can be polled by the collector? A flexible approach allows for differing designs where processing can be distributed between the exporter and the collector to varying extents, but since bandwidth consumed and processing power are linked, I'm not sure I understand where bi-directional reporting fits. When the routing engine is under stress, the exporter will not have some magic reserve of processing power to do "flow matching", but if it resorts to uni-directional flow reporting, it exacerbates the situation by consuming more bandwidth, leaving only a fall-back to packet sampling. And, if that happens, what kind of impact does it have on bi-directional flow reporting? -Robert -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Fri Jan 11 16:25:02 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA00093 for ; Fri, 11 Jan 2002 16:25:01 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16P8ud-0005Lr-00 for ipfix-list@mil.doit.wisc.edu; Fri, 11 Jan 2002 15:08:31 -0600 Received: from zrc2s0jx.nortelnetworks.com ([47.103.122.112]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 16P8uZ-0005L0-00 for ipfix-arch@net.doit.wisc.edu; Fri, 11 Jan 2002 15:08:27 -0600 Received: from zsc4c000.us.nortel.com (zsc4c000.us.nortel.com [47.81.138.47]) by zrc2s0jx.nortelnetworks.com (Switch-2.2.0/Switch-2.2.0) with ESMTP id g0BL40b23665; Fri, 11 Jan 2002 15:04:01 -0600 (CST) Received: by zsc4c000.us.nortel.com with Internet Mail Service (5.5.2653.19) id ; Fri, 11 Jan 2002 13:04:00 -0800 Message-ID: <4F973E944951D311B6660008C7917CF00844F08C@zsc4c008.us.nortel.com> From: "Reinaldo Penno" To: "'calato@riverstonenet.com'" , "'Ganesh Sadasivan'" Cc: "'arch'" Subject: RE: [ipfix-arch] configuration information. Date: Fri, 11 Jan 2002 13:03:59 -0800 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01C19AE3.7D9A6A50" Precedence: bulk Sender: majordomo listserver This message is in MIME format. Since your mail reader does not understand this format, some or all of this message may not be legible. ------_=_NextPart_001_01C19AE3.7D9A6A50 Content-Type: text/plain; charset="iso-8859-1" > -----Original Message----- > From: calato@riverstonenet.com [mailto:calato@riverstonenet.com] > Sent: Friday, January 11, 2002 10:37 AM > To: Ganesh Sadasivan > Cc: arch > Subject: Re: [ipfix-arch] configuration information. > > > Ganesh Sadasivan wrote: > > > > Caloto, > > > > calato@riverstonenet.com wrote: > > > > > Ganesh Sadasivan wrote: > > > > > > > > > > > > calato@riverstonenet.com wrote: > > > > > > > > > > > > > > > > > > If you have very specific templates such as srcIP, > > > > > > > destIP, etc, it's > > > > > > > > > not to hard find records in common. But you > can have things > > > > > > like: > > > > > > > > > > > > > > > > > > src_AnyIP Dest_Yahoo.com Proto_Any and > > > > > > > > > src_AnyIP Dest_AS3462 Proto_HTTP. > > > > > > > > > > > > > > I assume here you mean that there are 2 counters and > > > > > > > 2 "flows" reported. One has (to use cisco > terms) a template > > > > > > > of "dest-ip, byes, packets" and the other > "dest-AS, proto, > > > > > > > byes, packets". > > > > > > > > > > > > > > If the device uses a multi-match method > then the device > > > > > > > "knows" that a packet may be added to both > buckets. It > > > > > > > should tell the collector that these > templates overlap. > > > > > > > > > > > > I agree. But it should say that not only these > templates overlap but > > > > > > what 5-tuple? is overlapping. (also see below) > > > > > > > > > > > > > If on the other hand the device uses a > first match method > > > > > > > then the counts do not overlap. > > > > > > > > > > > > Should we assume that you can have different templates per > > > > > > Card/Interface if the cards/interfaces have > intelligence? In this case > > > > > > they could overlap. > > > > > > > > > > Again, the device "knows" how it is counting. If the > > > > > devices is able to track packets based on multiple > > > > > keys then some thought needs to be given to whether > > > > > or not those key sets overlap. For example, if one key > > > > > set is "in-interface, src-ip, dst-ip" and the other is > > > > > "in-interface, src-AS, dst-AS" then the associated > > > > > counters do not overlap. If the application wants > > > > > a total for src-AS1 then it can add the bytes from > > > > > first ket set to the second. > > > > > > > > > > Overlaping counters are not something determined > > > > > after the fact based on data seen, it is determined > > > > > ahead of time for an entire class (or key set) of > > > > > records. > > > > > > > > > > > > > This is really not true. In the above example there are multiple > > > > ways of measuring: > > > > 1. All packets are tracked based on both the key sets. > > > > There is 100% duplication. > > > > 2. Packets are tracked based on the first matching key set. > > > > A single src-ip can have multiple src-ASs. Assuming the ASs > > > > for src-IP1 are src-AS1 and src-AS2, and say only > > > > src-AS1 is setup to match and the keyset > {in-interface, src-AS, dst-AS} > > > > gets matched first, part of the packets that come on src-IP1 > > > > gets matched to the first key set and the other > part gets matched > > > > to the {in-interface, src-ip, dst-ip} key set. The > collector can > > > > get the src-ASs per src-IP information through an > external BGP > > > > daemon or some other means. > > > > Even if the collector has knowledge of how many > origin ASs are there > > > > for src-IP1, it still cannot find out how many > packets passed through > > > > src-IP1. > > > > > > I don't follow you here. If the src IP matches the > > > first rule then I stop. I don't care what the AS is > > > because I don't look further. The point is that > > > only one counter bucket is ever updated. Thus I know > > > I have no duplication of counts. > > > > You are correct if the src IP is matched first. If the src > AS is matched > > first ,and only one of the src AS corresponding to src IP > is setup to > > match (not because of mis configuration, but because of the dynamic > > nature of AS assignment to the IP), then the > > counts for the src IP gets split across 2 different key > sets. There will be > > no easy way to determine the actual number of packets that came on > > this src IP. > > If the goal was to know how much data src-IP1 generated > then I would suggest that this is a miss configuration. > The src-IP1 rule should have been first. You can't > go from a higher level aggregation like AS to a lower > level one like IP. humm..What's more aggregate? AS3461 or 200.244/16? Or even 9/8? > > But there is still no question on overlap. There is no > overlap because only one bucket is updated for a packet. you are right, but you are not sure if there was (or would be) overlap or not in a first match approach. Let's suppose you have two rules, one for SIP and another one for UDP. No all UDP packets are SIP packets, and not all SIP packets are UDP. You then receive SIP-TCP, TCP, UDP and SIP-UDP packets. The first three packets are okay, but the fourth packets would match both rules, but these rules measure different things. Which rule would you put first? Rule is not a superset of Rule 2 and vice-versa. I imagine then that the argument is to configure more and more granular rules... > > My original point was the collector will need > some overlap information in order to make sense of > the data. > > And I think your point was that it is too hard to > know what buckets overlap. I still don't quite follow > why that is. well, you can always argue that the "device knows" and end the discussion, but the fact is that a packet can be counted twice in several situations if the "device does not know". In a first match scenario, there is no > overlap. I'm free to add the counts together if > my collector analysis shows them to be related. > Without overlap info, I cannot do that. > > We might need a phone call to get to the bottom. > Feel free to call me at 603-738-4206. I'm on > the east coast in the US. > > > > Ganesh > > > > > > > > What am I missing? > > > > > > > > > > > This is a fairly simple example. Looks to me that the > complexity on the > > > > collector side so that it can automatically infer the > relationships between > > > > key sets is quite high. > > > > > > > > Thanks > > > > Ganesh > > > > > > -- > > > Help mailto:majordomo@net.doit.wisc.edu and say > "help" in message body > > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > > "unsubscribe ipfix" in message body > > > Archive http://ipfix.doit.wisc.edu/archive/ > > -- > Help mailto:majordomo@net.doit.wisc.edu and say "help" > in message body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ > ------_=_NextPart_001_01C19AE3.7D9A6A50 Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable RE: [ipfix-arch] configuration information.

> -----Original Message-----
> From: calato@riverstonenet.com [mailto:calato@riverstonenet.com= ]
> Sent: Friday, January 11, 2002 10:37 AM
> To: Ganesh Sadasivan
> Cc: arch
> Subject: Re: [ipfix-arch] configuration = information.
>
>
> Ganesh Sadasivan wrote:
> >
> > Caloto,
> >
> > calato@riverstonenet.com wrote:
> >
> > > Ganesh Sadasivan wrote:
> > > >
> > > > <snip>
> > > > calato@riverstonenet.com = wrote:
> > > >
> > > > >
> > > > > > > > > If you = have very specific templates such as srcIP,
> > > > > > > destIP, etc, = it's
> > > > > > > > > not to = hard find records in common. But you
> can have things
> > > > > > like:
> > > > > > > > >
> > > > > > > > > = src_AnyIP   Dest_Yahoo.com    = Proto_Any   and
> > > > > > > > > = src_AnyIP   Dest_AS3462       = Proto_HTTP.
> > > > > > >
> > > > > > = >       I assume here you mean that = there are 2 counters and
> > > > > > = >       2 "flows" reported. = One has (to use cisco
> terms) a template
> > > > > > = >       of "dest-ip, byes, = packets" and the other
> "dest-AS, proto,
> > > > > > = >       byes, packets".
> > > > > > >
> > > > > > = >       If the device uses a = multi-match method
> then the device
> > > > > > = >       "knows" that a = packet may be added to both
> buckets. It
> > > > > > = >       should tell the collector that = these
> templates overlap.
> > > > > >
> > > > > > I agree. But it should = say that not only these
> templates overlap but
> > > > > > what 5-tuple? is = overlapping. (also see below)
> > > > > >
> > > > > > = >       If on the other hand the = device uses a
> first match method
> > > > > > = >       then the counts do not = overlap.
> > > > > >
> > > > > > Should we assume that = you can have different templates per
> > > > > > Card/Interface if the = cards/interfaces have
> intelligence? In this case
> > > > > > they could = overlap.
> > > > >
> > > > = >         Again, the device = "knows" how it is counting. If the
> > > > = >         devices is able to = track packets based on multiple
> > > > = >         keys then some = thought needs to be given to whether
> > > > = >         or not those key = sets overlap. For example, if one key
> > > > = >         set is = "in-interface, src-ip, dst-ip" and the other is
> > > > = >         = "in-interface, src-AS, dst-AS" then the associated
> > > > = >         counters do not = overlap. If the application wants
> > > > = >         a total for = src-AS1 then it can add the bytes from
> > > > = >         first ket set to = the second.
> > > > >
> > > > = >         Overlaping = counters are not something determined
> > > > = >         after the fact = based on data seen, it is determined
> > > > = >         ahead of time for = an entire class (or key set) of
> > > > = >         records.
> > > > >
> > > >
> > > > This is really not true. In the = above example there are multiple
> > > > ways of measuring:
> > > > 1. All packets are tracked based = on both the key sets.
> > > >     There is = 100% duplication.
> > > > 2. Packets are tracked based on = the first matching key set.
> > > >     A single = src-ip can have multiple src-ASs. Assuming the ASs
> > > >     for = src-IP1 are src-AS1 and src-AS2, and say only
> > > >     src-AS1 = is setup to match and the keyset
> {in-interface, src-AS, dst-AS}
> > > >     gets = matched first, part of the packets that come on src-IP1
> > > >     gets = matched to the first key set and the other
> part gets matched
> > > >     to the = {in-interface, src-ip, dst-ip} key set. The
> collector can
> > > >     get = the  src-ASs per src-IP information through an
> external BGP
> > > >     daemon = or some other means.
> > > >     Even if = the collector has knowledge of how many
> origin ASs are there
> > > >     for = src-IP1, it still cannot find out how many
> packets passed through
> > > >     = src-IP1.
> > >
> > = >         I don't follow you = here. If the src IP matches the
> > = >         first rule then I = stop. I don't care what the AS is
> > = >         because I don't = look further.  The point is that
> > = >         only one counter = bucket is ever updated. Thus I know
> > = >         I have no = duplication of counts.
> >
> > You are correct if the src IP is matched = first. If the src
> AS is matched
> > first ,and only one of the src AS = corresponding to src IP
> is setup to
> > match (not because of mis configuration, = but because of the dynamic
> > nature of AS assignment to the IP), then = the
> > counts for the src IP gets split across 2 = different key
> sets. There will be
> > no easy way to determine the actual number = of packets that came on
> > this src IP.
>
>       If the goal was = to know how much data src-IP1 generated
>       then I would = suggest that this is a miss configuration.
>       The src-IP1 rule = should have been first. You can't
>       go from a higher = level aggregation like AS to a lower
>       level one like = IP.

humm..What's more aggregate? AS3461 or 200.244/16? Or = even 9/8?

>
>       But there is = still no question on overlap. There is no
>       overlap because = only one bucket is updated for a packet.

you are right, but you are not sure if there was (or = would be) overlap or not in a first match approach. Let's suppose you = have two rules, one for SIP and another one for UDP. No all UDP packets = are SIP packets, and not all SIP packets are UDP.

You then receive SIP-TCP, TCP, UDP and SIP-UDP = packets. The first three packets are okay, but the fourth packets would = match both rules, but these rules measure different things. Which rule = would you put first? Rule is not a superset of Rule 2 and = vice-versa.

I imagine then that the argument is to configure more = and more granular rules...

 
>
>       My original = point was the collector will need
>       some overlap = information in order to make sense of
>       the data.
>
>       And I think your = point was that it is too hard to
>       know what = buckets overlap. I still don't quite follow
>       why that is. =

well, you can always argue that the "device = knows" and end the discussion, but the fact is that a packet can = be counted twice in several situations if the "device does not = know".

In a first match scenario, there is no
>       overlap. I'm = free to add the counts together if
>       my collector = analysis shows them to be related.
>       Without overlap = info, I cannot do that.
>
>       We might need a = phone call to get to the bottom.
>       Feel free to = call me at 603-738-4206. I'm on
>       the east coast = in the US.
> >
> > Ganesh
> >
> > >
> > = >         What am I = missing?
> > >
> > > >
> > > > This is a fairly simple example. = Looks to me that the
> complexity on the
> > > > collector side so that it can = automatically infer the
> relationships between
> > > > key sets is quite high.
> > > >
> > > > Thanks
> > > > Ganesh
> > >
> > > --
> > > = Help        mailto:majordomo@net.doit.wi= sc.edu and say
> "help" in message body
> > > Unsubscribe mailto:majordomo@net.doit.wi= sc.edu and say
> > > "unsubscribe ipfix" in = message body
> > > Archive     http://ipfix.doit.wisc.edu/archive/
>
> --
> Help        = mailto:majordomo@net.doit.wi= sc.edu and say "help"
> in message body
> Unsubscribe mailto:majordomo@net.doit.wi= sc.edu and say
> "unsubscribe ipfix" in message = body
> Archive     http://ipfix.doit.wisc.edu/archive/
>

------_=_NextPart_001_01C19AE3.7D9A6A50-- -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Fri Jan 11 16:54:40 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA01061 for ; Fri, 11 Jan 2002 16:54:40 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16P9OC-0005vt-00 for ipfix-list@mil.doit.wisc.edu; Fri, 11 Jan 2002 15:39:04 -0600 Received: from host3.riverstonenet.com ([63.113.148.3] helo=exc-sc1.yagosys.com) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 16P9OA-0005vA-00 for ipfix-arch@net.doit.wisc.edu; Fri, 11 Jan 2002 15:39:02 -0600 Received: from riverstonenet.com (134.141.180.91 [134.141.180.91]) by exc-sc1.yagosys.com with SMTP (Microsoft Exchange Internet Mail Service Version 5.5.2653.13) id X7WV5CQ7; Fri, 11 Jan 2002 13:38:01 -0800 Message-ID: <3C3F5B3B.ECF9545E@riverstonenet.com> Date: Fri, 11 Jan 2002 16:38:03 -0500 From: calato@riverstonenet.com X-Mailer: Mozilla 4.75C-CCK-MCD NSCPCD475 [en] (X11; U; SunOS 5.6 sun4u) X-Accept-Language: en MIME-Version: 1.0 To: Reinaldo Penno CC: "'Ganesh Sadasivan'" , "'arch'" Subject: Re: [ipfix-arch] configuration information. References: <4F973E944951D311B6660008C7917CF00844F08C@zsc4c008.us.nortel.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Reinaldo Penno wrote: > > > -----Original Message----- > > From: calato@riverstonenet.com [mailto:calato@riverstonenet.com] > > Sent: Friday, January 11, 2002 10:37 AM > > To: Ganesh Sadasivan > > Cc: arch > > Subject: Re: [ipfix-arch] configuration information. > > > > > > Ganesh Sadasivan wrote: > > > > > > Caloto, > > > > > > calato@riverstonenet.com wrote: > > > > > > > Ganesh Sadasivan wrote: > > > > > > > > > > > > > > > calato@riverstonenet.com wrote: > > > > > > > > > > > > > > > > > > > > > If you have very specific templates such as srcIP, > > > > > > > > destIP, etc, it's > > > > > > > > > > not to hard find records in common. But you > > can have things > > > > > > > like: > > > > > > > > > > > > > > > > > > > > src_AnyIP Dest_Yahoo.com Proto_Any and > > > > > > > > > > src_AnyIP Dest_AS3462 Proto_HTTP. > > > > > > > > > > > > > > > > I assume here you mean that there are 2 counters > and > > > > > > > > 2 "flows" reported. One has (to use cisco > > terms) a template > > > > > > > > of "dest-ip, byes, packets" and the other > > "dest-AS, proto, > > > > > > > > byes, packets". > > > > > > > > > > > > > > > > If the device uses a multi-match method > > then the device > > > > > > > > "knows" that a packet may be added to both > > buckets. It > > > > > > > > should tell the collector that these > > templates overlap. > > > > > > > > > > > > > > I agree. But it should say that not only these > > templates overlap but > > > > > > > what 5-tuple? is overlapping. (also see below) > > > > > > > > > > > > > > > If on the other hand the device uses a > > first match method > > > > > > > > then the counts do not overlap. > > > > > > > > > > > > > > Should we assume that you can have different templates per > > > > > > > > Card/Interface if the cards/interfaces have > > intelligence? In this case > > > > > > > they could overlap. > > > > > > > > > > > > Again, the device "knows" how it is counting. If the > > > > > > > devices is able to track packets based on multiple > > > > > > keys then some thought needs to be given to whether > > > > > > or not those key sets overlap. For example, if one > key > > > > > > set is "in-interface, src-ip, dst-ip" and the other > is > > > > > > "in-interface, src-AS, dst-AS" then the associated > > > > > > counters do not overlap. If the application wants > > > > > > a total for src-AS1 then it can add the bytes from > > > > > > first ket set to the second. > > > > > > > > > > > > Overlaping counters are not something determined > > > > > > after the fact based on data seen, it is determined > > > > > > ahead of time for an entire class (or key set) of > > > > > > records. > > > > > > > > > > > > > > > > This is really not true. In the above example there are > multiple > > > > > ways of measuring: > > > > > 1. All packets are tracked based on both the key sets. > > > > > There is 100% duplication. > > > > > 2. Packets are tracked based on the first matching key set. > > > > > A single src-ip can have multiple src-ASs. Assuming the > ASs > > > > > for src-IP1 are src-AS1 and src-AS2, and say only > > > > > src-AS1 is setup to match and the keyset > > {in-interface, src-AS, dst-AS} > > > > > gets matched first, part of the packets that come on > src-IP1 > > > > > gets matched to the first key set and the other > > part gets matched > > > > > to the {in-interface, src-ip, dst-ip} key set. The > > collector can > > > > > get the src-ASs per src-IP information through an > > external BGP > > > > > daemon or some other means. > > > > > Even if the collector has knowledge of how many > > origin ASs are there > > > > > for src-IP1, it still cannot find out how many > > packets passed through > > > > > src-IP1. > > > > > > > > I don't follow you here. If the src IP matches the > > > > first rule then I stop. I don't care what the AS is > > > > because I don't look further. The point is that > > > > only one counter bucket is ever updated. Thus I know > > > > I have no duplication of counts. > > > > > > You are correct if the src IP is matched first. If the src > > AS is matched > > > first ,and only one of the src AS corresponding to src IP > > is setup to > > > match (not because of mis configuration, but because of the > dynamic > > > nature of AS assignment to the IP), then the > > > counts for the src IP gets split across 2 different key > > sets. There will be > > > no easy way to determine the actual number of packets that came on > > > > this src IP. > > > > If the goal was to know how much data src-IP1 generated > > then I would suggest that this is a miss configuration. > > The src-IP1 rule should have been first. You can't > > go from a higher level aggregation like AS to a lower > > level one like IP. > > humm..What's more aggregate? AS3461 or 200.244/16? Or even 9/8? I meant IP/32 not prefixes. Sorry if that was not clear. > > > > > But there is still no question on overlap. There is no > > overlap because only one bucket is updated for a packet. > > you are right, but you are not sure if there was (or would be) overlap > or not in a first match approach. Let's suppose you have two rules, > one for SIP and another one for UDP. No all UDP packets are SIP > packets, and not all SIP packets are UDP. > > You then receive SIP-TCP, TCP, UDP and SIP-UDP packets. The first > three packets are okay, but the fourth packets would match both rules, > but these rules measure different things. Which rule would you put > first? Rule is not a superset of Rule 2 and vice-versa. Just like ACL's, you can have overlaping ACL's but the first one that is a hit wins. > > I imagine then that the argument is to configure more and more > granular rules... > > > > > > My original point was the collector will need > > some overlap information in order to make sense of > > the data. > > > > And I think your point was that it is too hard to > > know what buckets overlap. I still don't quite follow > > why that is. > > well, you can always argue that the "device knows" and end the > discussion, but the fact is that a packet can be counted twice in > several situations if the "device does not know". Great, then the device reports that all its flows may overlap. I'm no worse off than not saying anything. But there are many devices that have relatively simple counting mechanisms and the extra information will help down stream applications do some analysis that may otherwise not be doable. > > In a first match scenario, there is no > > overlap. I'm free to add the counts together if > > my collector analysis shows them to be related. > > Without overlap info, I cannot do that. > > > > We might need a phone call to get to the bottom. > > Feel free to call me at 603-738-4206. I'm on > > the east coast in the US. > > > > > > Ganesh > > > > > > > > > > > What am I missing? > > > > > > > > > > > > > > This is a fairly simple example. Looks to me that the > > complexity on the > > > > > collector side so that it can automatically infer the > > relationships between > > > > > key sets is quite high. > > > > > > > > > > Thanks > > > > > Ganesh > > > > > > > > -- > > > > Help mailto:majordomo@net.doit.wisc.edu and say > > "help" in message body > > > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > > > "unsubscribe ipfix" in message body > > > > Archive http://ipfix.doit.wisc.edu/archive/ > > > > -- > > Help mailto:majordomo@net.doit.wisc.edu and say "help" > > in message body > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > "unsubscribe ipfix" in message body > > Archive http://ipfix.doit.wisc.edu/archive/ > > -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Fri Jan 11 17:27:51 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA02020 for ; Fri, 11 Jan 2002 17:27:51 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16P9ty-0006f4-00 for ipfix-list@mil.doit.wisc.edu; Fri, 11 Jan 2002 16:11:54 -0600 Received: from host3.riverstonenet.com ([63.113.148.3] helo=exc-sc1.yagosys.com) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 16P9tw-0006eM-00 for ipfix-req@net.doit.wisc.edu; Fri, 11 Jan 2002 16:11:52 -0600 Received: from riverstonenet.com (134.141.180.91 [134.141.180.91]) by exc-sc1.yagosys.com with SMTP (Microsoft Exchange Internet Mail Service Version 5.5.2653.13) id X7WV5C6S; Fri, 11 Jan 2002 14:10:52 -0800 Message-ID: <3C3F62F0.ADFB1E4C@riverstonenet.com> Date: Fri, 11 Jan 2002 17:10:56 -0500 From: calato@riverstonenet.com X-Mailer: Mozilla 4.75C-CCK-MCD NSCPCD475 [en] (X11; U; SunOS 5.6 sun4u) X-Accept-Language: en MIME-Version: 1.0 To: Robert Lowe CC: ipfix-req@net.doit.wisc.edu Subject: Re: [ipfix-req] unidirection vs. bidirectional flows References: <5C8959A16A71B449AE793CF52FBBED66018FD4@ptah.newyork.qosient.com> <1275220606.1010592535@[192.168.102.164]> <20020109095148.A9422@doit.wisc.edu> <3C3F1895.D89D4BF9@riverstonenet.com> <3C3F5091.E77F3B6B@lawrence.edu> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Robert Lowe wrote: > > calato@riverstonenet.com wrote: > > > > I think one of Carter's points though was efficiency. > > So, to use your structure, instead of reporting this... > > > > +----------------------------------------------------------------------+ > > | FIR: (unicast, "normal" TCP session) | > > |+--------------------------------------------------------------------+| > > ||FLOW: 10.42.69.10:1024 -> 10.10.10.69:22 TCP || > > |+--------------------------------------------------------------------+| > > ||FLOW: 10.10.10.69:22 -> 10.42.69.10:1024 TCP || > > |+--------------------------------------------------------------------+| > > +----------------------------------------------------------------------+ > > > > You would be more efficient by reporting somehthing like this. > > > > +--------------------------------------------------------------------+| > > |FLOW: 10.42.69.10:1024 <-> 10.10.10.69:22 TCP | > > +--------------------------------------------------------------------+| > > > > In either case the collector needs to know how to deal > > with the information. In the first case, if the collector > > wants bi-directional it can combine the 2 records. If it > > wants uni-directional ignore the FIR part. > > A couple of simple-minded questions as a lurker here... > > If you do report a bi-directional flow, do you turn reported elements, > e.g. bytes or packets, into tuples within the flow record, one for > from-src and the other for from-dest? Or do you always assume an > aggregate? Assuming an aggregate is simpler, but not ideal. If you want to be able to create uni-directional flows I think you need to report tuples. > > > In the seconds case, if it wants bi-directional, just forward > > on. If it wants uni-directional, create 2 records and pass them > > along. > > > > You made this comment... > > > > > The advantage of this [the FIR] would be that the embedded flow > > > records could be understood by either an unidirectional or > > > bidirectionally aware collecter. The encapsulating "FIR" (that > > > binds the two together) could essentially turn into a "no-op" for > > > collectors that don't need to understand any application-level > > > relationship between those flows, for the given application. > > > > While it may make things slightly simpler in the collector the > > bandwidth cost is significant when you think of millions of > > flows per day. In the second case above. the cost of turning > > one record into 2 at the collector, would be less than the > > bandwidth cost of the frist case, IMHO. > > Given that the reporting method will have a bandwidth impact, does an > exporter export anything about the traffic it generates? Or does it > store aggregate statistics that can be polled by the collector? Some implementations don't keep track of flows to and from the exporter itself. But in that case I think it should maintain some internal statistics that can be retrieved out-of-band. For those that report flows to and from the exporter the info is already available. > > A flexible approach allows for differing designs where processing > can be distributed between the exporter and the collector to varying > extents, but since bandwidth consumed and processing power are linked, > I'm not sure I understand where bi-directional reporting fits. A few things. 1) if the bottle neck is CPU cycles because of routing you have lots of trouble. bi-directional or uni-directional probably is not much help. But if it is the "match" that is eating up all the CPU that's a different story. 2) If you're short on link capacity then bi-directional flows may help. But you only get a max of 50%. There are other aggregation schemes that may be of more use to reduce the about of data sent. 3) The other important thing bi-directional flows gives is a semantic relationship between 2 uni-directional flows. It may be difficult or impossible to calculate that relationship down stream. For example, someone mentioned looking at DNS packet contents. You wont have access to that down stream. > When > the routing engine is under stress, the exporter will not have some > magic reserve of processing power to do "flow matching", but if it resorts > to uni-directional flow reporting, it exacerbates the situation by > consuming more bandwidth, leaving only a fall-back to packet sampling. > And, if that happens, what kind of impact does it have on bi-directional > flow reporting? > > -Robert -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Fri Jan 11 17:35:31 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA02170 for ; Fri, 11 Jan 2002 17:35:31 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16PA0e-0006pQ-00 for ipfix-list@mil.doit.wisc.edu; Fri, 11 Jan 2002 16:18:48 -0600 Received: from relay1.pair.com ([209.68.1.20] helo=relay.pair.com) by mil.doit.wisc.edu with smtp (Exim 3.13 #1) id 16PA0b-0006pE-00 for ipfix-req@net.doit.wisc.edu; Fri, 11 Jan 2002 16:18:45 -0600 Received: (qmail 53802 invoked from network); 11 Jan 2002 22:18:44 -0000 Received: from pool-162-83-252-213.ny5030.east.verizon.net (HELO sphynx) (162.83.252.213) by relay1.pair.com with SMTP; 11 Jan 2002 22:18:44 -0000 X-pair-Authenticated: 162.83.252.213 Reply-To: From: "Carter Bullard" To: , "'Sebastian Zander'" Cc: "'req'" Subject: RE: [ipfix-req] unidirection vs. bidirectional flows Date: Fri, 11 Jan 2002 17:18:35 -0500 Organization: QoSient, LLC Message-ID: <5C8959A16A71B449AE793CF52FBBED66018FE2@ptah.newyork.qosient.com> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook, Build 10.0.3416 Importance: Normal In-Reply-To: <3C3F2660.F03D3D19@riverstonenet.com> X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Gentle people, I completely agree with Paul. Bi-directionality is just one relationship we'll want to report in our flow data model. The whole point is to not find the simplest model, but rather a working model that optimizes a few key points. I believe that transport efficiency is one of the more important ones. I don't have any problem with uni-directional flow descriptors, they happen to also work very well for bi-directional flow metric reporting. But IPFIX MUST be able to efficiently report the additional attributes of the flow, such as return status and metrics. Duplicating an entire IPFIX flow record, and a binding record would not be the most efficient way of doing it. Carter Carter Bullard QoSient, LLC 300 E. 56th Street, Suite 18K New York, New York 10022 carter@qosient.com Phone +1 212 588-9133 Fax +1 212 588-9134 http://qosient.com > -----Original Message----- > From: majordomo listserver > [mailto:majordomo@mil.doit.wisc.edu] On Behalf Of > calato@riverstonenet.com > Sent: Friday, January 11, 2002 12:53 PM > To: Sebastian Zander > Cc: req > Subject: Re: [ipfix-req] unidirection vs. bidirectional flows > > > Sebastian Zander wrote: > > > > I am trying to summarize the discussion from the requirements > > viewpoint: > > > > Do we need to add a requirement on the flow model to the > requirements > > draft? > > > > If yes then looking at the discussion it seems that there is rough > > consensus > > on: MUST support uni-directional, additionally MAY/SHOULD(?) support > > bi-directional if the exporter supports flow matching. > > I think it is a MUST on uni-directional. I think > bi-directional is a data efficiency question. > > If I as the exporter "know" these 2 flows > are bi-directional, then I want to do 2 things > 1) save space by reporting information once and > 2) indicate the relationship between these > uni-directional flows. > > There may be lots of relationships we want be able > to represent. Such as, these flows are part of the > same multi-cast. I don't see why we need to call > out bi-directional as a special case in the > requirements. > > As for the data model, there are lots of ways to > save space and we should look at that as a topic > of its own. > > As for the relationship issue, again, we need to look > at that on its own. Some relationships we will want > in the IPFIX data and some we will want to calculate > down stream. > > > Paul > > > > > > > > Correct? > > > > Cheers, > > > > Sebastian > > > > -- > > Sebastian Zander E-mail: zander@fokus.fhg.de > > FhI FOKUS / Global Networking (GloNe) Tel: +49-30-3463-7287 > > Kaiserin-Augusta-Allee 31 Fax: +49-30-3463-8287 > > D-10589 Berlin, Germany > www.fokus.fhg.de/usr/sebastian.zander > > > > -- > > Help > mailto:majordomo@net.doit.wisc.edu and say "help" in message body > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe > > ipfix" in message body > > Archive http://ipfix.doit.wisc.edu/archive/ > > -- > Help mailto:majordomo@net.doit.wisc.edu and say "help" > in message body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ > > -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Fri Jan 11 17:57:38 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA02545 for ; Fri, 11 Jan 2002 17:57:32 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16PAMV-0007N1-00 for ipfix-list@mil.doit.wisc.edu; Fri, 11 Jan 2002 16:41:23 -0600 Received: from relay1.pair.com ([209.68.1.20] helo=relay.pair.com) by mil.doit.wisc.edu with smtp (Exim 3.13 #1) id 16PAMT-0007Ms-00 for ipfix-req@net.doit.wisc.edu; Fri, 11 Jan 2002 16:41:21 -0600 Received: (qmail 56464 invoked from network); 11 Jan 2002 22:41:19 -0000 Received: from pool-162-83-252-213.ny5030.east.verizon.net (HELO sphynx) (162.83.252.213) by relay1.pair.com with SMTP; 11 Jan 2002 22:41:19 -0000 X-pair-Authenticated: 162.83.252.213 Reply-To: From: "Carter Bullard" To: "'Robert Lowe'" , Cc: Subject: RE: [ipfix-req] unidirection vs. bidirectional flows Date: Fri, 11 Jan 2002 17:41:11 -0500 Organization: QoSient, LLC Message-ID: <5C8959A16A71B449AE793CF52FBBED66018FE4@ptah.newyork.qosient.com> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook, Build 10.0.3416 Importance: Normal In-Reply-To: <3C3F5091.E77F3B6B@lawrence.edu> X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Hey Robert, Rather than thinking that flow matching is an option for a flow monitor that can be turned on and off, why not consider the existing flow monitors that do bi-directional flow matching as there only function. The problem becomes "the monitor has bi-directional flow data, and it needs to transport it using IPFIX." Should this monitor be burdened with having to generate two IPFIX records, when one could easily do the trick? Why throw away the processing that the bi-directional monitor spent realizing the relationship? Carter Carter Bullard QoSient, LLC 300 E. 56th Street, Suite 18K New York, New York 10022 carter@qosient.com Phone +1 212 588-9133 Fax +1 212 588-9134 http://qosient.com > -----Original Message----- > From: majordomo listserver > [mailto:majordomo@mil.doit.wisc.edu] On Behalf Of Robert Lowe > Sent: Friday, January 11, 2002 3:53 PM > To: calato@riverstonenet.com > Cc: ipfix-req@net.doit.wisc.edu > Subject: Re: [ipfix-req] unidirection vs. bidirectional flows > > > calato@riverstonenet.com wrote: > > > > I think one of Carter's points though was efficiency. > > So, to use your structure, instead of reporting this... > > > > > +------------------------------------------------------------- > ---------+ > > | FIR: (unicast, "normal" TCP session) > | > > > |+------------------------------------------------------------ > --------+| > > ||FLOW: 10.42.69.10:1024 -> 10.10.10.69:22 TCP > || > > > |+------------------------------------------------------------ > --------+| > > ||FLOW: 10.10.10.69:22 -> 10.42.69.10:1024 TCP > || > > > |+------------------------------------------------------------ > --------+| > > > +------------------------------------------------------------- > ---------+ > > > > You would be more efficient by reporting somehthing like this. > > > > > +------------------------------------------------------------- > -------+| > > |FLOW: 10.42.69.10:1024 <-> 10.10.10.69:22 TCP > | > > > +------------------------------------------------------------- > -------+| > > > > In either case the collector needs to know how to deal > > with the information. In the first case, if the collector > > wants bi-directional it can combine the 2 records. If it > > wants uni-directional ignore the FIR part. > > A couple of simple-minded questions as a lurker here... > > If you do report a bi-directional flow, do you turn reported > elements, > e.g. bytes or packets, into tuples within the flow record, one for > from-src and the other for from-dest? Or do you always assume an > aggregate? Assuming an aggregate is simpler, but not ideal. > > > In the seconds case, if it wants bi-directional, just forward > > on. If it wants uni-directional, create 2 records and pass them > > along. > > > > You made this comment... > > > > > The advantage of this [the FIR] would be that the embedded flow > > > records could be understood by either an unidirectional or > > > bidirectionally aware collecter. The encapsulating "FIR" (that > > > binds the two together) could essentially turn into a "no-op" for > > > collectors that don't need to understand any application-level > > > relationship between those flows, for the given application. > > > > While it may make things slightly simpler in the collector the > > bandwidth cost is significant when you think of millions of > > flows per day. In the second case above. the cost of turning > > one record into 2 at the collector, would be less than the > > bandwidth cost of the frist case, IMHO. > > Given that the reporting method will have a bandwidth impact, does an > exporter export anything about the traffic it generates? Or does it > store aggregate statistics that can be polled by the collector? > > A flexible approach allows for differing designs where processing > can be distributed between the exporter and the collector to varying > extents, but since bandwidth consumed and processing power are linked, > I'm not sure I understand where bi-directional reporting fits. When > the routing engine is under stress, the exporter will not have some > magic reserve of processing power to do "flow matching", but > if it resorts > to uni-directional flow reporting, it exacerbates the situation by > consuming more bandwidth, leaving only a fall-back to packet > sampling. > And, if that happens, what kind of impact does it have on > bi-directional > flow reporting? > > -Robert > > -- > Help mailto:majordomo@net.doit.wisc.edu and say "help" > in message body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ > > -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Fri Jan 11 18:39:35 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id SAA02932 for ; Fri, 11 Jan 2002 18:39:35 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16PB27-0000no-00 for ipfix-list@mil.doit.wisc.edu; Fri, 11 Jan 2002 17:24:23 -0600 Received: from weird-brew.cisco.com ([144.254.15.118] helo=strange-brew.cisco.com) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 16PB25-0000nT-00 for ipfix@net.doit.wisc.edu; Fri, 11 Jan 2002 17:24:21 -0600 Received: from cisco.com (bclaise-isdn-home2.cisco.com [10.49.4.219]) by strange-brew.cisco.com (8.8.8+Sun/8.8.8) with ESMTP id AAA23741; Sat, 12 Jan 2002 00:23:45 +0100 (MET) Message-ID: <3C3F7400.DBEAABE0@cisco.com> Date: Sat, 12 Jan 2002 00:23:45 +0100 From: Benoit Claise X-Mailer: Mozilla 4.76 [en]C-CCK-MCD (Windows NT 5.0; U) X-Accept-Language: en MIME-Version: 1.0 To: "K.C. Norseth" CC: Ganesh Sadasivan , Simon Leinen , ipfix@net.doit.wisc.edu Subject: Re: [ipfix] IPFIX Requirement presentation References: <20011214150236.A29404@doit.wisc.edu><3C1BDF61.B0BE8081@cisco.com> <001901c19805$a6ff6b60$850f880a@slc252750> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit K.C. > In the case of an etxternal probe where there is only one interface and the > probe is hooked to a mirrored port or bridge, the place where the probe is > connected, ther is no incomoing or outgoing interface. What would the > interface be that would satisfy MUST? In that case, you separate the flows automatically per interface, as there is only one interface. So I should say that you are automatically compliant with the following paragraph: 3.1. Interfaces The measuring device MUST be able to separate flows by the incoming interface or by the outgoing interface or by both of them. Now, whether you report the interface or not is something else. We agree it doesn't make sense here to export the value! In both cases (reporting or not the interface), you are compliant with the following paragraph, as long as your IPFIX implementation on the probe allows you to report the interface: 5.1. Information Model The information model for a flow measurement report is the list of attributes of a flow to be contained in the report (including the semantics of the attributes). This section lists attributes a measuring device MUST or MAY be able to report. This does not imply that a report MUST contain all REQUIRED attributes, but that it MUST be possible to configure the device in a way that all of the REQUIRED attributes are contained in a report for each measured flow. ... The measuring device MUST be able to report the following attributes for each measured flow: ... 18. unique identifier of the observation point ... Regards, Benoit > > > K.C. > > ----- Original Message ----- > From: Simon Leinen > To: Ganesh Sadasivan > Cc: > Sent: Sunday, January 06, 2002 5:56 AM > Subject: Re: [ipfix] IPFIX Requirement presentation > > > >>>>> "gs" == Ganesh Sadasivan writes: > > > I want to bring up this point which I had raised on the > > > requirement spec. at the WG meeting. > > > It is from slide 5 on requirements where the flow definition has > > > been changed to include "incoming or outgoing interface" as a > > > MUST. > > > > The corresponding part of the requirements draft > > (ietf-ipfix-requirements-00) reads: > > > > 3.1. Interfaces > > > > The measuring device MUST be able to separate flows by the incoming > > interface or by the outgoing interface or by both of them. > > > > Note that this says "MUST be able to separate", not "MUST separate". > > Therefore I think your reasoning below doesn't apply: > > > > > There are cases like a SP is interested in getting all the > > > packest that go between 2 ASs. This does not have any strict > > > relevance to interface. An AS can be sourced by Multiple > > > interfaces in a router. Similar would be case if somebody > > > considers to define a flow based on other packet fields like > > > src/dst IP etc. In the general scheme of flow defintion ,I do not > > > see why there should be a restriction to include the interfaces > > > as a MUST. Claiming this as "most of the apps need the interfaces > > > today" is not a valid argument as this model is used to serve for > > > future apps. too. Therefore I request the WG to consider > > > removing this restriction. > > > > It is probably the case for *every* flow attribute that one can always > > come up with a sensible application that doesn't care about it. So it > > makes looks reasonable to make all flow attributes optional, in the > > sense that when an application isn't interested in them, the exporter > > doesn't have to send them in order to avoid waste of resources. This > > could be a requirement for the Flow Information Export protocol. > > > > But it also makes sense to make the *ability* of reporting certain > > attributes mandatory, so that applications can rely on them being > > available from all IPFIX compliant exporters. > > > > Personally I find the input and output interface sufficiently > > important to make their support mandatory. In particular the input > > interface is essential when one wants to do analysis of large-scale > > routing asymmetries. Input and output interfaces are also extremely > > useful when one collects flows at different points in the networks and > > wants to avoid counting the same flows twice. And the input interface > > cannot be derived from the source address and routing information, as > > can/must be done with e.g. the "source netmask", "upstream AS" or > > "source AS". > > > > Regards, > > -- > > Simon Leinen simon@babar.switch.ch > > SWITCH http://www.switch.ch/misc/leinen/ > > > > Computers hate being anthropomorphized. > > > > -- > > Help mailto:majordomo@net.doit.wisc.edu and say "help" in message > body > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > "unsubscribe ipfix" in message body > > Archive http://ipfix.doit.wisc.edu/archive/ > > > > -- > Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Tue Jan 15 18:05:31 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id SAA15769 for ; Tue, 15 Jan 2002 18:05:31 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16QcMo-00071k-00 for ipfix-list@mil.doit.wisc.edu; Tue, 15 Jan 2002 16:47:42 -0600 Received: from zrc2s0jx.nortelnetworks.com ([47.103.122.112]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 16QcMl-00071J-00 for ipfix-arch@net.doit.wisc.edu; Tue, 15 Jan 2002 16:47:39 -0600 Received: from zsc4c000.us.nortel.com (zsc4c000.us.nortel.com [47.81.138.47]) by zrc2s0jx.nortelnetworks.com (Switch-2.2.0/Switch-2.2.0) with ESMTP id g0FMdRh02727; Tue, 15 Jan 2002 16:39:27 -0600 (CST) Received: by zsc4c000.us.nortel.com with Internet Mail Service (5.5.2653.19) id ; Tue, 15 Jan 2002 14:39:26 -0800 Message-ID: <4F973E944951D311B6660008C7917CF0084AB2A2@zsc4c008.us.nortel.com> From: "Reinaldo Penno" To: "'calato@riverstonenet.com'" Cc: "'Ganesh Sadasivan'" , "'arch'" Subject: RE: [ipfix-arch] configuration information. Date: Tue, 15 Jan 2002 14:39:16 -0800 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01C19E15.76CF6650" Precedence: bulk Sender: majordomo listserver This message is in MIME format. Since your mail reader does not understand this format, some or all of this message may not be legible. ------_=_NextPart_001_01C19E15.76CF6650 Content-Type: text/plain; charset="iso-8859-1" ..... > > Great, then the device reports that all its flows > may overlap. I'm no worse off than not saying anything. > But there are many devices that have relatively simple > counting mechanisms and the extra information will > help down stream applications do some analysis that may > otherwise not be doable. > > > > okay. So, can we say that "in the absence of explicit signalling from the device, the collector MUST consider that all flows reported can overlap. In the case the device reports overlapping between certain flows, the collectror SHOULD take this into consideration, but MUST NOT assume that other flows will not overlap" is it good enough? Comments? -Reinaldo ------_=_NextPart_001_01C19E15.76CF6650 Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable RE: [ipfix-arch] configuration information.

<snip>...<snip>..


>
>       Great, then the = device reports that all its flows
>       may overlap. I'm = no worse off than not saying anything.
>       But there are = many devices that have relatively simple
>       counting = mechanisms and the extra information will
>       help down stream = applications do some analysis that may
>       otherwise not be = doable.
>
>
> >

okay. So, can we say that

"in the absence of explicit signalling from the = device, the collector MUST consider that all flows reported can = overlap. In the case the device reports overlapping between certain = flows, the collectror SHOULD take this into consideration, but MUST NOT = assume that other flows will not overlap"

is it good enough? Comments?

-Reinaldo

------_=_NextPart_001_01C19E15.76CF6650-- -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Wed Jan 23 15:22:32 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA23864 for ; Wed, 23 Jan 2002 15:22:31 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16TTZD-00033c-00 for ipfix-list@mil.doit.wisc.edu; Wed, 23 Jan 2002 14:00:20 -0600 Received: from mailhost.auckland.ac.nz ([130.216.1.4]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 16TTZA-00033K-00 for ipfix@net.doit.wisc.edu; Wed, 23 Jan 2002 14:00:16 -0600 Received: from auckland.ac.nz (root@ccu1.auckland.ac.nz [130.216.3.1]) by mailhost.auckland.ac.nz (8.9.2/8.9.2/8.9.2-ua) with ESMTP id JAA20042 for ; Thu, 24 Jan 2002 09:00:10 +1300 (NZDT) Message-Id: <200201232000.JAA20042@mailhost.auckland.ac.nz> Date: Wed, 23 Jan 2002 13:55:43 -0800 (PST) From: n.brownlee@auckland.ac.nz Subject: [ipfix] Comments on 'goals of IPFX WG' To: ipfix@net.doit.wisc.edu In-Reply-To: <20020116143008.B18433@doit.wisc.edu> MIME-Version: 1.0 Content-Type: TEXT/plain; charset=us-ascii Precedence: bulk Sender: majordomo listserver Hello all: THe IPFIX working group is well under way, with our three teams busy on finishing the 'requirements' document, and beginning the 'architecture' and 'data model' documents. Watching the threads on the various sub-mailing lists, especially the one on bi-directional vs uni-directional flows, it occurred to me that we should pause a moment and review the goals of the WG. Here are my opinions on this - who knows, maybe they could make some 'introductory text' for one of the documents. Anyway, I'd appreciate any comments, suggestions for improvements, etc. .. 1. IPFIX is a standards effort aimed at producing a standard method for exporting flow information. The key to this is to develop an easily-understandable, generalised, extensible data model for flow information, together with a clearly-defined method for moving flow information from an exporter to a collector. The WG's goals are to produce three RFCs: 'IPFIX Requirements,' 'IPFIX Data Model' and'IPFIX Architecture.' 2. IPFIX is *not* trying to standardise a flow meter, i.e. a network entity which measures flows - we already have RFCs which do that, e.g. those for RTFM (2720 .. 2724). Existing widely-used implementations can remain different, but might perhaps be extended to report flow information in the IPFIX lingua franca. 3. Instead, we recognise that network equipment vendors already have flow measurement systems implemented as part of their equipment. Our goal is to make the IPFIX flow information export system so good technically that all vendors will be happy to implement it, for the benefit of all users of the data. A useful side-effect will be that IPFIX will make it easier to build and maintain interoperable flow data collecting systems. Viewed in this light, the only difference between uni- and bi-directional flows is that bi-directional flows need attributes (e.g. packet and byte counters) for each direction, rather than for only one direction. So long as these attributes are supported in the IPFIX data model, they can be exported by bi-directional meters and recognised by IPFIX collectors. In other words, flow information *collectors* need to implement *all* the IPFIX flow attributes, but *exporters* need only implement a subset of them. In the same way, a collector will certainly need to know the attributes describing each flow (e.g. its source and destination IP addresses, port numbers, etc., etc.), so the data model must include these attributes. The balance between 'data model' - describing all the attributes used in describing flows, reporting packet and byte counts, whether information is to be pushed to a collector or pulled by it, and so on - and the 'architecture' - describing how this is done, using whatever transport protocol(s) the WG eventally decides on - is going to require quite a lot of our design effort! Cheers, Nevil ----------------------------------------------------------------------- Nevil Brownlee Director, Technology Development Phone: +64 9 373 7599 x8941 ITSS, The University of Auckland FAX: +64 9 373 7021 Private Bag 92019, Auckland, New Zealand -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Wed Jan 23 16:10:31 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA25534 for ; Wed, 23 Jan 2002 16:10:31 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16TUP2-00047C-00 for ipfix-list@mil.doit.wisc.edu; Wed, 23 Jan 2002 14:53:52 -0600 Received: from zrc2s0jx.nortelnetworks.com ([47.103.122.112]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 16TUP0-00046Y-00 for ipfix-arch@net.doit.wisc.edu; Wed, 23 Jan 2002 14:53:50 -0600 Received: from zsc4c000.us.nortel.com (zsc4c000.us.nortel.com [47.81.138.47]) by zrc2s0jx.nortelnetworks.com (Switch-2.2.0/Switch-2.2.0) with ESMTP id g0NKrIQ19221 for ; Wed, 23 Jan 2002 14:53:18 -0600 (CST) Received: by zsc4c000.us.nortel.com with Internet Mail Service (5.5.2653.19) id ; Wed, 23 Jan 2002 12:53:17 -0800 Message-ID: <4F973E944951D311B6660008C7917CF00864AB2C@zsc4c008.us.nortel.com> From: "Reinaldo Penno" To: "'arch'" Subject: [ipfix-arch] Echo Date: Wed, 23 Jan 2002 12:53:15 -0800 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01C1A44F.FA661410" Precedence: bulk Sender: majordomo listserver This message is in MIME format. Since your mail reader does not understand this format, some or all of this message may not be legible. ------_=_NextPart_001_01C1A44F.FA661410 Content-Type: text/plain; charset="iso-8859-1" test... did something happen to this list? -Reinaldo ------_=_NextPart_001_01C1A44F.FA661410 Content-Type: text/html; charset="iso-8859-1" [ipfix-arch] Echo

test...

did something happen to this list?

-Reinaldo

------_=_NextPart_001_01C1A44F.FA661410-- -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Wed Jan 23 16:38:24 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA26827 for ; Wed, 23 Jan 2002 16:38:19 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16TUp5-0004hf-00 for ipfix-list@mil.doit.wisc.edu; Wed, 23 Jan 2002 15:20:47 -0600 Received: from sj-msg-core-1.cisco.com ([171.71.163.11]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 16TUp3-0004gw-00 for ipfix@net.doit.wisc.edu; Wed, 23 Jan 2002 15:20:45 -0600 Received: from mira-sjcd-1.cisco.com (mira-sjcd-1.cisco.com [171.69.43.44]) by sj-msg-core-1.cisco.com (8.11.3/8.9.1) with ESMTP id g0NLKEL22272; Wed, 23 Jan 2002 13:20:14 -0800 (PST) Received: from cisco.com (dhcp-171-71-137-54.cisco.com [171.71.137.54]) by mira-sjcd-1.cisco.com (Mirapoint) with ESMTP id AAY20886; Wed, 23 Jan 2002 13:20:21 -0800 (PST) Message-ID: <3C4F2B14.21FE106F@cisco.com> Date: Wed, 23 Jan 2002 13:28:52 -0800 From: Ganesh Sadasivan X-Mailer: Mozilla 4.76 [en]C-CCK-MCD (Windows NT 5.0; U) X-Accept-Language: en MIME-Version: 1.0 To: n.brownlee@auckland.ac.nz CC: ipfix@net.doit.wisc.edu Subject: Re: [ipfix] Comments on 'goals of IPFX WG' References: <200201232000.JAA20042@mailhost.auckland.ac.nz> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Hi Nevil, n.brownlee@auckland.ac.nz wrote: > Hello all: > > THe IPFIX working group is well under way, with our three teams busy > on finishing the 'requirements' document, and beginning the > 'architecture' and 'data model' documents. > > Watching the threads on the various sub-mailing lists, especially the > one on bi-directional vs uni-directional flows, it occurred to me that > we should pause a moment and review the goals of the WG. Here are my > opinions on this - who knows, maybe they could make some 'introductory > text' for one of the documents. Anyway, I'd appreciate any comments, > suggestions for improvements, etc. .. > > 1. IPFIX is a standards effort aimed at producing a standard method > for exporting flow information. The key to this is to develop > an easily-understandable, generalised, extensible data model for > flow information, together with a clearly-defined method for > moving flow information from an exporter to a collector. The > WG's goals are to produce three RFCs: 'IPFIX Requirements,' > 'IPFIX Data Model' and'IPFIX Architecture.' > > 2. IPFIX is *not* trying to standardise a flow meter, i.e. a network > entity which measures flows - we already have RFCs which do that, > e.g. those for RTFM (2720 .. 2724). Existing widely-used > implementations can remain different, but might perhaps be extended > to report flow information in the IPFIX lingua franca. I agree with you to this point. The question I have is to what extent does the IPFIX arch. focus on moving the method of flow selection and relationship between the different collected types of flows w.r.t an observation point to the collector? The collector needs this information in order to interpret the flow data it receives. Thanks Ganesh > > > 3. Instead, we recognise that network equipment vendors already have > flow measurement systems implemented as part of their equipment. > Our goal is to make the IPFIX flow information export system > so good technically that all vendors will be happy to implement > it, for the benefit of all users of the data. A useful side-effect > will be that IPFIX will make it easier to build and maintain > interoperable flow data collecting systems. > > Viewed in this light, the only difference between uni- and > bi-directional flows is that bi-directional flows need attributes (e.g. > packet and byte counters) for each direction, rather than for only one > direction. So long as these attributes are supported in the IPFIX data > model, they can be exported by bi-directional meters and recognised by > IPFIX collectors. In other words, flow information *collectors* need to > implement *all* the IPFIX flow attributes, but *exporters* need only > implement a subset of them. > > In the same way, a collector will certainly need to know the attributes > describing each flow (e.g. its source and destination IP addresses, port > numbers, etc., etc.), so the data model must include these attributes. > > The balance between 'data model' - describing all the attributes used > in describing flows, reporting packet and byte counts, whether > information is to be pushed to a collector or pulled by it, and so on - > and the 'architecture' - describing how this is done, using whatever > transport protocol(s) the WG eventally decides on - is going to require > quite a lot of our design effort! > > Cheers, Nevil > > ----------------------------------------------------------------------- > Nevil Brownlee Director, Technology Development > Phone: +64 9 373 7599 x8941 ITSS, The University of Auckland > FAX: +64 9 373 7021 Private Bag 92019, Auckland, New Zealand > > -- > Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Sun Jan 27 18:05:02 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id SAA04455 for ; Sun, 27 Jan 2002 18:04:57 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16UxjH-0003pN-00 for ipfix-list@mil.doit.wisc.edu; Sun, 27 Jan 2002 16:24:51 -0600 Received: from mgw-dax2.ext.nokia.com ([63.78.179.217]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 16UxjE-0003pE-00 for ipfix@net.doit.wisc.edu; Sun, 27 Jan 2002 16:24:48 -0600 Received: from davir02nok.americas.nokia.com (davir02nok.americas.nokia.com [172.18.242.85]) by mgw-dax2.ext.nokia.com (Switch-2.1.0/Switch-2.1.0) with ESMTP id g0RMQSQ14439 for ; Sun, 27 Jan 2002 16:26:28 -0600 (CST) Received: from daebh001.NOE.Nokia.com (unverified) by davir02nok.americas.nokia.com (Content Technologies SMTPRS 4.2.5) with ESMTP id ; Sun, 27 Jan 2002 16:24:47 -0600 Received: from bsebe001.NOE.Nokia.com ([172.19.160.13]) by daebh001.NOE.Nokia.com with Microsoft SMTPSVC(5.0.2195.2966); Sun, 27 Jan 2002 16:24:47 -0600 X-MimeOLE: Produced By Microsoft Exchange V6.0.5762.3 content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Subject: RE: [ipfix] Comments on 'goals of IPFX WG' Date: Sun, 27 Jan 2002 17:24:46 -0500 Message-ID: Thread-Topic: [ipfix] Comments on 'goals of IPFX WG' Thread-Index: AcGkVRumofowXZg0SvyjQhaNgUGzQgDKpplw From: To: , Cc: X-OriginalArrivalTime: 27 Jan 2002 22:24:47.0071 (UTC) FILETIME=[6D5FF6F0:01C1A781] Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by ietf.org id SAA04455 Hello Ganesh, See my inline comments. > >I agree with you to this point. The question I have is to what extent >does the IPFIX arch. focus on moving the method of flow selection and >relationship between the different collected types of flows w.r.t an >observation point to the collector? The collector needs this >information >in order to interpret the flow data it receives. > Architecture should describe entities ( like collector etc ) what was already defined in the IPFIX requirement document in the form of a big-picture. The requirement document has covered both architectural requirement and protocol requirements, in the IPFIX arch.document we can explicitly describe bring out and explain the architecture. Apart from this we can describe the different scenarios and can state what is the scope of the IPFIX. Regards Ramg -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Mon Jan 28 15:51:52 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA08395 for ; Mon, 28 Jan 2002 15:51:52 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16VIQ7-0001Lt-00 for ipfix-list@mil.doit.wisc.edu; Mon, 28 Jan 2002 14:30:27 -0600 Received: from c001-h011.c001.snv.cp.net ([209.228.32.125] helo=c001.snv.cp.net) by mil.doit.wisc.edu with smtp (Exim 3.13 #1) id 16VIQ4-0001KX-00 for ipfix-arch@net.doit.wisc.edu; Mon, 28 Jan 2002 14:30:24 -0600 Received: (cpmta 21003 invoked from network); 28 Jan 2002 12:29:53 -0800 Date: 28 Jan 2002 12:29:53 -0800 Message-ID: <20020128202953.21002.cpmta@c001.snv.cp.net> X-Sent: 28 Jan 2002 20:29:53 GMT Received: from [12.25.1.128] by mail.norseth.com with HTTP; 28 Jan 2002 12:29:53 PST Content-Type: text/plain Content-Disposition: inline Mime-Version: 1.0 To: ipfix-arch@net.doit.wisc.edu, ipfix-data@net.doit.wisc.edu From: kcn@norseth.com Cc: ipfix-chairs@net.doit.wisc.edu X-Mailer: Web Mail 3.9.3.5 Subject: [ipfix-arch] RE: Drafts and conference X-Sent-From: kcn@norseth.com Precedence: bulk Sender: majordomo listserver Folks, I am scheduling the meeting for 11:00am PST on Thursday. I will get the conference info out and an agenda when I get the confirmation. This meeting is not where we will discuss the in-depth technical aspects of the documents, but more defining the documents and making assignments so we can move onward. If you have any suggestions on the agenda or comments, send them to me and I will put them on the agenda that will go out. If there are any problems, let me know. Please let me know who is going to participate so I can have enough lines. K.C. On Mon, 28 January 2002, ram.gopal@nokia.com wrote: > > Hello KC, > > since you agreed to prepare the agenda please take the following items in the agenda. > > > (1) Table of content for Architecture document > (2) Table of Content for Data Model > (3) Identify the work items and prepare initial table of content > (4) Commitment from individual to contribute to relevant sections > > > We should not waste too much time on discussing the technical topics - teleconferencing is not the place. > > We should get commitment from individuals and they should circulate to the mailist for comments. > Finally one person should able to compile the document. > > > As u mentioned in your email that you are planning to host the conference - i do not have any problem in hosting the conference, let me know incase if you want me to do so > > > > Regards > Ramg -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Mon Jan 28 15:51:55 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA08406 for ; Mon, 28 Jan 2002 15:51:54 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16VIQ7-0001Lv-00 for ipfix-list@mil.doit.wisc.edu; Mon, 28 Jan 2002 14:30:27 -0600 Received: from c001-h011.c001.snv.cp.net ([209.228.32.125] helo=c001.snv.cp.net) by mil.doit.wisc.edu with smtp (Exim 3.13 #1) id 16VIQ4-0001KW-00 for ipfix-data@net.doit.wisc.edu; Mon, 28 Jan 2002 14:30:24 -0600 Received: (cpmta 21003 invoked from network); 28 Jan 2002 12:29:53 -0800 Date: 28 Jan 2002 12:29:53 -0800 Message-ID: <20020128202953.21002.cpmta@c001.snv.cp.net> X-Sent: 28 Jan 2002 20:29:53 GMT Received: from [12.25.1.128] by mail.norseth.com with HTTP; 28 Jan 2002 12:29:53 PST Content-Type: text/plain Content-Disposition: inline Mime-Version: 1.0 To: ipfix-arch@net.doit.wisc.edu, ipfix-data@net.doit.wisc.edu From: kcn@norseth.com Cc: ipfix-chairs@net.doit.wisc.edu X-Mailer: Web Mail 3.9.3.5 Subject: [ipfix-data] RE: Drafts and conference X-Sent-From: kcn@norseth.com Precedence: bulk Sender: majordomo listserver Folks, I am scheduling the meeting for 11:00am PST on Thursday. I will get the conference info out and an agenda when I get the confirmation. This meeting is not where we will discuss the in-depth technical aspects of the documents, but more defining the documents and making assignments so we can move onward. If you have any suggestions on the agenda or comments, send them to me and I will put them on the agenda that will go out. If there are any problems, let me know. Please let me know who is going to participate so I can have enough lines. K.C. On Mon, 28 January 2002, ram.gopal@nokia.com wrote: > > Hello KC, > > since you agreed to prepare the agenda please take the following items in the agenda. > > > (1) Table of content for Architecture document > (2) Table of Content for Data Model > (3) Identify the work items and prepare initial table of content > (4) Commitment from individual to contribute to relevant sections > > > We should not waste too much time on discussing the technical topics - teleconferencing is not the place. > > We should get commitment from individuals and they should circulate to the mailist for comments. > Finally one person should able to compile the document. > > > As u mentioned in your email that you are planning to host the conference - i do not have any problem in hosting the conference, let me know incase if you want me to do so > > > > Regards > Ramg -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Mon Jan 28 16:15:35 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA09041 for ; Mon, 28 Jan 2002 16:15:35 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16VIq3-0001xI-00 for ipfix-list@mil.doit.wisc.edu; Mon, 28 Jan 2002 14:57:15 -0600 Received: from c001-h010.c001.snv.cp.net ([209.228.32.124] helo=c001.snv.cp.net) by mil.doit.wisc.edu with smtp (Exim 3.13 #1) id 16VIq0-0001w6-00 for ipfix-arch@net.doit.wisc.edu; Mon, 28 Jan 2002 14:57:12 -0600 Received: (cpmta 22409 invoked from network); 28 Jan 2002 12:56:39 -0800 Date: 28 Jan 2002 12:56:39 -0800 Message-ID: <20020128205639.22408.cpmta@c001.snv.cp.net> X-Sent: 28 Jan 2002 20:56:39 GMT Received: from [12.25.1.128] by mail.norseth.com with HTTP; 28 Jan 2002 12:56:38 PST Content-Type: text/plain Content-Disposition: inline Mime-Version: 1.0 To: kcn@norseth.com From: kcn@norseth.com Cc: ipfix-arch@net.doit.wisc.edu, ipfix-data@net.doit.wisc.edu, ipfix-chairs@net.doit.wisc.edu X-Mailer: Web Mail 3.9.3.5 Subject: [ipfix-arch] Re: [ipfix-data] RE: Drafts and conference X-Sent-From: kcn@norseth.com Precedence: bulk Sender: majordomo listserver Folks, Conference call information: U.S. : 866-66-4CALL (2255) International: 603-337-1770 PassCode : 7997 Contact: K.C. Norseth kcn@norseth.com knorseth@enterasys.com Phone: 801.887.9823 Agenda coming tomorrow, stay tuned. If you have anything to add, let me know ASAP. K.C. On Mon, 28 January 2002, kcn@norseth.com wrote: > > Folks, > > I am scheduling the meeting for 11:00am PST on Thursday. I will get the conference info out and an agenda when I get the confirmation. > > This meeting is not where we will discuss the in-depth technical aspects of the documents, but more defining the documents and making assignments so we can move onward. > > If you have any suggestions on the agenda or comments, send them to me and I will put them on the agenda that will go out. > > If there are any problems, let me know. Please let me know who is going to participate so I can have enough lines. > > K.C. > > On Mon, 28 January 2002, ram.gopal@nokia.com wrote: > > > > > Hello KC, > > > > since you agreed to prepare the agenda please take the following items in the agenda. > > > > > > (1) Table of content for Architecture document > > (2) Table of Content for Data Model > > (3) Identify the work items and prepare initial table of content > > (4) Commitment from individual to contribute to relevant sections > > > > > > We should not waste too much time on discussing the technical topics - teleconferencing is not the place. > > > > We should get commitment from individuals and they should circulate to the mailist for comments. > > Finally one person should able to compile the document. > > > > > > As u mentioned in your email that you are planning to host the conference - i do not have any problem in hosting the conference, let me know incase if you want me to do so > > > > > > > > Regards > > Ramg > > > > -- > Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Mon Jan 28 16:16:11 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA09079 for ; Mon, 28 Jan 2002 16:16:11 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16VIq3-0001xJ-00 for ipfix-list@mil.doit.wisc.edu; Mon, 28 Jan 2002 14:57:15 -0600 Received: from c001-h010.c001.snv.cp.net ([209.228.32.124] helo=c001.snv.cp.net) by mil.doit.wisc.edu with smtp (Exim 3.13 #1) id 16VIq0-0001w8-00 for ipfix-data@net.doit.wisc.edu; Mon, 28 Jan 2002 14:57:13 -0600 Received: (cpmta 22409 invoked from network); 28 Jan 2002 12:56:39 -0800 Date: 28 Jan 2002 12:56:39 -0800 Message-ID: <20020128205639.22408.cpmta@c001.snv.cp.net> X-Sent: 28 Jan 2002 20:56:39 GMT Received: from [12.25.1.128] by mail.norseth.com with HTTP; 28 Jan 2002 12:56:38 PST Content-Type: text/plain Content-Disposition: inline Mime-Version: 1.0 To: kcn@norseth.com From: kcn@norseth.com Cc: ipfix-arch@net.doit.wisc.edu, ipfix-data@net.doit.wisc.edu, ipfix-chairs@net.doit.wisc.edu X-Mailer: Web Mail 3.9.3.5 Subject: Re: [ipfix-data] RE: Drafts and conference X-Sent-From: kcn@norseth.com Precedence: bulk Sender: majordomo listserver Folks, Conference call information: U.S. : 866-66-4CALL (2255) International: 603-337-1770 PassCode : 7997 Contact: K.C. Norseth kcn@norseth.com knorseth@enterasys.com Phone: 801.887.9823 Agenda coming tomorrow, stay tuned. If you have anything to add, let me know ASAP. K.C. On Mon, 28 January 2002, kcn@norseth.com wrote: > > Folks, > > I am scheduling the meeting for 11:00am PST on Thursday. I will get the conference info out and an agenda when I get the confirmation. > > This meeting is not where we will discuss the in-depth technical aspects of the documents, but more defining the documents and making assignments so we can move onward. > > If you have any suggestions on the agenda or comments, send them to me and I will put them on the agenda that will go out. > > If there are any problems, let me know. Please let me know who is going to participate so I can have enough lines. > > K.C. > > On Mon, 28 January 2002, ram.gopal@nokia.com wrote: > > > > > Hello KC, > > > > since you agreed to prepare the agenda please take the following items in the agenda. > > > > > > (1) Table of content for Architecture document > > (2) Table of Content for Data Model > > (3) Identify the work items and prepare initial table of content > > (4) Commitment from individual to contribute to relevant sections > > > > > > We should not waste too much time on discussing the technical topics - teleconferencing is not the place. > > > > We should get commitment from individuals and they should circulate to the mailist for comments. > > Finally one person should able to compile the document. > > > > > > As u mentioned in your email that you are planning to host the conference - i do not have any problem in hosting the conference, let me know incase if you want me to do so > > > > > > > > Regards > > Ramg > > > > -- > Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Mon Jan 28 17:09:06 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA09977 for ; Mon, 28 Jan 2002 17:09:06 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16VJhn-0003AR-00 for ipfix-list@mil.doit.wisc.edu; Mon, 28 Jan 2002 15:52:47 -0600 Received: from dplonka by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16VJhl-0003AL-00; Mon, 28 Jan 2002 15:52:45 -0600 Date: Mon, 28 Jan 2002 15:52:45 -0600 From: Dave Plonka To: ipfix@net.doit.wisc.edu Subject: [ipfix] arch and data con call (was "Re: Drafts and conference") Message-ID: <20020128155245.A11086@doit.wisc.edu> Reply-To: plonka@doit.wisc.edu References: <20020128202953.21002.cpmta@c001.snv.cp.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 1.0.1i In-Reply-To: <20020128202953.21002.cpmta@c001.snv.cp.net>; from kcn@norseth.com on Mon, Jan 28, 2002 at 12:29:53PM -0800 Organization: UW-Madison, DoIT, Network Services X-VMS-Error: %SYSTEM-E-BREAK, breakpoint fault at PC=00000000, PS=00000416 X-Shakespearean-Insult: Thou weedy base-court giglet Precedence: bulk Sender: majordomo listserver On Mon, Jan 28, 2002 at 12:29:53PM -0800, kcn@norseth.com wrote: > Folks, > > I am scheduling the meeting for 11:00am PST on Thursday. > I will get the conference info out and an agenda when I get the confirmation. Thanks KC! > This meeting is not where we will discuss the in-depth technical > aspects of the documents, but more defining the documents and making > assignments so we can move onward. > > If you have any suggestions on the agenda or comments, send them to me > and I will put them on the agenda that will go out. > > If there are any problems, let me know. Please let me know who is > going to participate so I can have enough lines. I'll participate in the call. We'll save Nevil the cost of the call from NZ ;^). It'd be great to have a volunteer on the call for each group ("arch" and "data") to serve as a scribe and prepare notes/minutes during the call. Let us know you're doing so at the beginning of the call and you can send the notes to me afterward if you'd prefer that I summarize them for the public mailing list. KC - one bit for the agenda - I'm wondering if the "arch" and "data" authors might have requirements of the "requirements" document. Please think about what changes, if any, you need to ask of the requirements authors so that you have a suitable foundation (terminology or whatever) for your document. Dave -- plonka@doit.wisc.edu http://net.doit.wisc.edu/~plonka ARS:N9HZF Madison, WI -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Thu Jan 31 16:47:03 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA13563 for ; Thu, 31 Jan 2002 16:47:03 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16WOfm-0002Ne-00 for ipfix-list@mil.doit.wisc.edu; Thu, 31 Jan 2002 15:23:10 -0600 Received: from dplonka by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16WOfi-0002NK-00 for ipfix@net.doit.wisc.edu; Thu, 31 Jan 2002 15:23:06 -0600 Date: Thu, 31 Jan 2002 15:23:06 -0600 From: Dave Plonka To: ipfix@net.doit.wisc.edu Subject: conference call minutes (was "Re: [ipfix] arch and data con call") Message-ID: <20020131152306.C4294@doit.wisc.edu> Reply-To: plonka@doit.wisc.edu References: <20020128202953.21002.cpmta@c001.snv.cp.net> <20020128155245.A11086@doit.wisc.edu> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 1.0.1i In-Reply-To: <20020128155245.A11086@doit.wisc.edu>; from plonka@doit.wisc.edu on Mon, Jan 28, 2002 at 03:52:45PM -0600 Organization: UW-Madison, DoIT, Network Services X-VMS-Error: %SYSTEM-W-ALRCURTID, a default transaction is currently defined X-Shakespearean-Insult: Thou dankish half-faced giglet Precedence: bulk Sender: majordomo listserver IPFIX folks, Below, please find the minutes that I prepared following a conference call that was arranged to organize the IPFIX Internet Draft efforts. Thanks to KC Norseth and Enterasys for hosting the conference call. A list of those that participated follows the minutes below. ---------------------------------------------------------------------- The call began Thu Jan 31 ~1300 CST 2002. The conference call roughly followed an agenda prepared by KC Norseth. During the discussion various people volunteered to be responsible for contributing info on particular topics. Their names are noted below in context. These topics were discussed: *) Why are three proposed draft documents laid out this way? * IPFIX Requirements * IPFIX Architecture it was noted that protocol doesn't belong in Architecture * IPFIX Data Model Specifically, the question was asked why not a protocol document. (We discussed that we're not chartered to design a protocol, but to concentrate on standardizing existing practice.) There is some confusion about what items belong in the Architecture rather than Data model and vice versa. *) We discussed points suggested for the Architecture draft... The first was "Summary of Operation": It was noted that some things about the flow definition (such as what causes a flow to be instantiated, and what temporal characteristics they have) are important to specifying the IPFIX architecture. *) It was noted that the components or entities (and possibly interfaces between) are a key portion of the Architecture document. Capabilities negotiation was also mentioned for the Architecture. Kevin Zhang volunteered to work on this. It was mentioned that a work-in-progress Internet Draft document by Benoit and Ganesh does a good job of laying this out. It was also mentioned that the Architecture doc should lay out a number of architecture/component "scenarios". It was discussed as to whether or not the Requirements document should specify the flow definition. There was not support voiced for making it more specific. There were other kudos for Benoit's & Ganesh's document for its description of the flow "concept". *) Message formats It was noted that "messages formats" mostly belongs in a protocol specification so may be out of scope. However, it was noted that the items of information would be necessary for the Architecture specification. Some folks referred to this as the "information model". KC Norseth, Paul Calato, and Simon Leinen volunteered to work on specifying this information. It was noted that the handshaking or interfaces between components does belong in the Architecture. It was considered whether or not the "information model" belongs in architecture or data model. It was suggested that perhaps it would be best to focus on enumerating the important topics, and fit them into the documents afterward. This idea was supported by others as well. Kevin Zhan, Ram Gopal, Man Li, and +Benoit Claise volunteered to work on the Architecture. Juergen Quittek volunteered to work work on Terminology. *) It was mentioned it was hoped that Benoit's & Ganesh's document might serve as a strawman, or starting point for the next WG document. The question was raised as to what would be best to do immediately with that document. It was suggested that it be submitted as an individual draft, and it will be considered as a template or as input to working group documents, like other individual drafts that have been prepared by other IPFIX participants. *) KC Norseth volunteered to work on Error Detection / Handling, and data recovery. Other issues were raised as possibilities for needed to be addressed in some fashion by the Architecture document. These include NAT, firewalls and middleboxes. Reinaldo Penno volunteered to work on the NAT/firewall, and midcom issues/interactions. *) KC Norseth volunteered to serve as document editor. *) The point was raised that the upcoming IPFIX document(s) should address or mention interactions with other IETF-specified topics such as AAA, and Intrusion Detection. Tanja Zseby volunteered some information about AAA and configuration for flow export. *) It was noted that the I-D draft deadline is soon for the documents as listed in our WG milestones. The submission deadlin is Feb. 22 for IETF 53. As such, thee was support for setting a deadling of Friday, Feb. 8, 2002 for those that volunteer info (above) to submit it to the editor, preferably via the IPFIX mailing list . *) It was suggested that, until we collect the information from those volunteers and whether it fits into Architecture and Data Model documents, perhaps we should refer to it as a (single) "design" document for the time being. We signed off from the call at Thu Jan 31 ~1400 CST 2002. The following (at least) joined the conference call: George Carle Tanja Zseby Paul Calato Dave Plonka Kevin Zhang KC Norseth Bill Richards Will Eatherton Benoit Claise Ganesh Sadasivan Vamsi Valluri Ram Gopal Jc Martin Carter Bullard Juergen Quittek Reinaldo Penno Nevil Brownlee Simon Leinen ---------------------------------------------------------------------- Any errors are mine, but please follow-up to the mailing list if you have substantive corrections. Respectfully, Dave -- plonka@doit.wisc.edu http://net.doit.wisc.edu/~plonka ARS:N9HZF Madison, WI -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Thu Jan 31 22:09:31 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id WAA19310 for ; Thu, 31 Jan 2002 22:09:31 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16WTnJ-0007mQ-00 for ipfix-list@mil.doit.wisc.edu; Thu, 31 Jan 2002 20:51:17 -0600 Received: from d2cspimg001.smartpipes.com ([63.89.185.24]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 16WTnG-0007le-00 for ipfix@net.doit.wisc.edu; Thu, 31 Jan 2002 20:51:14 -0600 Received: by D2CSPIMG001.smartpipes.com with Internet Mail Service (5.5.2653.19) id ; Fri, 1 Feb 2002 02:50:44 -0000 Message-ID: <4652644B98DFF34696801F8F3070D3FE0118864A@D2CSPEXM001.smartpipes.com> From: "Wang, Cliff" To: "'plonka@doit.wisc.edu'" Cc: ipfix@net.doit.wisc.edu Subject: RE: conference call minutes (was "Re: [ipfix] arch and data con c all") Date: Fri, 1 Feb 2002 02:50:43 -0000 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain Precedence: bulk Sender: majordomo listserver I (from SmartPipes)participated the call also and volunteered to write the security section and works on the NAT part with Reinaldo Penno. -----Original Message----- From: Dave Plonka [mailto:plonka@doit.wisc.edu] Sent: Thursday, January 31, 2002 4:23 PM To: ipfix@net.doit.wisc.edu Subject: conference call minutes (was "Re: [ipfix] arch and data con call") IPFIX folks, Below, please find the minutes that I prepared following a conference call that was arranged to organize the IPFIX Internet Draft efforts. Thanks to KC Norseth and Enterasys for hosting the conference call. A list of those that participated follows the minutes below. ---------------------------------------------------------------------- The call began Thu Jan 31 ~1300 CST 2002. The conference call roughly followed an agenda prepared by KC Norseth. During the discussion various people volunteered to be responsible for contributing info on particular topics. Their names are noted below in context. These topics were discussed: *) Why are three proposed draft documents laid out this way? * IPFIX Requirements * IPFIX Architecture it was noted that protocol doesn't belong in Architecture * IPFIX Data Model Specifically, the question was asked why not a protocol document. (We discussed that we're not chartered to design a protocol, but to concentrate on standardizing existing practice.) There is some confusion about what items belong in the Architecture rather than Data model and vice versa. *) We discussed points suggested for the Architecture draft... The first was "Summary of Operation": It was noted that some things about the flow definition (such as what causes a flow to be instantiated, and what temporal characteristics they have) are important to specifying the IPFIX architecture. *) It was noted that the components or entities (and possibly interfaces between) are a key portion of the Architecture document. Capabilities negotiation was also mentioned for the Architecture. Kevin Zhang volunteered to work on this. It was mentioned that a work-in-progress Internet Draft document by Benoit and Ganesh does a good job of laying this out. It was also mentioned that the Architecture doc should lay out a number of architecture/component "scenarios". It was discussed as to whether or not the Requirements document should specify the flow definition. There was not support voiced for making it more specific. There were other kudos for Benoit's & Ganesh's document for its description of the flow "concept". *) Message formats It was noted that "messages formats" mostly belongs in a protocol specification so may be out of scope. However, it was noted that the items of information would be necessary for the Architecture specification. Some folks referred to this as the "information model". KC Norseth, Paul Calato, and Simon Leinen volunteered to work on specifying this information. It was noted that the handshaking or interfaces between components does belong in the Architecture. It was considered whether or not the "information model" belongs in architecture or data model. It was suggested that perhaps it would be best to focus on enumerating the important topics, and fit them into the documents afterward. This idea was supported by others as well. Kevin Zhan, Ram Gopal, Man Li, and +Benoit Claise volunteered to work on the Architecture. Juergen Quittek volunteered to work work on Terminology. *) It was mentioned it was hoped that Benoit's & Ganesh's document might serve as a strawman, or starting point for the next WG document. The question was raised as to what would be best to do immediately with that document. It was suggested that it be submitted as an individual draft, and it will be considered as a template or as input to working group documents, like other individual drafts that have been prepared by other IPFIX participants. *) KC Norseth volunteered to work on Error Detection / Handling, and data recovery. Other issues were raised as possibilities for needed to be addressed in some fashion by the Architecture document. These include NAT, firewalls and middleboxes. Reinaldo Penno volunteered to work on the NAT/firewall, and midcom issues/interactions. *) KC Norseth volunteered to serve as document editor. *) The point was raised that the upcoming IPFIX document(s) should address or mention interactions with other IETF-specified topics such as AAA, and Intrusion Detection. Tanja Zseby volunteered some information about AAA and configuration for flow export. *) It was noted that the I-D draft deadline is soon for the documents as listed in our WG milestones. The submission deadlin is Feb. 22 for IETF 53. As such, thee was support for setting a deadling of Friday, Feb. 8, 2002 for those that volunteer info (above) to submit it to the editor, preferably via the IPFIX mailing list . *) It was suggested that, until we collect the information from those volunteers and whether it fits into Architecture and Data Model documents, perhaps we should refer to it as a (single) "design" document for the time being. We signed off from the call at Thu Jan 31 ~1400 CST 2002. The following (at least) joined the conference call: George Carle Tanja Zseby Paul Calato Dave Plonka Kevin Zhang KC Norseth Bill Richards Will Eatherton Benoit Claise Ganesh Sadasivan Vamsi Valluri Ram Gopal Jc Martin Carter Bullard Juergen Quittek Reinaldo Penno Nevil Brownlee Simon Leinen ---------------------------------------------------------------------- Any errors are mine, but please follow-up to the mailing list if you have substantive corrections. Respectfully, Dave -- plonka@doit.wisc.edu http://net.doit.wisc.edu/~plonka ARS:N9HZF Madison, WI -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/