Subject: RE: [ipfix-arch] configuration information.

From: "Reinaldo Penno"<[email-masked]>
Date: Fri, 11 Jan 2002 13:03:59 -0800

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]]
> Sent: Friday, January 11, 2002 10:37 AM
> To: Ganesh Sadasivan
> Cc: arch
> Subject: Re: [ipfix-arch] configuration information.
> 
> 
> Ganesh Sadasivan wrote:
> > 
> > Caloto,
> > 
> > [email-masked] wrote:
> > 
> > > Ganesh Sadasivan wrote:
> > > >
> > > > <snip>
> > > > [email-masked] wrote:
> > > >
> > > > >
> > > > > > > > > If you have very specific templates such as srcIP,
> > > > > > > destIP, etc, it's
> > > > > > > > > not to hard find records in common. But you 
> can have things
> > > > > > like:
> > > > > > > > >
> > > > > > > > > src_AnyIP   Dest_Yahoo.com    Proto_Any   and
> > > > > > > > > src_AnyIP   Dest_AS3462       Proto_HTTP.
> > > > > > >
> > > > > > >       I assume here you mean that there are 2 counters and
> > > > > > >       2 "flows" reported. One has (to use cisco 
> terms) a template
> > > > > > >       of "dest-ip, byes, packets" and the other 
> "dest-AS, proto,
> > > > > > >       byes, packets".
> > > > > > >
> > > > > > >       If the device uses a multi-match method 
> then the device
> > > > > > >       "knows" that a packet may be added to both 
> buckets. It
> > > > > > >       should tell the collector that these 
> templates overlap.
> > > > > >
> > > > > > I agree. But it should say that not only these 
> templates overlap but
> > > > > > what 5-tuple? is overlapping. (also see below)
> > > > > >
> > > > > > >       If on the other hand the device uses a 
> first match method
> > > > > > >       then the counts do not overlap.
> > > > > >
> > > > > > Should we assume that you can have different templates per
> > > > > > Card/Interface if the cards/interfaces have 
> intelligence? In this case
> > > > > > they could overlap.
> > > > >
> > > > >         Again, the device "knows" how it is counting. If the
> > > > >         devices is able to track packets based on multiple
> > > > >         keys then some thought needs to be given to whether
> > > > >         or not those key sets overlap. For example, if one key
> > > > >         set is "in-interface, src-ip, dst-ip" and the other is
> > > > >         "in-interface, src-AS, dst-AS" then the associated
> > > > >         counters do not overlap. If the application wants
> > > > >         a total for src-AS1 then it can add the bytes from
> > > > >         first ket set to the second.
> > > > >
> > > > >         Overlaping counters are not something determined
> > > > >         after the fact based on data seen, it is determined
> > > > >         ahead of time for an entire class (or key set) of
> > > > >         records.
> > > > >
> > > >
> > > > This is really not true. In the above example there are multiple
> > > > ways of measuring:
> > > > 1. All packets are tracked based on both the key sets.
> > > >     There is 100% duplication.
> > > > 2. Packets are tracked based on the first matching key set.
> > > >     A single src-ip can have multiple src-ASs. Assuming the ASs
> > > >     for src-IP1 are src-AS1 and src-AS2, and say only
> > > >     src-AS1 is setup to match and the keyset 
> {in-interface, src-AS, dst-AS}
> > > >     gets matched first, part of the packets that come on src-IP1
> > > >     gets matched to the first key set and the other 
> part gets matched
> > > >     to the {in-interface, src-ip, dst-ip} key set. The 
> collector can
> > > >     get the  src-ASs per src-IP information through an 
> external BGP
> > > >     daemon or some other means.
> > > >     Even if the collector has knowledge of how many 
> origin ASs are there
> > > >     for src-IP1, it still cannot find out how many 
> packets passed through
> > > >     src-IP1.
> > >
> > >         I don't follow you here. If the src IP matches the
> > >         first rule then I stop. I don't care what the AS is
> > >         because I don't look further.  The point is that
> > >         only one counter bucket is ever updated. Thus I know
> > >         I have no duplication of counts.
> > 
> > You are correct if the src IP is matched first. If the src 
> AS is matched
> > first ,and only one of the src AS corresponding to src IP 
> is setup to
> > match (not because of mis configuration, but because of the dynamic
> > nature of AS assignment to the IP), then the
> > counts for the src IP gets split across 2 different key 
> sets. There will be
> > no easy way to determine the actual number of packets that came on
> > this src IP.
> 
> 	If the goal was to know how much data src-IP1 generated
> 	then I would suggest that this is a miss configuration.
> 	The src-IP1 rule should have been first. You can't
> 	go from a higher level aggregation like AS to a lower
> 	level one like IP. 

humm..What's more aggregate? AS3461 or 200.244/16? Or even 9/8?

> 
> 	But there is still no question on overlap. There is no 
> 	overlap because only one bucket is updated for a packet.

you are right, but you are not sure if there was (or would be) overlap or
not in a first match approach. Let's suppose you have two rules, one for SIP
and another one for UDP. No all UDP packets are SIP packets, and not all SIP
packets are UDP.

You then receive SIP-TCP, TCP, UDP and SIP-UDP packets. The first three
packets are okay, but the fourth packets would match both rules, but these
rules measure different things. Which rule would you put first? Rule is not
a superset of Rule 2 and vice-versa.

I imagine then that the argument is to configure more and more granular
rules...

> 
> 	My original point was the collector will need
> 	some overlap information in order to make sense of 
> 	the data.
> 
> 	And I think your point was that it is too hard to
> 	know what buckets overlap. I still don't quite follow 
> 	why that is. 

well, you can always argue that the "device knows" and end the discussion,
but the fact is that a packet can be counted twice in several situations if
the "device does not know". 

In a first match scenario, there is no
> 	overlap. I'm free to add the counts together if
> 	my collector analysis shows them to be related.
> 	Without overlap info, I cannot do that.
> 
> 	We might need a phone call to get to the bottom.
> 	Feel free to call me at 603-738-4206. I'm on
> 	the east coast in the US.
> > 
> > Ganesh
> > 
> > >
> > >         What am I missing?
> > >
> > > >
> > > > This is a fairly simple example. Looks to me that the 
> complexity on the
> > > > collector side so that it can automatically infer the 
> relationships between
> > > > key sets is quite high.
> > > >
> > > > Thanks
> > > > Ganesh
> > >
> > > --
> > > Help        mailto:[email-masked] and say 
> "help" in message body
> > > Unsubscribe mailto:[email-masked] and say
> > > "unsubscribe ipfix" in message body
> > > Archive     http://ipfix.doit.wisc.edu/archive/
> 
> --
> Help        mailto:[email-masked] and say "help" 
> in message body
> Unsubscribe mailto:[email-masked] and say
> "unsubscribe ipfix" in message body
> Archive     http://ipfix.doit.wisc.edu/archive/
>

---

From: [email-masked]
Date: Fri, 11 Jan 2002 16:38:03 -0500

> humm..What's more aggregate? AS3461 or 200.244/16? Or even 9/8?

	I meant IP/32 not prefixes. Sorry if that was not clear.

> you are right, but you are not sure if there was (or would be) overlap
> or not in a first match approach. Let's suppose you have two rules,
> one for SIP and another one for UDP. No all UDP packets are SIP
> packets, and not all SIP packets are UDP.
> 
> You then receive SIP-TCP, TCP, UDP and SIP-UDP packets. The first
> three packets are okay, but the fourth packets would match both rules,
> but these rules measure different things. Which rule would you put
> first? Rule is not a superset of Rule 2 and vice-versa.

	Just like ACL's, you can have overlaping ACL's but
	the first one that is a hit wins.

> 
> I imagine then that the argument is to configure more and more
> granular rules...
> 
> 
> >
> well, you can always argue that the "device knows" and end the
> discussion, but the fact is that a packet can be counted twice in
> several situations if the "device does not know".

	Great, then the device reports that all its flows
	may overlap. I'm no worse off than not saying anything.
	But there are many devices that have relatively simple
	counting mechanisms and the extra information will
	help down stream applications do some analysis that may 
	otherwise not be doable.

--
Help        mailto:[email-masked] and say "help" in message body
Unsubscribe mailto:[email-masked] and say
"unsubscribe ipfix" in message body
Archive     http://ipfix.doit.wisc.edu/archive/