Subject: [ipfix-arch] Re:

From: [email-masked]
Date: Tue, 18 Jun 2002 10:14:06 -0400

Ganesh Sadasivan wrote:
> 
> Paul,
> 
> > 5.1. Flow Classification
> >
> >    The collector MUST be able to map the flow to the
> corresponding
> >    property types defined by the flow type. This can
> be done only if
> >    the collector has a mapping from flow type
> identifier (carried in
> >    each flow record) to its actual structure. More
> details of how
> >    this can be achieved are described in section
> 5.???
> >
> >
> >    In addition the collector, when it receives the
> flow records, MAY
> >    need the following to interpret the flow records
> further:
> >
> >       A. Observation Point.
> >       B. Selection Criteria of Packets
> >
> >    As mentioned in section 2.1, a flow record can be
> better analyzed
> 
> [G] Notice that it mentioned that the flow
> can be "better analysed" and it is not a
> MUST. Eg. If 2 observation
> point A,B are measuring in flows in 2 different
> ways say A is sampling 1 in 5 and B is sampling
> 1 in 25. The collector can better analyse the
> information, if along with the flow information,
> the observation point is also specified.This is
> not something mandatory but is useful for a
> detailed analysis.

	Since the sampling rate needs to be reported with
	(or can be correlated to) the flow, why do I need
	the observation point. All the information needed
	is in front of me. 

	If you are saying the sampling rate is not correlated
	to the flow then how would an application interpret
	the flow information to begin with?

> 
> >    if the Observation Point from which it is
> measured is known. As
> >    such it is recommended that the flow record carry
> the Observation
> >    Point information along with the flow records
> when exported. In
> >    cases where there is a single observation point
> or where the
> >    observation point information is not relevant,
> the exporter MAY
> >    choose not to add this to the flow records.
> >
> 
>         I don't understand this section. What is flow
> classification?
> 
>         Also, A flow should be able to be interpreted
> regardless of
>         knowing the observation point. A single observation
> point may
>         map flows in 2 or more different ways. The fact that
> different
>         observation points map flows based on different
> criteria
>         is irrelevant. All the information needed to
> interpret the flow
>         must be passed up.
> 
> > 5.2.1. Function on properties that determines a flow
> type (Fi)
> >
> >    Packets that satisfy a function on the fields
> defined by the
> >    packet header fields or fields obtained while
> doing the packet
> >    processing or the properties of the packet
> itself.
> >
> >    Example:
> >
> >    Mask/Match of the fields that define a filter.
> The filter may be
> >    defined as {Protocol == TCP, Destination Port
> between 80 and 120}.
> >
> >
> >    could be used in any sequence to select
> >    packets.
> >
> 
>         I think it would it be better to say "Multiple such
> functions"
>         instead of filters?

	Agreed.

> 
> > 5.3. Selection Criteria for flows for export
> >
> >    The measurement device MAY define additional
> rules so that only
> >    certain flows records are picked up for export.
> This MAY be done
> >    by either of the two types of methods defined in
> 5.2.1 and
> >    5.2.2 or a combination of them.
> >
> >    Example:
> >
> >    Only the flow records which meet the following
> selection criteria
> >    are exported.
> >
> >       1 All flow records whose destination IP
> address matches
> >         {20.3.1.5}.
> >       2 Every other (.i.e. sampling rate 1 in 2)
> flow record whose
> >         destination IP address matches {160.0.1.30}.
> >
> 
>         I'm not sure I agree with this. I think you have
> defined
>         2 flows in #1 and #2.
> 
> [G] That is true.
> 
> Given that, both flows are exported.
>         I think maybe the "Selection Criteria" is really part
> of
>         the Flow Type.
> 
> [G] It could be a post-flow-measurement criteria
> also. So need not be fixed to flow-type.

	At a conceptual level I don't like having to different
	way of defining the same thing. At an implementation
	level, I can see the need to do it at different spots.
	
	In order to combine the 2, I think I would allow for 
	Flow-type to be applied in more than one place on the
	device. So the measuring device could apply flow-type 1 
	and then the post-flow-measurement to apply another 
	flow-type definition.
	
> 
> >       * Packet Counter
> >       * Dropped Packet Counter
> >       * Byte Counter
> >       * Dropped Byte Counter
> >       * Timestamp of the First Packet Observed
> >       * Timestamp of the Last Packet Observed
> 
>         Generally the last time stamp is the time at which
> he flow has expired (as described earlier).
> 
> The last
>         packet observed is not available unless each packet
>         is processed in software. So either we need another
>         field or a different definition.
> 
> [G] Just curious. Why do we need a last-packet
>  observed field?

	You need something that denotes the end of the
	flow. There are several ways to do it, start-time
	and duration, end-time and duration, start-time
	and end-time. My only point was the if end-time
	is defined as "last packet observed" the info
	will not be available unless the packets are
	processed in software.

> 
> >
> >       * Unique ID of the Observation Point
> >       * Unique ID of the Measuring Device
> 
>         As stated above, I disagree that these are part of
> flow definition.
> 
> __________________________________________________
> Do You Yahoo!?
> Yahoo! - Official partner of 2002 FIFA World Cup
> http://fifaworldcup.yahoo.com

--
Help        mailto:[email-masked] and say "help" in message body
Unsubscribe mailto:[email-masked] and say
"unsubscribe ipfix" in message body
Archive     http://ipfix.doit.wisc.edu/archive/

---

From: David Moore <[email-masked]>
Date: Tue, 18 Jun 2002 09:20:11 -0700

> 	will not be available unless the packets are

Why will the info not be available unless the packets are processed in
software?

The architecture document
(http://norseth.org/ietf/ipfix/draft-ietf-ipfix-architecture-02.txt)
and ipfix requirements document
(http://www.ccrle.nec.de/draft-ietf-ipfix-reqs-03.txt) both have
this picture.  Well the reqs document doesn't have sampling and
and says that can go other places, but otherwise the same.

                       packet header capturing
                                 |
                            timestamping
                                 |
                                 v
                          +----->+
                          |      |
                          |   sampling Si (1:1 in case of no sampling)
                          |      |
                          | classifying Fi (NULL when No criteria)
                          |      |
                          +------+
                                 |
                                 |
                                 v
                               Flows

Which suggest that one would be getting the timestamps in hardware, since it
places timestamping ahead of classifying, presumably to minimize the overhead
of variable length computation in the jittering the timestamps.

The ipfix reqs document also says in 5.4:
5.4.  Timestamps

   The metering process MUST be able to generate timestamps for the
   first and the last observed packet of a flow. The timestamp
   resolution MUST be at least the one of the sysUpTime [RFC1213], which
   is one centisecond.

So if you really think last observed packet timestamps are difficult
somehow, you should probably explain why, since all of the docs
are currently suggesting they be there.  And assuming that you are
getting timestamps as close to the packet reception as possible
(ie, before classifying), I don't see why it's difficult to have
this information.  But I'm guessing that is the problem, namely you
don't want to do timestamps on the card, just do it on the first
packet via a special case?

-- david

---

From: [email-masked]
Date: Tue, 18 Jun 2002 12:51:18 -0400

> packet via a special case?
	Right. I don't think most hardware timestamps every packet.
	Which is why I said we either need another field or
	another definition. But as you pointed out, there are
	other areas of the doc that will need to change too.

---

From: Ganesh Sadasivan <[email-masked]>
Date: Wed, 19 Jun 2002 10:28:14 -0700 (PDT)

> 	the flow information to begin with?

What I meant is that the observation point is the
correlation point between the configuration and the
flow records. Is it not better than sending the
config with the flow records?

> > >         destination IP address matches

I don't disagree here. The main reason for this
separation is because of the metering process
definition in the requirement spec. The input to
metering porcess is always packets. So to provision
a room for post-flow-measurement, a differentiating
block is introduced.

> > he flow has expired (as described earlier).
=== message truncated ===

---

From: [email-masked]
Date: Wed, 19 Jun 2002 14:28:22 -0400

Ganesh Sadasivan wrote:
> flow records. 

	You could have an ID that maps to a flow definition and 
	then the ID is reported with the flow. The observation 
	point ID is not needed. So the definition gets reported 
	once and the ID is reported with each flow. I'm proposing 
	this kind of separation. Down stream I want the data to stand
	on its own without any required knowldge of devices.

> Is it not better than sending the

	Sounds like we're on the same page. 

	But I've raised this issue before and will again
	when I give comments on the latest req and arch docs.

	I see an iterative process here. Requiring packets as
	an input I think is a mistake. I should be able to input
	"flow data" reguardless of where it comes from.
	Then I can apply these functions in a successive fasion
	at many points in the system.

---

From: Ganesh Sadasivan <[email-masked]>
Date: Thu, 20 Jun 2002 18:51:58 -0700 (PDT)

--- [email-masked] wrote:
> > > > >    The collector MUST be able to map the
> flow to
> > > the
> > > > corresponding
> > > > >    property types defined by the flow type.
> This
> > > can
> > > > be done only if
> > > > >    each flow record) to its actual
> structure.
> > > More
> > > > details of how
> > > > >    In addition the collector, when it
> receives
> > > the
> > > > flow records, MAY
> > > > >    As mentioned in section 2.1, a flow
> record
> > > can be
> > > > better analyzed
> > > >

Ok. The degenerate case is that one could have 
config per observation ID in which case reusing
the observation ID is useful. Otherwise you
are correct, it is better t have a separate ID.

I did not understand your last sentence.
"Down stream I want the data to stand
on its own without any required knowldge of
devices."

-Ganesh