Subject: RE: [ipfix] Changes to architecture doc.

From: "Carter Bullard" <[email-masked]>
Date: Wed, 21 May 2003 11:57:43 -0400

Hey Alex,
   What is the relationship between transport of
flow records and the generation of flow records?
IPFIX does not define a flow generation process,
it defines a flow record transport protocol.

Carter

> -----Original Message-----
> From: Alex Audu [mailto:[email-masked]] 
> Sent: Wednesday, May 21, 2003 11:52 AM
> To: Carter Bullard
> Cc: 'Maurizio Molina'; 'Ganesh Sadasivan'; [email-masked]
> Subject: Re: [ipfix] Changes to architecture doc.
> 
> 
> This, in my view, is a huge limitiation of IPFIX. PSAMP does 
> a better job of
> specifying
> the packet selection process as well as the other aspects 
> involved in getting the
> data to
> the collector.  IPFIX is being suggested as a viable export 
> for PSAMP. If so, it
> needs to
> address the issue of data selection.
> 
> Regards,
> Alex.
> 
> Carter Bullard wrote:
> 
> > Maurizio,
> >
> > I believe that this is out of scope with IPFIX.
> > We're suppose to be defining transport of flow
> > records, not defining flow monitors.
> >
> > Carter
> >
> > > -----Original Message-----
> > > From: majordomo listserver
> > > [mailto:[email-masked]] On Behalf Of Maurizio Molina
> > > Sent: Wednesday, May 21, 2003 6:01 AM
> > > To: Ganesh Sadasivan
> > > Cc: [email-masked]
> > > Subject: Re: [ipfix] Changes to architecture doc.
> > >
> > >
> > >
> > >
> > > Ganesh Sadasivan wrote:
> > >
> > > > Hi,
> > > >
> > > > In going through the architecture spec. I find  that 
> the following
> > > > topics
> > > > needs to be modified. I'll list all of them in one e-mail
> > > and then on a
> > > > topic-by-topic we can have separate e-mail threads.
> > > >
> > > > BTW I could not get a copy of the arch. spec from the link
> > > >
> > > http://www.ietf.org/internet-drafts/draft-ietf-ipfix-architect
> > > ure-02.txt
> > > >
> > > > This has been deleted from the internet drafts database
> > > because it was
> > > > not updated for more than 6 months. I guess the same is
> > > true with data
> > > > model
> > > > document.
> > > >
> > > > The alternate link that I got it from is :
> > > >
> > > http://www.ietf.org/proceedings/02nov/I-D/draft-ietf-ipfix-arc
> > > hitecture-02.txt
> > > >
> > >
> > > ...........
> > >
> > > >
> > > > * Section 4.5 :
> > > > "  There MAY be additional rules defined within the
> > > observation domain
> > > >    so that only certain flows records are picked up for 
> export. "
> > > > Though rules are defined w.r.t observation domain,
> > > > the entity that applies these rules is the IPFIX protocol.
> > >
> > > Hi,
> > > I'd like to discuss the type of packet sampling/flow
> > > selection that seems to be
> > > considered by IPFIX so far.
> > > Re-reading section 4.5 of the arch. spec., and the previous
> > > ones (4.4.2, 4.4.3,
> > > 4.4.4), I got the impression that only the following types of
> > > sampling/slection at
> > > the packet or flow level are considered:
> > > 1) Sampling of packets within a flow (sec. 4.4.4): Packets
> > > undergo a filter to see
> > > if they match a flow to be monitored; if they do, they are
> > > (possibly) sampled (to
> > > alleviate METER processing load).
> > > 2) Selection of flows to export (sec. 4.5): all defined flows
> > > are monitored
> > > (possibly packet sampling within flows as defined above is
> > > applied), then a
> > > decisions is taken about whether to export the entire pool of
> > > monitored flows or
> > > just a subset of it. This, with the goal to alleviate
> > > EXPORTER/COLLECTOR load.
> > >
> > > Imo, what's not explicitly considered is a third case, where
> > > there is a packet
> > > sampling but "oriented" to flow selection, i.e. packets
> > > undergo a filter match but
> > > then they're not "blindly" (i.e. n out of N or similar)
> > > sampled, but rather
> > > sampled or discarded on the basis of some property of the
> > > flow they matched. The
> > > property may be static (e.g. "flows with high monitoring
> > > priority") or dynamically
> > > built  (e.g. flows carrying more traffic than
> > > others->sometime called "elephant"
> > > flows). This type of sampling can alleviate both METER and
> > > EXPORTER/COLLECTOR
> > > processing load.
> > > The interest for this type of "sampling/selection" (actually,
> > > it's both packet
> > > sampling and flow selection) is that for some applications it
> > > may be preferred to
> > > receive reliable figures of  the volume of a subset of flows
> > > rather that less
> > > reliable estimates of the volume of ALL the flows (as
> > > sampling of type 1 would
> > > provide). Furthermore, differently from flow selection of
> > > type 2), it's not
> > > required to keep a flow record for ALL the flows before
> > > deciding whether to export
> > > them or not, as the decision  whether to open a new flow
> > > record is taken "on the
> > > fly" when packets arrive. Then it's potentially more scalable.
> > >
> > > That said, what are the issues that a standard body should
> > > consider? Imo, they are
> > > mainly two:
> > > - the definition of policies for seclecting packets on the
> > > basis the property of
> > > flow they match (i.e. high priority monitoring flow
> > > definition, or flows carrying
> > > high traffic volumes, or other....). This, without dwelling
> > > into the details of
> > > algorithms may implement this "on line" detection of  properties.
> > > - the type of information that has to be exported: e.g. which
> > > percentage of
> > > traffic is being discarded? Otherwise a collector would only
> > > get the report about
> > > the monitored flows (high priority or high volume, or
> > > other...), but it's
> > > different to know that they compose the 99% oe the 10% of the
> > > traffic....
> > >
> > > In the requirement draft,
> > > http://www.ietf.org/internet-drafts/draft-ietf-ipfix-reqs-09.t
> > xt, in sec. 5.2
> > (sampling) and 5.3 (overload behaviour) there's already 
> some text that
> > partially,
> > but not completely and not exactly, supports the type of 
> sampling/selection
> > I've
> > described  above.
> > I'd be glad to know whether people in the group share this 
> view that this
> > type of
> > sampling/selection has to be more explicitly considered. 
> The next step would
> > then
> > be to discuss if/what needs to be standardized to support it.
> >
> > Regards,
> > Maurizio
> >
> > --
> > Help        mailto:[email-masked] and say 
> "help" in message
> > body
> > Unsubscribe mailto:[email-masked] and say
> > "unsubscribe ipfix" in message body
> > Archive     http://ipfix.doit.wisc.edu/archive/
> >
> > --
> > Help        mailto:[email-masked] and say 
> "help" in message body
> > Unsubscribe mailto:[email-masked] and say
> > "unsubscribe ipfix" in message body
> > Archive     http://ipfix.doit.wisc.edu/archive/
> 
> 

--
Help        mailto:[email-masked] and say "help" in message body
Unsubscribe mailto:[email-masked] and say
"unsubscribe ipfix" in message body
Archive     http://ipfix.doit.wisc.edu/archive/

---

From: Maurizio Molina <[email-masked]>
Date: Wed, 21 May 2003 18:40:09 +0200

> it defines a flow record transport protocol.

Hi Carter,
then what is your interpretation of the following part of the requirement
draft (5.2), that basically says that information about (packet) sampling
MUST be well defined and reported to the collector if the sampling changes?
This information is about the generation of flow records, isn't it? Then if
it's out of scope, we should drop this text from the requirements as
well...
Maurizio

**************** From req. draft **********
    The metering process MAY support packet sampling. If sampling is
   supported the sampling configuration MUST be well defined. The
   sampling configuration includes the sampling method and all its
   parameters.

   If the sampling configuration is changed during operation, the new
   sampling configuration with its parameters MUST be indicated to all
   collecting processes receiving the affected flow records. Changing
   the sampling configuration includes: adding a sampling function to
   the metering process, removing a sampling function from the metering
   process, change sampling method, and change sampling parameter(s).

   In case of any change in the sampling configuration, all flow records
   metered by the previous sampling configuration MUST be terminated and
   exported according to the export configuration. The metering process
   MUST NOT merge the flow records generated with the new sampling
   configuration with the flow records generated with the previous
   sampling configuration.
******** End From Req. draft *****************

---

From: Alex Audu <[email-masked]>
Date: Wed, 21 May 2003 13:38:49 -0500

Carter,

Transport and generation of flow records seem to be orthogonal processes,
until you start
asking for ways to control or modify the generation process itself, or to
communicate the
character of the generated flows to the remote collector. Some of the
issues currently
being discussed (what to do about lost flow records) could be linked to the
lack of
specification of the data model used in generating the flows.

Alex..