Subject: [IPFIX] Flow-selection-tech: review

From: Benoit Claise <[email-masked]>
Date: Wed, 16 Feb 2011 08:01:23 +0100

Dear all,

I've been reviewing the draft, mainly in the context of the IPFIX 
Mediation Protocol draft, to make that all the mediation related drafts 
are in line:
requirements (RFC 5982), framework (RFC-editor), anonymization, flow 
selection techniques, aggregation, and finally the protocol.
Sorry if I'm late, this flow selection technique is the last one I reviewed.

The important issues start with "important"

- Reference. Sometimes referred as the RFC number [RFC5475 
<http://tools.ietf.org/html/rfc5475>], and sometimes as [IPFIX-REQ 
<http://tools.ietf.org/html/draft-ietf-ipfix-flow-selection-tech-04#ref-IPFIX-REQ>]

- "Flow selection reduces the resource demands for capturing, storing, 
exporting and post-processing flow-based measurement results."
Actually, reduces the resource demands for post-processing flow-based 
measurement results, that's for sure.
While, reduces  the resource demands for capturing, storing -> it 
depends: if you have to look at the all packets, hence all flows, in 
order to do the flow selection (like the elephant), you don't gain much.
You might want to clarify. Somehow, you have to go in the conclusions of 
[EsVa01] to discover this.

- Terminology

   This document is consistent with the terminology introduced in
    [RFC5470  <http://tools.ietf.org/html/rfc5470>], [RFC5475  <http://tools.ietf.org/html/rfc5475>] and [IPFIX-REQ  <http://tools.ietf.org/html/draft-ietf-ipfix-flow-selection-tech-04#ref-IPFIX-REQ>].  Further some additional terms
    are presented which extend the terminology.

    * Flow

       A Flow is defined as a set of packets with common properties
       passing an Observation Point in the network during a certain time
       interval.

Actually "Flow" is not an additional term, as it's defined already in RFC5101.
However, the definitions are not quite the same. Isn't it confusing?

The "Flow record" definition is exactly the same as RFC5101. Why redefined?

Important point: this is the first draft in the IPFIX series that doesn't use capitalized definitions.
All IPFIX draft uses a sentence such as:
    At the beginning of the terminology section, you might add a sentence such as:
    In this document, as in [RFC5101  <http://tools.ietf.org/html/rfc5101>] and [RFC5476  <http://tools.ietf.org/html/rfc5476>], the first letter of
    each IPFIX-specific and PSAMP-specific term is capitalized along with
    the IPFIX Mediation-specific terms defined here.
However, some sentences use the capitalization.
    "As an example, if an IPFIX Mediator interacts with a set of IPFIX
    Collectors, flow records arriving at ..."

the "Intermediate Process" and "IPFIX Mediator" definitions should be referred fromhttp://tools.ietf.org/html/draft-ietf-ipfix-mediators-framework-09

Important point: I believe that your definition of "Flow Selection Process" should be a special case of the Intermediate Selection Process, which is defined as, in bothhttp://tools.ietf.org/html/draft-ietf-ipfix-mediators-framework-09  andhttp://tools.ietf.org/html/draft-claise-ipfix-mediation-protocol-02:

    Intermediate Selection Process

    An Intermediate Selection Process is an Intermediate Process that
    selects records from a sequence based upon criteria-evaluated record
    values and passes only those records that match the criteria (e.g.,
    Filtering only records from a given network to a given Collector).

How is the IPFIX Exporter different from the Exporter specified in RFC5101?
RFC5101:
    Exporter

       A device that hosts one or more Exporting Processes is termed an
       Exporter.
Your draft:
    * IPFIX Exporter

       The IPFIX exporter is a software that receives measurement data
       from e.g. the Metering Process and exports this data to a
       collector.

Somehow, based on the terminology section, I'm missing the link with the other drafts, such as the mediation framework.

- Important: RFC 2119 language
    "Ideally, we would like to keep also a timestamp for the first (T_fd)
    and last (T_ld) not yet exported packets belonging to every discarded
    flow record. "
    use MAY?

    "Another information that can be easily maintained is the number of
    discarding actions, along with the timestamps of the first and last
    action.  This information SHOULD not be used by applications to re-
    normalize the received per flow statistics (because a flow may be
    discarded and created multiple times), but rather to monitor and
    control the performance of the implemented policy."
    Second sentence use SHOULD (->  btw, SHOULD NOT)
    So the first sentence should use MAY?

    "For this reason, flow exporting protocol specification does
    not include flow selection during the recording process as a
    mandatory function even if the information model has been designed to
    enable such function."

    I could give other examples, but my point is that I don't know how to be compliant to this standard track future RFC, from an implementation point of view.

Along the same lines, which flow selection techniques in section 6 must a compliant implementation support?

- Important: IPR
Along the same line as the previous point, you refer tohttp://conferences.sigcomm.org/imc/2001/imw2001-papers/03.pdf  in section 4.3.  Flow selection during the exporting process

"  An example of such a policy could
    be to export only the flow records associated to flows whose
    accounted traffic is below a certain threshold, or to implement a
    more complex mechanism such as the one described in [DuLT01a  <http://tools.ietf.org/html/draft-ietf-ipfix-flow-selection-tech-04#ref-DuLT01a>] or
    [DuLT01b  <http://tools.ietf.org/html/draft-ietf-ipfix-flow-selection-tech-04#ref-DuLT01b>]."

Should it be implemented or not to be compliant with that future RFC?
When I spoke with Nick Duffield some years ago, there was a patent on this.
And I don't see any IPR on this draft.

- Important: Section 5 doesn't refer at all to the Intermediate Process specifies in the mediation framework

- Important: I have now finished the section 5, and still not sure how the flow recording process is different from the metering process.
I can't find this term in the IPFIX architecture.
However, we have some IEs that are defined specifically for the Flow Recording...

- Important: why do we have "Flow recording and selection" on the Original Exporter while we only have "Flow Selection" on the Mediator?
If I look at with IE I should implement, I see:
If there
    7  <http://tools.ietf.org/html/draft-ietf-ipfix-flow-selection-tech-04#section-7>.  Information model for flow selection information exporting . .14  <http://tools.ietf.org/html/draft-ietf-ipfix-flow-selection-tech-04#page-14>
      7.1  <http://tools.ietf.org/html/draft-ietf-ipfix-flow-selection-tech-04#section-7.1>.  Meter process related (TBD1-TBD3)  . . . . . . . . . . . .16  <http://tools.ietf.org/html/draft-ietf-ipfix-flow-selection-tech-04#page-16>
      7.2  <http://tools.ietf.org/html/draft-ietf-ipfix-flow-selection-tech-04#section-7.2>.  Flow recording process related (TBD4-TBD11)  . . . . . . .18  <http://tools.ietf.org/html/draft-ietf-ipfix-flow-selection-tech-04#page-18>
      7.3  <http://tools.ietf.org/html/draft-ietf-ipfix-flow-selection-tech-04#section-7.3>.  Flow exporting process related (TBD12-TBD19) . . . . . . .21  <http://tools.ietf.org/html/draft-ietf-ipfix-flow-selection-tech-04#page-21>
How does it translate for Mediator, which only contains "Flow Selection" according to this figure?

- IE naming convention. In the description, can you please have the meaning.
For example: fsMeterUnmeasPacketCountTsFirst  is the concatenation of FlowSelectionMeasuredUnmeteredPacketCounterTimeStampFirst.
Even after the reading, I'm not 100%.
So if someone only looks at the IANA registry...
For example, what is fsFrecPacketInDroppedRecsCountTsFirst, FlowSelectionFrec... unless it's FlowSelectionFlowRec... with a capitalized R
fsFrecFrecDroppedCountTsFirst, fsFrecFrecDroppedCountTsLast: why twice Frec in the names
Btw, what's the size limit for a IE name?

- fsMeterUnmeasPacketCountTsFirst

    Description:

       Specifies the timestamp of the first packet not measured because
       of the use of the flow state dependent sampling.  Together with
       the IE fsMeterUnmeasPacketCountTsLast it allows to evaluate the
       count of packets that were not measured because of the use of the
       flow sampling.

"measured" by who? by the Metering Process, so it should be "metered", right?
Also, "by the Metering Process" should be added in the description, right?
Same remark for 7.1.*

- Important

        7.2.6. fsFrecUnexportedFrecCount

    Description:

       This Information Element specifies the count of the flow records
       currently existing in the flow recording process containing at
       least one non-exported packet.

        7.2.7. fsFrecUnexportedPacketInFrecCount

    Description:

       This Information Element specifies the count of non-exported
       packets contained in flow records of the flow recording process.

        7.2.8. fsFrecUnexportedBytesInFrecCount

    Description:

       This Information Element specifies the count of non-exported bytes
       contained in flow records of the flow recording process.

Looking at figure 1, the Flow Recording is before the Exporting Process.
So all flow records in the Flow Recording are not exported, right? What do I miss?

- fsExpUnexportedCount  should at least contain FlowRecord (or FRec or whatever) in the name

- Important: you have defined a new IE selectorMethod
Why not reuse selectorAlgorithm from RFC 5476?

- Important: IPFIX IANA specifies the IE 60, ipVersion
You redefined this.

      9.6. flowIPAddressVersion

    Description:

       This Information Element specifies the IP version field of the
       packets belonging to flow records which SHOULD be considered as
       not eligible for removal.

    Abstract Data Type: unsigned8

    Data Type Semantics: identifier

    ElementId: 60

    Status: current

    Reference: see [RFC5102  <http://tools.ietf.org/html/rfc5102>] for the definition of the ipVersion
    information element and its Id.

You can not do this.
Same remark for flowIPv4SourceAddress, flowIPv6SourceAddress, flowIPv6DestinationAddress

Regards, Benoit.