Subject: Re: [IPFIX] draft-ietf-ipfix-a9n

From: Benoit Claise <[email-masked]>
Date: Mon, 20 Feb 2012 21:44:51 +0100

Forwarding to the IPFIX mailing list for visibility.
Thanks Rahul

Regards, Benoit.
> Hi,
>
> Here are my comments.
>
> I think correlation is important part of aggregation and this draft should
> include a section on it.
> There are different cases of correlation which the draft should touch.
>
> 1. Key fields of Aggregated Flow is same or the subset of all original
> Flows
>       * In this case the correlation is straight forward. The values from
> the original flows are simply merged (added/replaced/etc) into matching
> aggregated flow.
> 2. Key fields of Aggregated Flow is larger than at least one original flow.
>       * An example is best to describe this case. One original flow contain
> (5-tuple, src-interface, byte) and another original flow contains
> (5-tuple, transaction-delay). Now the aggregated flow has the key fields
> of srcip, dstip and src-interface). One of the flow doesn't have
> information on per source-interface basis. So the aggregator will perform
> aggregation at two level and represent that data in ipfix-structured
> format. The above data would look like
>
> Srcip, dstip, transaction_delay, { List of {src-interface, byte } }
> =====  =====  =================  ==================================
> A,     B,     200,               { (Ethernet0/0, 1500),
>                                     (Ethernet1/0, 1600) .. }
> A,     C,     300,               { (Ethernet0/0, 1500) }
>
>
>
> Section 5.1.1 Distributing values across Interval
> -------------------------------------------------
> * should add 'synchronized expiry of Original Flows' - Under this method
> Flow C would be exported three times with start-time/end-time matching the
> aggregation interval start-time/end-time.
>
> * There will always going to some original flows which will arrive late
> and will not be accounted. It will be good have a cumulative counter of
> such late arrivals on per template id (aggregated flow) basis. This
> cumulative count should be periodically exported along with end-time
> (time-of-snapshot). The counter will provide a degree of confidence in the
> data exported by the aggregator for each template.
>
> Section 5.2 Spatial aggregation
> -------------------------------
> * Is it a good idea to add an example of deriving a key in aggregate flow
> from multiple keys or non-keys or combination of Original keys. E.g
> 5-tuple translated to a session-id from metadata table.
> * Along similar line, derivation of aggregate flow key from multiple keys
> from different Original flows. One flow reports 5-tuple and byte-count.
> The other flow reports 5-tuple and the user-id. The aggregated flow has
> the user-id as key and byte-count as collect. I guess this can be
> considered a correlation.
>
>
> Thanks,
> -Rahul
>
>
>

---

From: Brian Trammell <[email-masked]>
Date: Tue, 21 Feb 2012 11:23:08 +0100

Hi, Rahul,

Thank you very much for the comments! Comments thereon inline.

> There are different cases of correlation which the draft should touch.

I'm not sure I agree, but it depends on what you mean by correlation. More details below.

> the original flows are simply merged (added/replaced/etc) into matching

There is a _lot_ of complexity hiding in that "added/replaced/etc". For "added" (that is, the reduced flow key as per section 5.2 of -a9n represents a different Original Flow, i.e., a completely different set of observed packets from the point of view of the sockets at the endpoints) it is so straightforward it's already covered: this isn't really correlation, but spatial aggregation, in the terminology of this document.

For "replaced" this requires the correlation process to know the configuration of the Original Exporters so that it can differentiate Original Flows representing different observed packets from the endpoint POV from Original Flows representing the same packets. This is where correlation gets interesting, and it's my opinion that it's also out of scope for this document.

For "etc" -- here's where it gets interesting, because there might be significant analysis behind trying to figure out what you should do with a set/cluster of N flows that look like each other. And now we're out of scope for the WG, IMO.

> Srcip, dstip, transaction_delay, { List of {src-interface, byte } }

Correlating {5-tuple, src-interface, octet-count} and {5-tuple, delay} into {5-tuple, src-interface, octet-count, delay} is a completely different operation than aggregating {5-tuple, src-interface, octet-count, delay} into {src address, delay}, for example, and it really doesn't make any sense to me to try to wedge these into the same framework. Sure, it makes sense for a lot of applications to correlate before aggregating (and the document touches on this already) but that doesn't mean they're the same operation.

I will note that this second example (tweaked; see next para) would be a good basis for a new draft on an intermediate correlation process. But I don't think that draft is this draft.

(As an aside, I find this example scary because it hides a key flow attribute, byte count, behind 6313 structured data, without any particular reason to do so. I am concerned that such overuse of 6313 poses a serious threat to the interoperability and adoption of IPFIX. But that is out of scope for this email.)

So, with respect to the inclusion of these comments in the draft, I am strongly disinclined to do so.  As author, I'll note that we decided that correlation was explicitly out of scope for this draft (paragraph 4 of section 3). As a WG contributor, I'll note that correlation is explicitly not mentioned in the charter item for this draft, and that correlation and aggregation are treated separately in 5982 and 6183. So repurposing this draft at this point would go against what I believe is the sense of the WG to date with respect to the relationship between correlation and aggregation, so I wouldn't be comfortable creeping the scope of the draft without a recharter.

> Section 5.1.1 Distributing values across Interval

I don't see how the result of that flow is an Aggregated Flow as per section 2.

> data exported by the aggregator for each template.

You can always keep this from happening by lengthening the time you are willing to wait for aggregation for active + idle + export queue delay + 0.5RTT + collector queue delay; in most sane cases the active timeout should dominate this. 

It probably makes sense to note this in section 6 though.

> Section 5.2 Spatial aggregation

Interesting suggestions, but I didn't (and can't now) see any way to apply them without inventing new Information Elements to hold the combined keys; the idea was to keep the examples as simple and understandable as possible without confusing the issue by adding additional irrelevant detail. However, it does make it difficult to 

Let me think a bit and see if I can come up with a good combined-key example.

Best regards,

Brian

---

From: Rahul Patel <[email-masked]>
Date: Tue, 21 Feb 2012 10:34:13 -0500

Hi Brian,

Comments inline..[RP]

On 2/21/12 5:23 AM, "Brian Trammell" <[email-masked]> wrote:

>There is a _lot_ of complexity hiding in that "added/replaced/etc". For
>"added" (that is, the reduced flow key as per section 5.2 of -a9n
>represents a different Original Flow, i.e., a completely different set of
>observed packets from the point of view of the sockets at the endpoints)
>it is so straightforward it's already covered: this isn't really
>correlation, but spatial aggregation, in the terminology of this document.
>
>For "replaced" this requires the correlation process to know the
>configuration of the Original Exporters so that it can differentiate
>Original Flows representing different observed packets from the endpoint
>POV from Original Flows representing the same packets. This is where
>correlation gets interesting, and it's my opinion that it's also out of
>scope for this document.
>
>For "etc" -- here's where it gets interesting, because there might be
>significant analysis behind trying to figure out what you should do with
>a set/cluster of N flows that look like each other. And now we're out of
>scope for the WG, IMO.

[RP] 'added/replaced/etc' referred to non-key fields operation during
aggregation. Counters will be added, min and max values may be replaced,
averages would be recalculated, etc.

>Correlating {5-tuple, src-interface, octet-count} and {5-tuple, delay}
>into {5-tuple, src-interface, octet-count, delay} is a completely
>different operation than aggregating {5-tuple, src-interface,
>octet-count, delay} into {src address, delay}, for example, and it really
>doesn't make any sense to me to try to wedge these into the same
>framework. Sure, it makes sense for a lot of applications to correlate
>before aggregating (and the document touches on this already) but that
>doesn't mean they're the same operation.
>
>I will note that this second example (tweaked; see next para) would be a
>good basis for a new draft on an intermediate correlation process. But I
>don't think that draft is this draft.
>
>(As an aside, I find this example scary because it hides a key flow
>attribute, byte count, behind 6313 structured data, without any
>particular reason to do so. I am concerned that such overuse of 6313
>poses a serious threat to the interoperability and adoption of IPFIX. But
>that is out of scope for this email.)
>
>So, with respect to the inclusion of these comments in the draft, I am
>strongly disinclined to do so.  As author, I'll note that we decided that
>correlation was explicitly out of scope for this draft (paragraph 4 of
>section 3). As a WG contributor, I'll note that correlation is explicitly
>not mentioned in the charter item for this draft, and that correlation
>and aggregation are treated separately in 5982 and 6183. So repurposing
>this draft at this point would go against what I believe is the sense of
>the WG to date with respect to the relationship between correlation and
>aggregation, so I wouldn't be comfortable creeping the scope of the draft
>without a recharter.

[RP] I am fine with a different draft and reference it in this draft.
 
>I don't see how the result of that flow is an Aggregated Flow as per

[RP] It is an aggregated flow. It is very similar to Proportional Uniform
Distribution (section 5.1.1) but it is more accurate

   |                |                |                |
   | |<--Flow A-->| |                |                |
   |       10       |                |                |
   |                |                |                |
   |        |<--Flow B-->|           |                |
   |            6   | 2              |                |  <== Bytes in each
interval
   |                |                |                |
   |          |<-------------Flow C-------------->|   |
   |             4  |    32          |   10           |  <== Bytes in each
interval
   |                |                |                |
   |   interval 0   |   interval 1   |   interval 2   |

Using Synchronized method
=========================
Aggregated Flow will have following bytes
Interval 0 = (10 + 6 + 4) = 20
Interval 1 = (0 + 2 + 32) = 34
Interval 2 = (0 + 0 + 10) = 10

Using Proportional Uniform Distribution,
========================================
(assuming Interval is 30 seconds
 Flow B overlapped 15 secs of interval 0, 7.5 secs interval 1)
 Flow C overlapped 10 secs of interval 0, 30 secs of interval 1, 20 secs
interval 2)

Aggregate Flow will have following bytes
Interval 0 = (10 + (6+2)*(15/22.5) + 46*(10/60)) = 10+5.33+7.66 = 23
Interval 1 = (0 + (6+2)*(7.5/22.5) + 46*(30/60)) = 0+2.66+23    = 25.66

Interval 0 = (0 + 0 + 46*(20/60))                = 0+0+15.33    = 15.33

>You can always keep this from happening by lengthening the time you are
>willing to wait for aggregation for active + idle + export queue delay +
>0.5RTT + collector queue delay; in most sane cases the active timeout
>should dominate this.
>

[RP] Some device may not want hold the data for longer to avoid larger
memory foot print. In those case it may be desired to have such counters.
I guess one could say that it is implementation detail.

Thanks,

-Rahul

>Interesting suggestions, but I didn't (and can't now) see any way to
>apply them without inventing new Information Elements to hold the
>combined keys; the idea was to keep the examples as simple and
>understandable as possible without confusing the issue by adding
>additional irrelevant detail. However, it does make it difficult to
>

---

From: Brian Trammell <[email-masked]>
Date: Tue, 21 Feb 2012 16:55:28 +0100

Hi, Rahul,

Comments on comments on comments... :)

> [RP] I am fine with a different draft and reference it in this draft.

This draft already makes a forward reference to a future correlation draft. Given that a correlation draft would at the earliest be ready for publication in the late 2013-2014 timeframe, it is IMO unrealistic to hold the aggregation draft up waiting for it by referencing it normatively.

> Distribution (section 5.1.1) but it is more accurate

Ah. Okay. I see what you mean. Since in this case the Aggregation Process has access to the original packet timings from the packets making up the Original Flow, is this not covered by Direct distribution?

> I guess one could say that it is implementation detail.

Yes, but not holding the data for longer is basically equivalent to having a shorter active timeout, no?

Best regards,

Brian

---

From: Rahul Patel <[email-masked]>
Date: Tue, 21 Feb 2012 11:26:03 -0500

Hi Brain,

"This draft already makes a forward reference to a future correlation
draft. Given that a correlation draft would at the earliest be ready for
publication in the late 2013-2014 timeframe, it is IMO unrealistic to hold
the aggregation draft up waiting for it by referencing it normatively."

[RP] agree.

Regarding Direct distribution - I am not sure if it covers the
synchronized method. The description in direct distribution is not enough.
May be an example would help.

Regarding the late counter/active timeout - In most cases shorter active
timeout would work.

Thanks

-Rahul

On 2/21/12 10:55 AM, "Brian Trammell" <[email-masked]> wrote:

>This draft already makes a forward reference to a future correlation
>draft. Given that a correlation draft would at the earliest be ready for
>publication in the late 2013-2014 timeframe, it is IMO unrealistic to
>hold the aggregation draft up waiting for it by referencing it
>normatively.
>
>Ah. Okay. I see what you mean. Since in this case the Aggregation Process
>has access to the original packet timings from the packets making up the
>Original Flow, is this not covered by Direct distribution?
>

---

From: Benoit Claise <[email-masked]>
Date: Fri, 24 Feb 2012 11:48:35 +0100

Rahul,

Thanks for your review.
>> Srcip, dstip, transaction_delay, { List of {src-interface, byte } }
Maybe we can point to RFC6313 as a potential solution, but not specify 
the rules in this draft.
Btw, the rules would be very difficult to specify.
So, some generic text.
>> Section 5.1.1 Distributing values across Interval
Make sense to me.
>> data exported by the aggregator for each template.
So you want something similar to "The Metering Process Statistics Option 
Template" from RFC5101, http://tools.ietf.org/html/rfc5101#page-23, but 
for the Aggregation Intermediate Aggregation Process, right?

Regards, Benoit.
> _______________________________________________
> IPFIX mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/ipfix
>
>

---

From: Rahul Patel <[email-masked]>
Date: Fri, 24 Feb 2012 10:07:50 -0500

Hi Benoit,

>Btw, the rules would be very difficult to specify.

[RP] That would be good.

>for the Aggregation Intermediate Aggregation Process, right?

[RP] exactly.

-Rahul

---

From: Brian Trammell <[email-masked]>
Date: Fri, 24 Feb 2012 22:24:31 +0100

>> for the Aggregation Intermediate Aggregation Process, right?

Still not convinced this is needed given the nature of active timeouts; however, that may well be an implementation-specific (or application-specific) opinion. 

In that case, I agree we should add it but I'm not convinced that this is an aggregation-specific thing. If you're going to count records that were dropped at an intermediate process due to missing a time window, couldn't this affect non-aggregation intermediate processes as well? How about adding this to MEDPROTO instead?

Best regards,

Brian

---

From: Benoit Claise <[email-masked]>
Date: Mon, 27 Feb 2012 10:53:30 +0100

Brian, Rahul,
> In that case, I agree we should add it but I'm not convinced that this is an aggregation-specific thing. If you're going to count records that were dropped at an intermediate process due to missing…
That makes sense in the MEDPROTO  
(http://tools.ietf.org/html/draft-ietf-ipfix-mediation-protocol-00)
We would need to take as a basis the  "The Metering Process Reliability 
Statistics Options Template" from 
http://tools.ietf.org/html/draft-ietf-ipfix-protocol-rfc5101bis-00#page-26, 
which solves already one issue compared to RC5101, by inserting the 
"meteringProcessId".
See my comments in UPPER CASE.

    (scope) observationDomainId
                            An identifier of an Observation Domain that
                            is locally unique to the Exporting Process.
                            This Information Element MUST be defined as a
                            Scope Field.

    (scope) meteringProcessId  =>  NEED IAPPROCESSID
                            The identifier of the Metering Process for
                            which lack of reliability is reported. This
                            Information Element MUST be defined as a
                            Scope Field.

    ignoredPacketTotalCount  =>  NEED SOMETHING SUCH AS IGNOREDFLOW..
                            The total number of IP packets that the
                            Metering Process did not process.

    ignoredOctetTotalCount  =>  DON'T NEED THIS ONE
                            The total number of octets in observed  IP
                            packets that the Metering Process did not
                            process.

    time first packet ignored  =>  THIS RELATES TO THE FLOW, BUT THE IE MIGHT
                            BE THE SAME
                            The timestamp of the first IP packet that was
                            ignored by the Metering  Process.  For this
                            timestamp, any of the following timestamp can
                            be used: observationTimeSeconds,
                            observationTimeMilliseconds,
                            observationTimeMicroseconds, or
                            observationTimeNanoseconds.

    time last packet ignored
                            The timestamp of the last IP packet that was
                            ignored by the Metering  Process.  For this
                            timestamp, any of the following timestamp can
                            be used: observationTimeSeconds,
                            observationTimeMilliseconds,
                            observationTimeMicroseconds, or
                            observationTimeNanoseconds.

would this work?

Regards, Benoit.