Subject: RE: [ipfix] bytes and packet counters, tstamp of last report

From: "MEYER,JEFFREY D (HP-Cupertino,ex1)" <[email-masked]>
Date: Thu, 9 Oct 2003 11:32:13 -0400

Maurizio,

  Thanks for your comment.  You are correct, the current wording in
the Info Model was lifted from previous content and is ambiguous.
This is a problem.

  Following the existing widely deployed Netflow v2,5,7 behavior 
out there, the wording should be:

    The packet count is the count since the last report for this 
    flow if any.

  Wording about "running counter" should be removed.

  The unidirectional vs. bidirectional statement is also somewhat
ambiguous, but may be acceptable.

  Whether or not a separate "PacketCounter" info element is required
for IPFIX I think is open, since it hasn't been required up to
now.  But if it is introduced, the info model should include an
additional annotation to distinguish "counters" from "counts".

Regards,

  Jeff Meyer

> -----Original Message-----
> From: Maurizio Molina [mailto:[email-masked]]
> Sent: Thursday, October 09, 2003 2:29 AM
> To: MEYER,JEFFREY D (HP-Cupertino,ex1)
> Cc: 'Tal Givoly'; [email-masked]; [email-masked]
> Subject: Re: [ipfix] bytes and packet counters, tstamp of last report
> 
> 
> Jeff,
> I posted my original message because I think that the 
> interpretation of 
> an information model should be univoque, and therefore a 
> statement like 
> "this counter can be this or that..." doesn't make sense (and in this 
> respect, I think that the info model is actually broken).
> My guess is that this "ambiguity" was put on purpose waiting for an 
> agreement about what the counters should really reperesent 
> (absolute or 
> delta).
>  From the replies on the ML, it seems that this is still a subject of 
> discussion.
> Therefore, I think the correct way to do is to provide 
> support, in the 
> information model, for both type of counters as separate information 
> elements (because sub-typing s not allowed, as Stewart pointed out). 
> Then the discussion on the merits of  absolute and delta 
> counters should 
> turn out in indications (in the protocol specification) about whether 
> these counters SHOULD, MAY, MUST, etc.... be exported.
> Ah, BTW, there's also the same issue for the 
> droppedPacketCount (6.22) 
> and droppedByteCount (6.26).
> Regards,
> Maurizio
> 
> MEYER,JEFFREY D (HP-Cupertino,ex1) wrote:
> 
> >Hi,
> >
> >
> >  In some use cases counters may have desirable properties as Tal
> >has described.  But like many things, counters also have their 
> >limitations and downsides.
> >
> >  In the basic Flow Export scenario which is exemplified by existing
> >Netflow implementations, a flow record indicates a summary of
> >observed behavior.
> >
> >  Consider two flow events:
> >
> >      sourceAddress=1.2.3.4
> >      destinationAddress=1.2.3.5
> >      packetCount=1000
> >      byteCount=100000
> >      flowCreationTime=2003-10-08T10:00:00Z
> >      flowEndTime=2003-10-08T10:00:05Z
> >      TcpControlBits=0x13
> >
> >
> >      sourceAddress=1.2.3.4
> >      destinationAddress=1.2.3.6
> >      packetCount=1000
> >      byteCount=200000
> >      flowCreationTime=2003-10-08T10:00:02Z
> >      flowEndTime=2003-10-08T10:00:06Z
> >      TcpControlBits=0x13
> >
> >  Each of these completely describes a conversation, as it 
> includes both
> >  the fin and syn TCP flags.  Start time, end time and 
> absolute counts.
> >
> >  Given the large percentage of flows which when exported, describe a
> >  complete conversation, I don't see the advantage of using counters.
> >
> >  I'm not even sure how one would propose the use of counters as an
> >  alternative.  Would you require two records be sent for each
> >  flow (one with byteCounter and packetCounter=0 and another with
> >  the final value?)
> >
> >  Since most flows can be represented by a single record, doubling
> >  the number of records to accomodate counters seems a bit odd.
> >  Counters also require the holding of state, which produces
> >  a much greater burden on the collector.  Since the necessary
> >  state must be held by the observer, why double the workload?
> >  I.e. a single   counter is pretty useless.  I need to delta it
> >  against a previously observed matching counter to arrive at a
> >  quantity.
> >
> >
> >  For information models such as PSAMP, there may be a compelling
> >  use case.
> >
> >  So I imagine adding an explicit annotation in the information
> >  model for counters would be worthwile.  However, I think that
> >  counter behavior such as when it is zeroed out and its rollover
> >  behavior need to be appropriately documented and thought out.
> >  (e.g. on restart, the counters presumably reset to zero, so
> >  here is additional state a collector may need to monitor to
> >  reconcile (if possible) existing observations).
> >
> >  For IPFIX's base information model, I would strongly discourage
> >  "fixing" something which isn't broken.
> >
> >
> >Regards,
> >
> >  Jeff Meyer
> >
> >_________________________________________________________________
> >Get MSN 8 Dial-up Internet Service FREE for one month.  
> Limited time offer--
> >
> >sign up now!   http://join.msn.com/?page=dept/dialup
> >
> >  
> >
> >>-----Original Message-----
> >>From: majordomo listserver 
> >>[mailto:[email-masked]]On Behalf
> >>Of Tal Givoly
> >>Sent: Wednesday, October 08, 2003 9:14 AM
> >>To: [email-masked]; Maurizio Molina
> >>Cc: [email-masked]
> >>Subject: RE: [ipfix] bytes and packet counters, tstamp of 
> last report
> >>
> >>
> >>Both counters have value (running and delta). Obviously, they 
> >>must be kept
> >>discrete from an information model perspective as they 
> >>represent different
> >>information. Devices sometimes maintain one, sometimes the 
> other, and
> >>sometimes both. Just as an example, as far as I recall, 
> >>NetFlow v5-8 doesn't
> >>maintain running counters (counter to your suggestion) EVEN 
> THOUGH it
> >>operates over a non-reliable and non-congestion-aware 
> >>transport. Our probe
> >>product emits both for various different purposes.
> >>
> >>Tal
> >>
> >>-----Original Message-----
> >>From: majordomo listserver 
> >>[mailto:[email-masked]]On Behalf
> >>Of Stewart Bryant
> >>Sent: Wednesday, October 08, 2003 8:09 AM
> >>To: Maurizio Molina
> >>Cc: [email-masked]'
> >>Subject: Re: [ipfix] bytes and packet counters, tstamp of 
> last report
> >>
> >>
> >>
> >>
> >>Maurizio Molina wrote:
> >>
> >>    
> >>
> >>>Hi,
> >>>the IPFIX info model states (sec. 6.10) that
> >>>
> >>>   The packet count can be a running counter and is the 
> >>>      
> >>>
> >>count from the
> >>    
> >>
> >>>  beginning of the flow establishment.
> >>>
> >>>  The packet count can be a delta counter and is the count 
> since the
> >>>  last report for this flow.
> >>>
> >>>      
> >>>
> >>I would prefer that we only supported running counters because:
> >>
> >>a) Supporting two types is more scope for non-interworking, and
> >>    you can always convert from one to the other at the collector.
> >>
> >>b) Running counters also work over an unreliable transport
> >>    whereas delta counters mandate the use of a reliable transport.
> >>
> >>c) Running counters are most likely what the hardware is keeping
> >>    anyway, therefore delta counters are more work and more storage
> >>    at the exporter.
> >>
> >>If we decide that we need both then we have to represent them
> >>as two different information elements because the info model does
> >>not support sub-typing.
> >>
> >>Stewart
> >>
> >>    
> >>
> >>>(There's the same statement for byte counts in 6.11).
> >>>
> >>>To me, this doesn't clarify wheter it is both...and  (2 
> counters) or
> >>>either .... or (1 counter).
> >>>
> >>>Moreover, there is currently no room for a field containing the
> >>>timestamp of the last report of a flow. I think that keeping this
> >>>timestamp is helpful for many applications and the info 
> model should
> >>>support it.
> >>>Regards,
> >>>Maurizio
> >>>
> >>>
> >>>
> >>>--
> >>>Help        mailto:[email-masked] and say 
> >>>      
> >>>
> >>"help" in message
> >>    
> >>
> >>>body
> >>>Unsubscribe mailto:[email-masked] and say
> >>>"unsubscribe ipfix" in message body
> >>>Archive     http://ipfix.doit.wisc.edu/archive/
> >>>
> >>>
> >>>      
> >>>
> >>--
> >>Help        mailto:[email-masked] and say "help" 
> >>in message
> >>body
> >>Unsubscribe mailto:[email-masked] and say
> >>"unsubscribe ipfix" in message body
> >>Archive     http://ipfix.doit.wisc.edu/archive/
> >>
> >>
> >>--
> >>Help        mailto:[email-masked] and say "help" 
> >>in message body
> >>Unsubscribe mailto:[email-masked] and say
> >>"unsubscribe ipfix" in message body
> >>Archive     http://ipfix.doit.wisc.edu/archive/
> >>
> >>    
> >>
> 
> 
> 

--
Help        mailto:[email-masked] and say "help" in message body
Unsubscribe mailto:[email-masked] and say
"unsubscribe ipfix" in message body
Archive     http://ipfix.doit.wisc.edu/archive/

---

From: "MEYER,JEFFREY D (HP-Cupertino,ex1)" <[email-masked]>
Date: Fri, 10 Oct 2003 14:05:02 -0400

Benoit,
 
  Adding these three information items to the Info Model sounds fine.  
 
   What is the rollover and reset behavior of these counters?  E.g. when an
observer is started due to reboot or other administrative action the
counters are set to zero.  Is there a way to distinguish between a counter
reset and rollover?
 
  Presumably these should be long's in the information model.  Is there a
practical lower bound on an implementation in terms of size, e.g. 32-bits?
 
  Also what are the triggers for these to be exported?  Some timer,
threshold, or unspecified operation?  In any of these cases, is there a
guarantee that there will not be > 1 rollover between exports?
 
  I would think most of these aspects should be captured in the information
model.  Hopefully they also illustrate some of the problematic behavior of
dealing with counters, not insurmountable, but certainly irritating.
 
Regards,

  Jeff Meyer

-----Original Message-----
From: Benoit Claise [mailto:[email-masked]]
Sent: Friday, October 10, 2003 4:11 AM
To: MEYER,JEFFREY D (HP-Cupertino,ex1)
Cc: 'Tal Givoly'; [email-masked]; Maurizio Molina;
[email-masked]

Hi,

Hi,

observed behavior.

      sourceAddress=1.2.3.4

      packetCount=1000

      byteCount=100000

      TcpControlBits=0x13

      sourceAddress=1.2.3.4

      packetCount=1000

      byteCount=200000

      TcpControlBits=0x13

  Each of these completely describes a conversation, as it includes both

  the fin and syn TCP flags.  Start time, end time and absolute counts.

  the final value?)

  quantity.

  use case.

I fully agree with Jeff here regarding the flow records but there are
actually 2 aspects to this problem.

1. 
The flow record. 
There is an expiration process for the flow records.
http://www.ietf.org/internet-drafts/draft-ietf-ipfix-protocol-00.txt
<http://www.ietf.org/internet-drafts/draft-ietf-ipfix-protocol-00.txt> 

4.1 Flow Expiration 

   A Flow is considered to be inactive if no packets belonging to the 

   Flow have been observed at the Observation Point for a given timeout 

   interval otherwise it is considered as an active flow.  

   A Flow can be exported under the following conditions: 

      1. If the Metering Process can detect the end of a Flow, it 

      SHOULD export the Flow Records at the end of the Flow. For 

      example, a Flow generated by TCP [3] type of traffic where the 

      FIN or RST bits indicate the end of the Flow. 

      2. If the Flow has been inactive for a certain period of time. 

      This inactivity timeout SHOULD be configurable, with a minimum 

      value of 0 for an immediate expiration. For example, a Flow 

      generated by UDP [2] type of traffic. 

      3. For long-lasting Flows, the Metering Process SHOULD export the 

      Flow Records on a regular basis. This periodicity SHOULD be 

      configurable. 

      4. If the Metering Process experiences internal constraints, a 

      Flow MAY be forced to expire prematurely (for example, counters 

      wrapping or low memory). 

What the IPFIX information model tries to say is that: a flow expires and we
send the packet/byte count for the flow duration. So this is actually a
running counter for the flow duration.

Note: a long last flow is actually composed a several flows (see condition 3
before). 

Ok, I understand that this could be better explained in the IPFIX
information model draft:

   The packet count can be a running counter and is the count from the

   The packet count can be a delta counter and is the count since the

http://www.ietf.org/internet-drafts/draft-claise-netflow-9-05.txt
<http://www.ietf.org/internet-drafts/draft-claise-netflow-9-05.txt>  speaks
of:

                                            Incoming counter with  

    IN_BYTES                     1    N     length N x 8 bits for bytes  

                                            associated with an IP Flow    

                                            Incoming counter with  

    IN_PKTS                      2    N     length N x 8 bits for  

                                            packets associated 

                                            with an IP Flow  

2. 

The second aspect is actually total counters.

Something that needs to added to the IPFIX protocol draft is described in
its "open issues" section:

   - The proposal on the table is to send a IPFIX Sync (this would be 

   an Options Data Records) message periodically (periodicity is 

   configurable), with the following information (aside the standard 

   IPFix header) 

           * Number of flow records sent (for each template?)  

           * Packets and bytes sent (for each template?) 

So we need to keep a few running counters.

In  http://www.ietf.org/internet-drafts/draft-claise-netflow-9-05.txt
<http://www.ietf.org/internet-drafts/draft-claise-netflow-9-05.txt> , there
are explicitely described:

                                            Counter with length   

                                            N x 8 bits for bytes 

    TOTAL_BYTES_EXP              40   N     for the number of bytes  

                                            exported by the Observation 

                                            Domain 

                                            Counter with length  

                                            N x 8 bits for bytes  

    TOTAL_EXP_PKTS_SENT          41   N     for the number of packets  

                                            exported by the Observation  

                                            Domain  

                                            Counter with length  

                                            N x 8 bits for bytes  

    TOTAL_FLOWS_EXP              42   N     for the number of Flows  

                                            exported by the Observation  

                                            Domain  

I propose to also explicitely described them in the IPFIX information model
draft.

Regards, Benoit.

Regards,

  Jeff Meyer

Get MSN 8 Dial-up Internet Service FREE for one month.  Limited time offer--

sign up now!    http://join.msn.com/?page=dept/dialup
<http://join.msn.com/?page=dept/dialup> 

-----Original Message-----

[ mailto:[email-masked] <mailto:[email-masked]>
]On Behalf

Of Tal Givoly

To:  [email-masked] <mailto:[email-masked]> ; Maurizio Molina

Cc:  [email-masked] <mailto:[email-masked]> 

Subject: RE: [ipfix] bytes and packet counters, tstamp of last report

must be kept

represent different

information. Devices sometimes maintain one, sometimes the other, and

NetFlow v5-8 doesn't

maintain running counters (counter to your suggestion) EVEN THOUGH it

transport. Our probe

Tal

-----Original Message-----

[ mailto:[email-masked] <mailto:[email-masked]>
]On Behalf

Of Stewart Bryant

To: Maurizio Molina

Cc: ' [email-masked] <mailto:[email-masked]> '

Maurizio Molina wrote:

Hi,

count from the

  The packet count can be a delta counter and is the count since the

      

    at the exporter.

not support sub-typing.

Stewart

To me, this doesn't clarify wheter it is both...and  (2 counters) or

timestamp is helpful for many applications and the info model should

support it.

Regards,

Maurizio

--

Help         mailto:[email-masked]
<mailto:[email-masked]>  and say 

"help" in message

body

Unsubscribe  mailto:[email-masked]
<mailto:[email-masked]>  and say

Archive      http://ipfix.doit.wisc.edu/archive/
<http://ipfix.doit.wisc.edu/archive/> 

--

Help         mailto:[email-masked]
<mailto:[email-masked]>  and say "help" 

in message

body

Unsubscribe  mailto:[email-masked]
<mailto:[email-masked]>  and say

Archive      http://ipfix.doit.wisc.edu/archive/
<http://ipfix.doit.wisc.edu/archive/> 

--

Help         mailto:[email-masked]
<mailto:[email-masked]>  and say "help" 

in message body

Unsubscribe  mailto:[email-masked]
<mailto:[email-masked]>  and say

Archive      http://ipfix.doit.wisc.edu/archive/
<http://ipfix.doit.wisc.edu/archive/> 

--

Help         mailto:[email-masked]
<mailto:[email-masked]>  and say "help" in message body

Unsubscribe  mailto:[email-masked]
<mailto:[email-masked]>  and say

Archive      http://ipfix.doit.wisc.edu/archive/
<http://ipfix.doit.wisc.edu/archive/>

---

From: "MEYER,JEFFREY D (HP-Cupertino,ex1)" <[email-masked]>
Date: Fri, 10 Oct 2003 14:36:10 -0400

Benoit,
 
  One other point on your e-mail.  In your discussion of "Long lived flows",
are you saying that if an observed connection lasts say 5 minutes, and
through configuration the system will export a long lived flow after 3
minutes of activity, that the second flow record's numbytes is measured from
the beginning of the flow (e..g it represents period t,t+5m vs. t+3m,t+5m,
where t is the actual start time of the long lived flow).
 
  I would have thought that the second record would indicate flow properties
SINCE any previous flow record.  That was the revised wording I had
suggested based on Maurizio's observation of ambiguity.
 
  If you are saying that the second exported flow for a long lived
connection is cumulative, then you've created a rather dicey problem.
Especially if a collection system is doing aggregation of these flows.
Because instead of a flow being a discrete self-contained piece of
information, one must now determine if it actually represents additional
counts added to flows previously reported.  If so this sets you up to have
to search an ENORMOUS space of old flows to do delta calculations.
 
  My impression of traditional NFvX behavior is that exporting a flow
effectively pops it out of the flow table.  Any subsequent traffic for the
same flow would first create a new flow entry and the counters effectively
reset each time a flow is exported.  Here is the current Cisco
documentation:

http://www.cisco.com/en/US/products/hw/switches/ps4324/products_configuratio
n_guide_chapter09186a008019d0da.html
<http://www.cisco.com/en/US/products/hw/switches/ps4324/products_configurati
on_guide_chapter09186a008019d0da.html> 

Configuring Netflow Aging Parameters
 
You can control when flows are purged from the software flow cache (and, if
configured, reported through NDE) with the configuration aging parameters,
Active and Inactive, of the ip flow-cache timeout command.

wp1011379Active Aging specifies the period of time in which a flow should be
removed from the software flow cache after the flow is created. Generally,
this parameter is used to periodically notify external collection devices
about active flows. This parameter operates independently of existing
traffic on the flow. Active timeout settings tend to be on the order of
minutes (default is 30min). 

  My reading here is that exporting and purging are one in the same.  I.e. I
never get a flow which may reflect counts which were already reported in a
previous flow.
 
  Are you proposing changing this behavior?  If so, this has significant
deliterious impact on collection.  I would not recommend changing.
 
Regards,
 
  Jeff Meyer

-----Original Message-----
From: MEYER,JEFFREY D (HP-Cupertino,ex1) 
Sent: Friday, October 10, 2003 11:05 AM
To: 'Benoit Claise'; MEYER,JEFFREY D (HP-Cupertino,ex1)
Cc: 'Tal Givoly'; [email-masked]; Maurizio Molina;
[email-masked]

Benoit,

 

 
Regards,

  Jeff Meyer

-----Original Message-----
From: Benoit Claise [mailto:[email-masked]]
Sent: Friday, October 10, 2003 4:11 AM
To: MEYER,JEFFREY D (HP-Cupertino,ex1)
Cc: 'Tal Givoly'; [email-masked]; Maurizio Molina;
[email-masked]

Hi,

Hi,

observed behavior.

      sourceAddress=1.2.3.4

      packetCount=1000

      byteCount=100000

      TcpControlBits=0x13

      sourceAddress=1.2.3.4

      packetCount=1000

      byteCount=200000

      TcpControlBits=0x13

  the final value?)

  quantity.

  use case.

1. 
The flow record. 

4.1 Flow Expiration 

      configurable. 

      wrapping or low memory). 

                                            Incoming counter with  

                                            Incoming counter with  

                                            packets associated 

                                            with an IP Flow  

2. 

   IPFix header) 

                                            Counter with length   

                                            N x 8 bits for bytes 

                                            Domain 

                                            Counter with length  

                                            N x 8 bits for bytes  

                                            Domain  

                                            Counter with length  

                                            N x 8 bits for bytes  

                                            Domain  

Regards, Benoit.

Regards,

  Jeff Meyer

  

-----Original Message-----

Of Tal Givoly

must be kept

represent different

NetFlow v5-8 doesn't

transport. Our probe

Tal

-----Original Message-----

Of Stewart Bryant

To: Maurizio Molina

Maurizio Molina wrote:

Hi,

count from the

      

    at the exporter.

not support sub-typing.

Stewart

support it.

Regards,

Maurizio

--

"help" in message

body

      

--

in message

body

--

in message body

    

--