Subject: [PMOL] Fwd: Comments on draft-akhter-opsawg-perfmon-method-02

From: Al Morton <[email-masked]>
Date: Wed, 16 May 2012 10:54:49 -0400

PM Directorate,

I reviewed one of Aamer's passive performance drafts
(attached below).

Aamer has a related draft, would anyone volunteer to
review this one:
http://tools.ietf.org/html/draft-akhter-opsawg-perfmon-ipfix

let me know, and thanks,
Al
PMdir admin

>Date: Sun, 13 May 2012 13:06:56 -0400
>To: "Aamer Akhter (aakhter)" <[email-masked]>,
>From: Al Morton <[email-masked]>
>Subject: Comments on draft-akhter-opsawg-perfmon-method-02
>Cc: [email-masked], [email-masked]
>
>Hi Aamer,
>
>I finally made some time to review one of your drafts:
>http://tools.ietf.org/html/draft-akhter-opsawg-perfmon-method-02
>
>and comments follow below.  Not sure exactly which list
>will be best, so I CC'd ipfix and opsawg.
>
>hope this helps,
>Al
>
>
>-=-=-=-=-=-=-=-=-=-=-=-=-=-
>
>Main Comments:
>
>1. Packet Loss Calculation Method (4.1.1)
>Each metric has a paragraph labeled Calculation Method, but
>they contain a useful discussion of the possible issues instead.
>Many details are left to the implementor, with the likely outcome
>that different implementations will produce different results.
>For example, "synchronization" is mentioned many times, but what
>does this process entail? Try to minimize interpretation,
>perhaps by including a high-level step-by-step procedure for
>processing each arriving packet, both during initiation
>(synchronization, when the first few packets arrive) and
>steady-state processing (which involves using the info from sync).
>Since this draft is proposed for the Standards Track, it
>really needs to provide definitions that will stand-up to
>scrutiny of multiple implementations.
>
>
>2. Packet Loss "Rate" (4.1.3)
>This metric is really a Statistic, or a metric derived from the
>fundamental metrics of Packet Loss and Expected Packet (counts)
>over a particular measurement interval.  Thus, there's no need
>to talk about measurement points, you've already defined these
>in the Loss and Expected metrics.
>Also, this is more accurately named "Loss Ratio", especially when
>expressed as a percentage as you've done.
>
>3. Jitter (4.1.5 and beyond)
>This metric will disappoint for use in SLAs or de-jitter buffer
>sizing if it is based on the RFCC3550 smoothed jitter calculation.
>Regardless of the measurement interval, only the last 16 packets
>have any influence over the current value (which is your Average?)
>
>General Editorial:
>There are a number of sentences that seem to contain alternate
>wordings, as though some deleted text returned somehow. Also,
>running a spell-check will help.
>
>Specific comments follow:
>
>3.1.  Quality of Service (QoS) Monitoring
>
>    The network operator needs to be able to gauge the end user's
>    satisfaction with the network service.  While there are many
>    components of the satisfaction such as pricing, packaging, offering,
>    etc., a major component of satisfaction is delivering a consistent
>    service.
>Consistent Service in what dimensions?  Is Service Performance meant here?
>
>
>4.  New Information Elements
>
>    The information elements are organized into two main groups:
>
>    Transport Layer:  Metrics that might be calculated from observations
>       at higher layers but essentially provide information about the
>       network transport of user date.
>s/date/data/
>
>
>4.1.1.  perfPacketLoss
>
>    Name:  perfPacketLoss
>
>In this section, s/loss packet/lost packet/
>
>Also, what is the 5-tuple? SSRC + Src IP and Port + Dst IP and Port?
>
>At the end:
>    Measurment Timing  To be able to calculate this metric a continuous
>       set of the flow's packets (as each would have an incrementing
>       sequence number) needs to be monitored.  Therefore, per-packet
>       sampling would prevent this metric from being calculated.
>s/per-packet/sub-rate/ ??
>
>
>4.1.2.  perfPacketExpected
>
>    Name:  perfPacketExpected
>...
>    Use and Applications  The perfPacketExpected is a mid-calculation
>       metric used in the generation of perfPacketLossRate.  It is
>       equivilent to the highest received packet sequence number at the
>       time of measurement.
>True when there is no roll-over in Seq numbers.
>
>                       ... As the value only increments when packets
>       are received, packet loss may be occouring at the time of
>       measurement but perfPacketExpected remains constant.
>This is an issue worth mitigating, some sort of average packet rate
>could be used to estimate the number of losses while the Expected count
>remains constant.
>
>
>    Calculation Method:  The subtraction of the last sequence number from
>       the first sequence number in monitoring interval yields the
>       expected count.  As discussed with perfPacketLost, there might be
>       a delay due to synchronization with the flow's sequence numbers
>       and in such times the value of the metric should be set to
>       0xFFFFFFFE.  Care has to be taken to account for cases where the
>       packet's sequence number field wraps.
>This is an opportunity for more specifics, as an example
>(I didn't check the Appendix 3 reference, no access today...)
>
>
>4.1.3.  perfPacketLossRate
>
>    Name:  perfPacketLossPercentage
>These aren't the same!  Which is it?
>
>
>4.1.4.  perfPacketLossEvent
>
>    Name:  perfPacketLossEvent
>Need "consecutive" in the name, or some other way to distinguish
>single losses from "bursts".
>
>...
>    Calculation Method:  This data value is a simplified version of the
>       Lost Packets metric.  Whereas Lost Packets counts individual
>       packet loss, the 'loss event count' metric counts sets of packets
>       that are lost.  For example, in the case of a sequence of packets:
>       1,3,6,7,10 the packets marked 2,4,5,8 and 9 are lost.  So, a total
>How long does the process wait to make the lost determination?
>
>
>4.1.5.  perfPacketInterArrivalJitterAvg
>
>    Name:  perfPacketInterArrivalJitterAvg
>
>    Description:  This metric measures the absolute deviation of the
>       difference in packet spacing at the measurement point compared to
>       the packet spacing at the sender.
>I don't think that's what 3550 Jitter does...
>
>...
>    Use and Applications  The inter arrival jitter data value can be used
>       be network operator to determine the network's impact to the
>       spacing in between a media stream's packets as they traverse the
>       network.
>What about Source Jitter?  this can be very significant, more than network.
>
>Again, Min and Max are statistics.
>
>
>
>4.2.  User and Application Layer
>
>4.2.1.  perfSessionSetupDelay
>
>    Name:  perfSessionSetupDelay
>This seems to be the only two-point metric, worth noting.
>and
>[I-D.ietf-pmol-sip-perf-metrics]
>is an RFC now (which mostly defined metrics, not methods).
>
>

---

From: "Jan Novak (janovak)" <[email-masked]>
Date: Thu, 17 May 2012 12:02:39 +0200

Hi Al,

I will have a look.

Jan

The climate of Edinburgh is such that the weak succumb young .... 
and the strong envy them.
                                 Dr. Johnson

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of
Al Morton
Sent: 16 May 2012 15:55
To: [email-masked]
Subject: [PMOL] Fwd: Comments on draft-akhter-opsawg-perfmon-method-02

PM Directorate,

let me know, and thanks,
Al
PMdir admin

>will be best, so I CC'd ipfix and opsawg.

_______________________________________________
PMOL mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/pmol

---

From: "Jan Novak (janovak)" <[email-masked]>
Date: Tue, 22 May 2012 10:58:00 +0200

Hi Amer,

I have reviewed your draft draft-akhter-opsawg-perfmon-ipfix-02.txt.

There seems to be a lot of text overlap with your methodology
document - section 1,3, 4 could probably be abbreviated or omitted
leaving the document just with raw IPFIX IE specifications or just
add the IE specification as sub-sections or a new section into
the first document ?? 

Section 2 uses definitions from RFC5610 - I think those you use there
are defined in RFC5102 as DataTypeSemantic, units and range while
RFC5610 specifies how this information should be exported - here
you are defining the IE itself so you should use the definitions
from RFC5102

Also the methodology documents already speaks in terms of IPFIX
IEs while you are trying to specify some performance metrics - the
methodology could have names and an exact definitions of the metric
and then a reference which IE represents the particular metric

RFC5102 section 2.1 specifies a template for IEs with a MUST
so the MUST entries should be literally followed in your IEs spec
- namely name, elementID, description, dataType and status.

RFC5102 section 2.1 specifies MAY entries for the template - like
DataTypeSemantic, units, name - might be preferable to follow the
naming as well

You interchanged ElementId with name - ElementId should be the
numerical ID of the particular IE, while name of the IE is actually
missing

Instead of using Observation Point - wouldn't be the scope of the
element
appropriate ?? Or if not then scope should be actually added - are the 
metrics (like perfPacketLoss) applicable to all the traffic seen
by the UUT (or more specifically passing through the Observation
Point) or to just individual flows ?? This should also be part of the
particular metric definition.

Will your IEs be enterprise IEs or IANA ones ??

Section 4.1.2 - Units packets ??

Section 4.1.3 - there is a mis-match between the definition and the
range 
- it should be limited to 0 - 100 + a value when the rate is unknown
This definition is also missing in section 4.1.3 of your methodology

Sections 4.3.1, 4.3.2 - the values are just numbers/ids so units
shouldn't be octets but "none" ??

The IPFIX guys here have had few discussions regarding IE definitions
explosions with all the needs like this - have you thought using RFC6313
now
(structured data) ??

I am not sure I would use RFC2321 as a reference work :-).
huic-ipfix-sipfix is not a work in progress - the ID expired 3 years
ago.
ie-doctors is a WG doc version 2 now -
draft-ietf-ipfix-ie-doctors-02.txt
pmol-metrics-framework is RFC 6390

The document would benefit from running it through spell checker.

Rgds, Jan