Subject: Re: [PMOL] New Version Notification - draft-ietf-xrblock-rtcp-xr-pdv-07.txt

From: Benoit Claise <[email-masked]>
Date: Tue, 02 Oct 2012 16:35:44 +0200

Dear all,

I had a discussion with Dan, and we settle on:
- RFC 6390 template is required for new perf metric definition
- RFC 6390 template is a nice-to-have when we refer to an existing perf 
metric
Nice-to-have because the performance metric reference doesn't always 
include all the required information about: measurement points, 
measurement timing, use and applications, reporting model, etc... but 
focus only on the "Method of Measurement or Calculation"

So basically, at this point, I will clear my DISCUSS-DISCUSS

Somehow, I've been trying to solve a growing problem, and potentially 
bigger problem:

Where does the list of performance metric definitions come from at
the IETF? We have multiple sources:
- IPPM for IP performance metrics.
- RTCP for RTP performance metrics:
- PMOL: Performance Metrics at Other Layers, with
   RFC 6076 on Basic Telephony SIP End-to-End Performance Metrics
- IPFIX will one day or the other exports performance metrics.
   I see for example
   http://tools.ietf.org/html/draft-akhter-opsawg-perfmon-ipfix-03.
   It's again a redefinition, and it should not be!

My concerns are that we start to define performance metrics in different parts
of the IETF, without consistency.
And even finding the performance metrics specified in the IETF is not an easy task.

I've convinced that the community will have solve this problem, and I 
will propose a meeting during the next IETF to try to come up with a 
solution. This meeting should include the PMOL directorate, the XRBLOCK 
chairs, and the IPPM chairs.
If someone not in the mentioned list wants to participate, let me know 
privately.

Regards, Benoit.

> Re: New Version Notification - draft-ietf-xrblock-rtcp-xr-pdv-07.txt 
> Hi Dan
>
> The original intent was that the template would be applied to the 
> original definition of a metric and not to documents in which the 
> metric is referenced. If a separate template is created each time a 
> metric is referenced then there is a risk that we would have multiple 
> definitions of the same metric - which are likely to be inconsistent.
>
> Best Regards
>
> Alan
>
>
> On 10/1/12 2:15 PM, "Dan (Dan)" <[email-masked]> wrote:
>
>     Hi Benoit and Gonzalo,
>
>     I want to make sure that we understand the advice for this I-D and
>     for other that are in process. Is the requirement to include
>     6390-formatted templates in all documents that define new metrics
>     or refer to already defined metrics -- even if they are defined by
>     other SDOs?
>
>     Dan
>
>
>
>
>     *From:* Benoit Claise [mailto:[email-masked]]
>     *Sent:* Monday, October 01, 2012 5:51 PM
>     *To:* Romascanu, Dan (Dan)
>     *Cc:* Gonzalo Camarillo; Qin Wu; [email-masked];
>     [email-masked]
>     *Subject:* Re: New Version Notification -
>     draft-ietf-xrblock-rtcp-xr-pdv-07.txt
>
>
>     Hi Dan, Qin,
>
>     [sorry for the delay in getting back to you]
>     I'm not after duplicating information where it's not necessary.
>     So basically, you want a new registry with new metrics, and the
>     first two are
>           *  0: MAPDV2, Clause 6.2.3.2 of [G.1020
>     <http://tools.ietf.org/html/draft-ietf-xrblock-rtcp-xr-pdv-08#ref-G.1020>
>     ],
>
>           *  1: 2-point PDV, Clause 6.2.4 of [Y.1540
>     <http://tools.ietf.org/html/draft-ietf-xrblock-rtcp-xr-pdv-08#ref-Y.1540>
>     ],
>     So I looked at Clause 6.2.3.2 of [G.1020
>     <http://tools.ietf.org/html/draft-ietf-xrblock-rtcp-xr-pdv-08#ref-G.1020>
>     ], and tried to fill in the RFC 6390 template fro MAPDV2
>       Normative
>
>           o  Metric Name:
>              Mean Absolute Packet Delay Variation 2 (MAPDV2)
>
>           o  Metric Description:
>              The Mean Absolute Packet Delay Variation 2
>              for RTP applications.
>
>           o  Method of Measurement or Calculation:
>              See Clause 6.2.3.2 of [G.1020
>     <http://tools.ietf.org/html/draft-ietf-xrblock-rtcp-xr-pdv-08#ref-G.1020>
>     ]
>
>           o  Units of Measurement:
>              ms or percentile depending on ...
>
>           o  Measurement Point(s) with Potential Measurement Domain:
>              The measurement of these metrics are made at the
>     receiving end of the RTP stream.
>
>           o  Measurement Timing:
>             This metric block relies on the measurement
>             interval given by the value of the "Measurement Duration
>     (Interval)"
>             field in the Measurement Information Block to indicate the
>     span of
>             the report and MUST be sent in the same compound RTCP
>     packet as the
>             Measurement Information Block
>
>      Informative
>
>           o  Implementation:
>              ...
>
>           o  Verification:
>              ...
>
>           o  Use and Applications:
>             For example, applications could use the
>             measurements of these metrics to help adjust the size of
>     adaptive
>             jitter buffers to improve performance.  Network managers
>     can use
>             these metrics to compare actual delay variation to targets
>     (i.e., a
>             numerical objective or Service Level Agreement) to help
>     ensure the
>             quality of real-time application performance.
>
>           o  Reporting Model:
>              Report Block Structure, as specified in <this RFC>
>
>     Obviously, I don't complete all the points, but I believe that you
>     get the message.
>
>     I'm certainly not after redefinitions, but a simple summary, with
>     references when appropriate, to make the lives of people who will
>     be searching through perf metrics... easier.
>     Remember that one day, we should have all these perf metrics
>     listed somewhere.
>
>     You know, simply by trying to fill in this template, I'm wondering?
>     Should the metric name be called RTP MAPDV2?
>     What if IPPM also proposes MAPDV2 for IP packets.
>
>     Regards, Benoit.
>
>         Hi Gonzalo,
>
>         See my previous comment. I respectfully disagree with the need
>         to reformat the metrics definitions using the template defined
>         in RFC 6390 when we re-use metrics already defined in other
>         documents. This is at best a duplication that seems to me
>         unnecessary and at worst risks to introduce inconsistencies
>         between the various definitions for the same metrics.
>
>         Regards,
>
>         Dan
>
>
>
>
>             -----Original Message-----
>             From: Gonzalo Camarillo
>             [mailto:[email-masked]]
>             Sent: Friday, September 21, 2012 11:07 AM
>             To: Qin Wu
>             Cc: [email-masked]; [email-masked];
>             draft-ietf-
>             [email-masked]
>             Subject: Re: New Version Notification -
>             draft-ietf-xrblock-rtcp-xr-pdv-
>             07.txt
>
>             Hi Qin,
>
>             per my conversation with Benoit in the telechat the other
>             day, I
>             understand he wants to make sure all drafts defining this
>             type of metric
>             include the relevant information. A good way to clearly
>             find that
>             information is to use the template in RFC 6390:
>
>             http://tools.ietf.org/html/rfc6390#section-5.4.4
>
>             So, could you please include such a template in the draft?
>
>             We also discussed with Benoit how to populate the template
>             and he was OK
>             with the template having pointers to sections of documents
>             where the
>             information can be found. So, you do not really need to
>             repeat all the
>             information you already added to the draft in the
>             template. You can just
>             point to the relevant section of the draft.
>
>             Thanks,
>
>             Gonzalo
>
>             On 21/09/2012 7:06 AM, Qin Wu wrote:
>
>                 Hi,Benoit:
>                 The PDV draft has already specified what you asked
>                 for. It is a
>                 misundersanding to think PDV doesn't follow
>                 RFC6390 performance metric template.
>                 Personally I think it is a bad idea to seek Pharisaism or
>                 doctrinairism. What is more important is to respect
>                 RFC6390
>
>             performance metric template by facts.
>
>                 If you really want to go doctrinairism, please tell
>                 what need to be
>
>             changed.
>
>
>                 BTW:   The new version has already incorporated extra
>                 proposed change
>
>             by WG after the draft gets back to WG list.
>
>
>                 Regards!
>                 -Qin
>                 ----- Original Message -----
>                 From: <[email-masked]>
>                 <mailto:[email-masked]>
>                 To: <[email-masked]>
>                 <mailto:[email-masked]> ;
>                 <[email-masked]>
>                 <mailto:[email-masked]> ;
>                 <[email-masked]>
>                 <mailto:[email-masked]> ;
>                 <[email-masked]> <mailto:[email-masked]>
>                 Sent: Friday, September 21, 2012 11:48 AM
>                 Subject: New Version Notification -
>                 draft-ietf-xrblock-rtcp-xr-pdv-07.txt
>
>
>
>                 A new version (-07) has been submitted for
>                 draft-ietf-xrblock-rtcp-xr-
>
>             pdv:
>
>                 http://www.ietf.org/internet-drafts/draft-ietf-xrblock-rtcp-xr-pdv-07.
>                 txt
>
>
>                 The IETF datatracker page for this Internet-Draft is:
>                 https://datatracker.ietf.org/doc/draft-ietf-xrblock-rtcp-xr-pdv/
>
>                 Diff from previous version:
>                 http://www.ietf.org/rfcdiff?url2=draft-ietf-xrblock-rtcp-xr-pdv-07
>
>                 IETF Secretariat.
>
>
>
>
>
>