Subject: [ippm] Revised: Review of draft-ietf-ippm-metric-registry-10 and draft-ietf-ippm-initial-registry-02

From: Michelle Cotton <[email-masked]>
Date: Tue, 7 Mar 2017 19:06:40 +0000

REVISED:
It looks like in my message below that I copied an old version of the review.
It did not include the other registrations in the Performance Metrics Registry from section 8.
Added them below.
Thanks,
Michelle

-----Original Message-----
From: Michelle Cotton <[email-masked]>
Date: Tuesday, March 7, 2017 at 10:00 AM
To: "MORTON, ALFRED C (AL)" <[email-masked]>
Cc: "[email-masked]" <[email-masked]>, Amanda Baber <[email-masked]>, Sabrina Tanamal <[email-masked]>
Subject: Review of draft-ietf-ippm-metric-registry-10 and draft-ietf-ippm-initial-registry-02

    Attn: Al Morton and IPPM Working Group
    
    At the request of Al Morton, I have re-reviewed the IPPM documents (draft-ietf-ippm-metric-registry-10 and draft-ietf-ippm-initial-registry-02) with a focus on the registry actions that are being requested by IANA Services and making sure we understand what is being requested.
    
    Apologies for the long message below.  There was lots of registry actions to work with.  ☺
    
    NOTE: The registry mock-ups below and initial entries may not perfectly line-up so please ignore spacing.
    
    We understand the draft-ietf-ippm-metric-registry-10 document to request the following:
    
    1) Registration of a URI prefix called urn:ietf:metrics: in the following registry:
    http://www.iana.org/assignments/params/params.xhtml#urn-subnamespaces
    
    The entry, “metrics” will look something like this:
    
    metrics      RFC-ietf-ippm-metric-registry      TBD (potentially http://www.iana.org/assignments/perf-metric-parameters)
    
    2) A new set of registries will be created at TBD (potentially 
    http://www.iana.org/assignments/perf-metric-parameters)

    IETF URN Sub-namespace for Metrics (urn:ietf:metrics)
    Registration Procedures: ??
    Reference: RFC-ietf-ippm-metric-registry
    
    Registered Metric Identifiers
    
    perf      RFC-ietf-ippm-metric-registry   

    At a URL TBD (potentially http://www.iana.org/assignments/perf-metric-parameters)
    
    Performance Metric Name Elements - Metric Types
    Registration Procedures: ??
    Reference: RFC-ietf-ippm-metric-registry
    
    MetricType   Description                          	Reference
    ————————————————————————————————————
    RTDelay    	Round Trip Delay
    RTDNS     	 Response Time Domain Name Service
    OWDelay   	 One Way Delay
    RTLoss    	 Round Trip Loss
    OWLoss   	  One Way Loss
    OWPDV    	  One Way Packet Delay Variation
    OWIPDV   	  One Way Inter-Packet Delay Variation
    OWReorder 	 One Way Packet Reordering
    OWDuplic   	One Way Packet Duplication
    OWBTC     	 One Way Bulk Transport Capacity
    OWMBM     	 One Way Model Based Metric
    SPMonitor  	Single Point Monitor
    MPMonitor 	 Multi-Point Monitor

    Performance Metric Name Elements - Methods
    Registration Procedures: ??
    Reference: RFC-ietf-ippm-metric-registry
    
    Methods 		Description              		       		Reference
    ———————		———————————————————————————
    Active 		depends on a dedicated measurement packet stream and observations of the stream
    Passive 		depends *solely* on observation of one or more existing packet streams
    HybridType1  	obervations on one stream that combine both active and passive methods
    HybridType2  	obervations on two or more streams that combine both active and passive methods
    Spatial      	Spatial Metric of RFC5644
    
    Performance Metric Name Elements - SubType Methods
    Registration Procedures: ??
    Reference: RFC-ietf-ippm-metric-registry
    
    SubType Methods 		Description                     		Reference
    ————————————————		———————————
    ICMP 			Internet Control Message Protocol
    IP 			Internet Protocol
    DSCPxx 			where xx is replaced by a Diffserv code point
    UDP			User Datagram Protocol
    TCP 			Transport Control Protocol
    Poisson 			Packet generation using Poisson distribution
    Periodic 		Periodic packet generation
    PayloadxxxxB 		where xxxx is replaced by an integer, the number of octets in the Payload
    SustainedBurst 		Capacity test, worst case
    StandingQueue 		test of bottleneck queue behavior
    @@@@  		<add others from MBM draft?>
    
    Performance Metric Name Elements - Units
    Registration Procedures: ??
    Reference: RFC-ietf-ippm-metric-registry
    
    Units 		Description   	           		       	Reference
    —————————	————————————————————————
    Seconds
    RatioPercent 	value multiplied by 100
    BPS          	Bits per Second
    EventTotal   	for unit-less counts
    Multiple     	more than one type of unit
    Enumerated  	 a list of outcomes
    Unit-less
    
    Performance Metric Name Elements - Output
    Registration Procedures: ??
    Reference: RFC-ietf-ippm-metric-registry
    
    Units 		Description   	           		       	Reference
    —————————	————————————————————————
    Singleton    	sometimes called raw data
    Minimum
    Maximum
    Median
    Mean
    95Percentile 	95th Percentile
    99Percentile 	99th Percentile
    StdDev 		Standard Deviation
    Variance
    PFI 		Pass, Fail, Inconclusive
    FlowRecords  	descriptions of flows observed

    NOTE:  For all the above entries, the reference will be RFC-ietf-ippm-metric-registry.  Please confirm what the registration procedures should be for adding to all the above registries.
    
    NOTE:  You will notice that I am suggesting breaking up the actual element registration and description in the above element registries.  I think it is a cleaner approach for the registry fields, however if you feel that they should be listed together with the description in parenthesis, please let us know.

    3) Creation of a new registry called Registered Performance Metrics, populated with the following initial assignments
    
    Registered Performance Metrics
    Registration Procedures: Expert Review
    
    Identifier	Name  URIs  Description  Reference   Change Controller  Version
    0      Reserved
    TBD1   RTDelay_Active_IP-UDP-Poisson_RFCXXXXsecY_Seconds_95percentile  (from section 4)
    TBD2   OWPDV_Active_IP-UDP-Poisson_RFCXXXXsecY_Seconds_95Percentile  (from section 5)
    TBD3   RTDNS_Active_IP-UDP-Poisson_RFCXXXXsecY_Seconds_95Percentile (from section 6)
    TBD4   OWDelay_Active_IP-UDP-Poisson-Payload250B_RFCXXXXsecY_Seconds_<statistic> (from section 7)
    TBD5   OWDelay_Active_IP-UDP-Poisson-Payload250B_RFCXXXXsecY_Seconds_95Percentile (from section 7)
    TBD6   OWDelay_Active_IP-UDP-Poisson-Payload250B_RFCXXXXsecY_Seconds_Mean (from section 7)
    TBD7   OWDelay_Active_IP-UDP-Poisson-Payload250B_RFCXXXXsecY_Seconds_Min (from section 7)
    TBD8   OWDelay_Active_IP-UDP-Poisson-Payload250B_RFCXXXXsecY_Seconds_Max (from section 7)
    TBD9   OWDelay_Active_IP-UDP-Poisson-Payload250B_RFCXXXXsecY_Seconds_StdDev (from section 7)
    TBD10 OWDelay_Active_IP-UDP-Periodic-Payload142B_RFCXXXXsecY_Seconds_<statistic> (from section 8)
    TBD11 OWDelay_Active_IP-UDP-Periodic-Payload142B_RFCXXXXsecY_Seconds_95Percentile (from section 8)
    TBD12 OWDelay_Active_IP-UDP-Periodic-Payload142B_RFCXXXXsecY_Seconds_Mean (from section 8)
    TBD13 OWDelay_Active_IP-UDP-Periodic-Payload142B_RFCXXXXsecY_Seconds_Min (from section 8)
    TBD14  OWDelay_Active_IP-UDP-Periodic-Payload142B_RFCXXXXsecY_Seconds_Max (from section 8)
    TBD15  OWDelay_Active_IP-UDP-Periodic-Payload142B_RFCXXXXsecY_Seconds_StdDev (from section 8)
    16-64511 Unassigned
    64512 to 65536 Reserved for private use
    
    NOTE:  For the above registrations, the summary information in the table will be completed using the information from draft-ietf-ippm-initial-registry.  The URI column will link to a template where the complete information about the registration can be found (copied directly from ietf-ippm-initial-registry).
    
    NOTE:  I suggest re-arranging some of the requests for registry creation so that they flow in order a bit better.  Some wording changes may be needed to further clarify and reflect what actions are being requested if what we outlined above is indeed correct.  If I missed anything, please do not hesitate to let me know.  If anything in my review above is not clear please let me know.
    
    We are happy to review new versions of the document.  
    We will also be in Chicago if meeting in person is easier :)

    Best regards,
    
    Michelle Cotton
    Protocol Parameters Engagement Manager
    IANA Services

---

From: "MORTON, ALFRED C (AL)" <[email-masked]>
Date: Wed, 8 Mar 2017 00:45:15 +0000

Hi Michelle,

Thanks very much for your early review and comments.
As you said, it's a long message, so I'll pull out a 
questions/issues and respond to them here. Revised 
versions of the drafts will follow shortly.

In Action 1), in addition to the new URI prefix 
called urn:ietf:metrics:  
we are also requesting new URI prefix 
called urn:ietf:metrics:perf  
The registration procedures for sub-namespaces of
urn:ietf:metrics: and urn:ietf:metrics:perf:
are specified in the new text of Section 10.1 in
draft-ietf-ippm-metric-registry (next version, 11).

The additional assignment of urn:ietf:metrics:perf
requires corresponding changes to Actions 2) and 3).

In Action 2), I think I understand that each Name Element
(such as MetricType and Method) will be a separate registry.
I have written a new section 10.2 in 
draft-ietf-ippm-metric-registry-11 to take this 
organization of the info into account
and provide the registration procedures (Expert Review).
The registries you created below look fine to me, great!
Breaking up the Names and Descriptions (as you did) is fine.

In Action 3), I caused some confusion by including the 
metric names with <statistic> at the end.  These were meant
to illustrate how we are substituting one of the Name elements
for <statistic>, but otherwise the name stays the same.
The names with the <statistic> element were not meant to be 
registered.

So, the Performance Metrics Registry should look like this:

    Identifier	Name  URIs  Description  Reference   Change Controller  Version
    0      Reserved
    TBD1   RTDelay_Active_IP-UDP-Poisson_RFCXXXXsecY_Seconds_95percentile  (from section 4)
    TBD2   OWPDV_Active_IP-UDP-Poisson_RFCXXXXsecY_Seconds_95Percentile  (from section 5)
    TBD3   RTDNS_Active_IP-UDP-Poisson_RFCXXXXsecY_Seconds_95Percentile (from section 6)
    TBD4   OWDelay_Active_IP-UDP-Poisson-Payload250B_RFCXXXXsecY_Seconds_95Percentile (from section 7)
    TBD5   OWDelay_Active_IP-UDP-Poisson-Payload250B_RFCXXXXsecY_Seconds_Mean (from section 7)
    TBD6   OWDelay_Active_IP-UDP-Poisson-Payload250B_RFCXXXXsecY_Seconds_Min (from section 7)
    TBD7   OWDelay_Active_IP-UDP-Poisson-Payload250B_RFCXXXXsecY_Seconds_Max (from section 7)
    TBD8   OWDelay_Active_IP-UDP-Poisson-Payload250B_RFCXXXXsecY_Seconds_StdDev (from section 7)
    TBD9   OWDelay_Active_IP-UDP-Periodic-Payload142B_RFCXXXXsecY_Seconds_95Percentile (from section 8)
    TBD10 OWDelay_Active_IP-UDP-Periodic-Payload142B_RFCXXXXsecY_Seconds_Mean (from section 8)
    TBD11 OWDelay_Active_IP-UDP-Periodic-Payload142B_RFCXXXXsecY_Seconds_Min (from section 8)
    TBD12  OWDelay_Active_IP-UDP-Periodic-Payload142B_RFCXXXXsecY_Seconds_Max (from section 8)
    TBD13  OWDelay_Active_IP-UDP-Periodic-Payload142B_RFCXXXXsecY_Seconds_StdDev (from section 8)
    14-64511 Unassigned
    64512 to 65536 Reserved for private use
    
I'll need to change the text of Section 7.1.2 and 8.1.2 of 
draft-ietf-ippm-initial-registry (version 03 when updated)
to try to make this more clear.

Also, there are three sections where the metric entries
in section 7 and 8 will change:

Name                    (7.1.2)
Description             (7.1.4)
Output
 - Reference Definition (7.4.2)

So, watch for the updates of these two drafts shortly.

thanks again!
Al

> From: Michelle Cotton [mailto:[email-masked]]
> Sent: Tuesday, March 07, 2017 2:07 PM
> To: MORTON, ALFRED C (AL)
> Cc: [email-masked]; Amanda Baber; Sabrina Tanamal
> Subject: Revised: Review of draft-ietf-ippm-metric-registry-10 and
> draft-ietf-ippm-initial-registry-02
> 
> REVISED:
> Cc: "[email-masked]" <[email-masked]>, Amanda Baber
> <[email-masked]>, Sabrina Tanamal <[email-masked]>
>     At the request of Al Morton, I have re-reviewed the IPPM documents
> (draft-ietf-ippm-metric-registry-10 and draft-ietf-ippm-initial-
> registry-02) with a focus on the registry actions that are being
> requested by IANA Services and making sure we understand what is being
> requested.
> 
>     NOTE: The registry mock-ups below and initial entries may not
> perfectly line-up so please ignore spacing.
> 
>     metrics      RFC-ietf-ippm-metric-registry      TBD (potentially
>     NOTE:  For all the above entries, the reference will be RFC-ietf-
> ippm-metric-registry.  Please confirm what the registration procedures
> should be for adding to all the above registries.
> 
>     NOTE:  You will notice that I am suggesting breaking up the actual
> element registration and description in the above element registries.  I
> think it is a cleaner approach for the registry fields, however if you
> feel that they should be listed together with the description in
> parenthesis, please let us know.
> 
> 
>     3) Creation of a new registry called Registered Performance Metrics,
> populated with the following initial assignments
> 
>     TBD1   RTDelay_Active_IP-UDP-
> Poisson_RFCXXXXsecY_Seconds_95percentile  (from section 4)
>     TBD4   OWDelay_Active_IP-UDP-Poisson-
> Payload250B_RFCXXXXsecY_Seconds_<statistic> (from section 7)
>     TBD5   OWDelay_Active_IP-UDP-Poisson-
> Payload250B_RFCXXXXsecY_Seconds_95Percentile (from section 7)
>     TBD6   OWDelay_Active_IP-UDP-Poisson-
> Payload250B_RFCXXXXsecY_Seconds_Mean (from section 7)
>     TBD7   OWDelay_Active_IP-UDP-Poisson-
> Payload250B_RFCXXXXsecY_Seconds_Min (from section 7)
>     TBD8   OWDelay_Active_IP-UDP-Poisson-
> Payload250B_RFCXXXXsecY_Seconds_Max (from section 7)
>     TBD9   OWDelay_Active_IP-UDP-Poisson-
> Payload250B_RFCXXXXsecY_Seconds_StdDev (from section 7)
>     TBD10 OWDelay_Active_IP-UDP-Periodic-
> Payload142B_RFCXXXXsecY_Seconds_<statistic> (from section 8)
>     TBD11 OWDelay_Active_IP-UDP-Periodic-
> Payload142B_RFCXXXXsecY_Seconds_95Percentile (from section 8)
>     TBD12 OWDelay_Active_IP-UDP-Periodic-
> Payload142B_RFCXXXXsecY_Seconds_Mean (from section 8)
>     TBD13 OWDelay_Active_IP-UDP-Periodic-
> Payload142B_RFCXXXXsecY_Seconds_Min (from section 8)
>     TBD14  OWDelay_Active_IP-UDP-Periodic-
> Payload142B_RFCXXXXsecY_Seconds_Max (from section 8)
>     TBD15  OWDelay_Active_IP-UDP-Periodic-
> Payload142B_RFCXXXXsecY_Seconds_StdDev (from section 8)
>     16-64511 Unassigned
>     NOTE:  For the above registrations, the summary information in the
> table will be completed using the information from draft-ietf-ippm-
> initial-registry.  The URI column will link to a template where the
> complete information about the registration can be found (copied
> directly from ietf-ippm-initial-registry).
> 
>     NOTE:  I suggest re-arranging some of the requests for registry
> creation so that they flow in order a bit better.  Some wording changes
> may be needed to further clarify and reflect what actions are being
> requested if what we outlined above is indeed correct.  If I missed
> anything, please do not hesitate to let me know.  If anything in my
> review above is not clear please let me know.
>