Subject: [PMOL] Thoughts on how we operate as a Directorate

From: Al Morton <[email-masked]>
Date: Mon, 16 Jan 2012 09:47:25 -0500

As this new year goes into full swing, I'd like to offer some
thoughts on how we might operate as a Directorate and
hear from others.

As our web page says: "RFC 6390 is used as a reference for the technical
and process issues." Have your copy handy when reviewing a draft.
A lot of interactions may be streamlined by pointing the authors to 6390,
and mentioning that it is a BCP.

I feel that our reviews should be *early* in the draft development
process, and that no performance-related draft should start IETF Last Call
without previous review against 6390, unless it truly slipped by us
and other folks who know about the Directorate.

There are two ways we can collect drafts for review:

- WG Chairs and ADs can/should suggest drafts to us

- Directorate Members will see drafts, hear about drafts with performance
work, etc.  We are the eyes and ears of the Directorate. We all have friends
working in other areas of IETF, let them know what we're looking for.

I suggest to split the early review into two categories:

1. Quick Scan - If you find a draft, do a quick scan or suggest that
others take a look with a message to pmol-list.  Finding a draft does
not mean that you will be designated to do a review (see below).

2. WG Candidate/Chartered Item - When a draft appears to have enough value
and interest to become a WG item, or the WG Chairs involved recommend a
Performance Metrics Directorate review on their own, then we do a more
complete review against 6390.

If these categories are useful, then I'd like to suggest that *two*
reviewers are assigned to WG Candidate/Chartered drafts and they follow
the draft after early review. This may mean looking at the draft again
in WG or IETF Last Call, assuming it goes that far. I suggest 2 reviewers
while we're all new at this, and to help ensure that one is still around
years later when Last Calls are done.

All drafts identified, quick scans, and reviews should be sent to the
pmol-list and the authors (and possibly the relevant WG chairs)

let us all know what you think,
Al
http://www.ietf.org/iesg/directorate/performance-metrics.html

---

From: Yaakov Stein <[email-masked]>
Date: Mon, 16 Jan 2012 18:05:13 +0000

Al and all, 

Perhaps I am the only one who didn't know about this work,
but I just recently saw  draft-tempia-opsawg-p3m-01 .

While it deals with packet loss and delay measurement,
it doesn't reference any of the IPPM work.
Indeed, there are no "Type-P" measurements (since individual packets are not identified), 
and the measured values do not necessarily directly map to our usual definitions 
(e.g., when color flips occurs every N packets, the averaging is per number of packets, not periodic sampling).

Please note IPR disclosure 1570 which cites European applications 
EP-068189(2008) on packet loss and EP-067991(2009) on delay measurement, 
but many other applications with the same assignee seem to be relevant,
including the more recent EP-066955(2010) and EP-063010(2011).

I think this is a good draft for early review,
and would like to hear discussion in the IPPM WG (before it is possibly closed down).

Y(J)S

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Al Morton
Sent: Monday, January 16, 2012 16:47
To: [email-masked]
Subject: [PMOL] Thoughts on how we operate as a Directorate

_______________________________________________
PMOL mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/pmol

---

From: Al Morton <[email-masked]>
Date: Mon, 16 Jan 2012 14:06:04 -0500

Hi Yaakov,

By jumping on the list with an early review, I'll assume you
agree with suggested methods of operation - that's good.

I agree this is a candidate for early review:
http://tools.ietf.org/html/draft-tempia-opsawg-p3m-01
In fact, I commented on a presentation of the 00 version
in the opsawg session (in Prague last year, IIRC).
I made some fairly serious comments at the mike,
and it will be interesting to see if the latest version
takes any of this into account.  It looks as though the
01 draft adds delay and delay variation measurement details.

The Title:
"A packet-based method for passive performance monitoring"
may explain why this ended up in opsawg last year.
IPPM's charter has been limited to active measurement, and
although there were suggestions to include passive during the
previous re-chartering discussions, no one spoke-up at the critical
point and passive measurement was excluded.

It would also be good to know where this draft stands in opsawg.
I'll inquire about that.
Al

At 01:05 PM 1/16/2012, Yaakov Stein wrote:
>(e.g., when color flips occurs every N packets, the averaging is per 
>number of packets, not periodic sampling).
>

---

From: Yaakov Stein <[email-masked]>
Date: Tue, 17 Jan 2012 08:08:13 +0000

Al

I understand that it doesn't fit IPPM, being a passive measurement technique.

However, I suggested IPPM to discuss it in order to understand the relationship
between what is measures ("packet loss" "delay" "delay variation")
and the careful definitions of the same terms that have been made by IPPM.

I happen to like the basic idea of the technique,
but am a bit worried about the IPR encumberments.

And yes, I agree with your suggested modus operandi.

Y(J)S

-----Original Message-----
From: Al Morton [mailto:[email-masked]] 
Sent: Monday, January 16, 2012 21:06
To: Yaakov Stein; [email-masked]
Subject: draft-tempia-opsawg-p3m-01 [was: [PMOL] Thoughts on how we operate...]

Hi Yaakov,

The Title:

---

From: Al Morton <[email-masked]>
Date: Tue, 17 Jan 2012 08:54:10 -0500

At 03:08 AM 1/17/2012, Yaakov Stein
wrote:

...I understand that it doesn't fit IPPM, being a passive measurement
technique.

However, I suggested IPPM to discuss it in order to understand the
relationship

between what is measures ("packet loss" "delay"
"delay variation")

and the careful definitions of the same terms that have been made by
IPPM.

I agree with that, in fact there are several passive methods in
development

that I know about. There's a noticeable amount of metric
standardization

projects around, and we (the Directorate) could help the IETF community
by

suggesting ways to get it organized.

Sadly yes, we mostly avoided this for a long time in IETF
measurements.

Many of us could have imposed IPR but didn't, preferring that
everyone

measure the same way without worrying about rights, too.

still waiting for word from opsawg,

Al

---

From: "Jan Novak (janovak)" <[email-masked]>
Date: Fri, 20 Jan 2012 17:54:13 +0100

Al,

There is not much to add to your "mode of operation"
Just how to capture the incoming drafts - being on
ietf-announce I see the file names but unless there is
already performance there, I wouldn't notice ...

Jan

The climate of Edinburgh is such that the weak succumb young .... 
and the strong envy them.
                                 Dr. Johnson

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of
Al Morton
Sent: 16 January 2012 14:47
To: [email-masked]
Subject: [PMOL] Thoughts on how we operate as a Directorate

As our web page says: "RFC 6390 is used as a reference for the technical
and process issues." Have your copy handy when reviewing a draft.
A lot of interactions may be streamlined by pointing the authors to
6390,
and mentioning that it is a BCP.

I feel that our reviews should be *early* in the draft development
process, and that no performance-related draft should start IETF Last
Call
without previous review against 6390, unless it truly slipped by us
and other folks who know about the Directorate.

- Directorate Members will see drafts, hear about drafts with
performance
work, etc.  We are the eyes and ears of the Directorate. We all have
friends
working in other areas of IETF, let them know what we're looking for.

2. WG Candidate/Chartered Item - When a draft appears to have enough
value
and interest to become a WG item, or the WG Chairs involved recommend a
Performance Metrics Directorate review on their own, then we do a more
complete review against 6390.

in WG or IETF Last Call, assuming it goes that far. I suggest 2
reviewers

---

From: "Romascanu, Dan (Dan)" <[email-masked]>
Date: Mon, 23 Jan 2012 16:49:26 +0100

Probably going beyond the name of the document is desirable - looking
into the content for strings like 'performance metrics', 'performance
monitoring', etc. could reveal things that are missed by just scanning
the titles. 

Dan

> Jan Novak (janovak)
> Sent: Friday, January 20, 2012 6:54 PM
> To: Al Morton; [email-masked]
> Subject: Re: [PMOL] Thoughts on how we operate as a Directorate
> 
> Al,
>

---

From: Al Morton <[email-masked]>
Date: Mon, 23 Jan 2012 12:16:00 -0500

At 08:54 AM 1/17/2012, Al Morton wrote:
>...still waiting for word from opsawg,
>Al

I just heard from Melinda Shore, the draft has not been active
recently (01 update was not mentioned on the list).

If the authors re-appear (because we're meeting in Europe next time?)
they'll let us know.

Al