From icar-bounces@ietf.org Sun Aug 01 11:30:02 2004 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1BrI4j-0003BF-GE for icar-web-archive@megatron.ietf.org; Sun, 01 Aug 2004 11:16:37 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA05787 for ; Sun, 1 Aug 2004 11:16:35 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1BrI6q-0004Ve-PK for icar-web-archive@ietf.org; Sun, 01 Aug 2004 11:19:13 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1BrHqy-0004NY-KB; Sun, 01 Aug 2004 11:02:24 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1BrHea-0006o8-PU for icar@megatron.ietf.org; Sun, 01 Aug 2004 10:49:36 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id KAA04411 for ; Sun, 1 Aug 2004 10:49:34 -0400 (EDT) Received: from netcore.fi ([193.94.160.1]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1BrHhF-000493-4M for icar@ietf.org; Sun, 01 Aug 2004 10:52:22 -0400 Received: from localhost (pekkas@localhost) by netcore.fi (8.11.6/8.11.6) with ESMTP id i71En2a31566 for ; Sun, 1 Aug 2004 17:49:02 +0300 Date: Sun, 1 Aug 2004 17:49:02 +0300 (EEST) From: Pekka Savola To: icar@ietf.org Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-Spam-Score: 0.0 (/) X-Scan-Signature: 7aafa0432175920a4b3e118e16c5cb64 Subject: [Icar] icar-experiment-early-review-00 comments X-BeenThere: icar@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Improved Cross-Area Review List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: icar-bounces@ietf.org Errors-To: icar-bounces@ietf.org X-Spam-Score: 0.0 (/) X-Scan-Signature: 0a7aa2e6e558383d84476dc338324fab Status: O a couple of short comments on the icar consensus doc.. semi-substantial ---------------- 5.1 Who requests a review ==> this doesn't specify what happens to the review requests which do not come from WG chairs/secretaries or ADs. In particular, if a doc author wants to get review of his shiny -00 submission nobody else has bothered to comment..... editorial --------- Quite apart from the ICAR process, any member of the IETF community may, of course, review a WG document at any time s/he wishes and provide that input to the WG. ==> this should maybe be generalized a bit for the case where it isn't a WG document (but individual submission to an AD). This cornercase seems to be missing from a couple of places in the draft.. o people who have authored 2 IETF RFCs approved for publication by the IESG, which includes non-WG documents submitted directly to the IESG, but not documents submitted directly to the RFC editor ==> This should probably be clearer worded like: o people who have authored at least 2 IETF RFCs approved for publication by the IESG (which also includes non-WG documents submitted directly to the IESG, but not documents submitted directly to the RFC editor) o current WG chairs whose WG has produced 2 RFCs ==> .. at least 2 ... ;-) -- not sure if this is pedantically required though.. It is up to the WG chair(s) / secretary to choose the reviewers who review the document. In order to do this, reviewers in the pool must be clearly labeled with their area(s) of expertise they feel qualified to provide . ==> ... "input on" -- Pekka Savola "You each name yourselves king, yet the Netcore Oy kingdom bleeds." Systems. Networks. Security. -- George R.R. Martin: A Clash of Kings _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From icar-bounces@ietf.org Wed Aug 04 20:29:10 2004 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1BsW6K-0005rq-79 for icar-web-archive@megatron.ietf.org; Wed, 04 Aug 2004 20:27:20 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id UAA23769 for ; Wed, 4 Aug 2004 20:27:18 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1BsW9h-0003o5-Se for icar-web-archive@ietf.org; Wed, 04 Aug 2004 20:30:50 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1BsW1h-0004vu-5f; Wed, 04 Aug 2004 20:22:33 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1BsVzO-0004KH-Ky for icar@megatron.ietf.org; Wed, 04 Aug 2004 20:20:10 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id UAA23408 for ; Wed, 4 Aug 2004 20:20:09 -0400 (EDT) Received: from rwcrmhc12.comcast.net ([216.148.227.85]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1BsW2i-0003ex-Kg for icar@ietf.org; Wed, 04 Aug 2004 20:23:40 -0400 Received: from dfnjgl21 (opene-130-129-130-45.ietf60.ietf.org[130.129.130.45]) by comcast.net (rwcrmhc12) with SMTP id <2004080500193001400ri44ke> (Authid: sdawkins@comcast.net); Thu, 5 Aug 2004 00:19:30 +0000 Message-ID: <003101c47a81$de7361d0$2d828182@DFNJGL21> From: "Spencer Dawkins" To: "ICAR Mailing List" Date: Wed, 4 Aug 2004 17:19:26 -0700 MIME-Version: 1.0 X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2800.1437 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1441 X-Spam-Score: 0.5 (/) X-Scan-Signature: 75ffc14afb41eeead069d201c6c1d81f Subject: [Icar] Notes from ICAR X-BeenThere: icar@ietf.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Spencer Dawkins List-Id: Improved Cross-Area Review List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============0205075687==" Sender: icar-bounces@ietf.org Errors-To: icar-bounces@ietf.org X-Spam-Score: 0.5 (/) X-Scan-Signature: d7802e9da0b1d11bf32ba03c41071b61 Status: O This is a multi-part message in MIME format. --===============0205075687== Content-Type: multipart/alternative; boundary="----=_NextPart_000_002E_01C47A47.319EE4A0" This is a multi-part message in MIME format. ------=_NextPart_000_002E_01C47A47.319EE4A0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable These are my notes from our session today (and I apologize in advance = for HTML), Spencr Improved Cross-Area Review WG (icar) =20 Wednesday, August 4 at 1530-1730 =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D =20 CHAIRS: Mark Allman Joel Halpern =20 AGENDA: =20 1. intro/agenda bashing - co-chairs 2. discussion of draft-ietf-icar-experiment-early-review-00.txt - David Partain =20 Draft to capture the state of WG discussions - supposed to capture WG = mindset =20 Open issues: =20 - guidelines, if any, for the form of the review? =20 >From TSV - ask for important/crucial issues at the top of the review, as = a summary of the review =20 >From ITU - major technical, minor technical, editorial categories =20 This isn't about binding, it's about demonstrating to ADs that "major = technical" items have been considered =20 Working groups have to consider comments, can't just ignore them =20 What is "must respond"? Working group is free to say "we're right", not = free to say "we've never heard this comment before" =20 - information about potential reviewers? How many requests, how many = reviews ... =20 Number of reviews is also a recognition item for reviewers =20 Average time to complete a review per page of document =20 Need to be collecting raw information as we go =20 Need to avoid "top 10" lists of reviewers, etc. =20 Don't set the threshold to become a reviewer too high - need to be = bringing in new reviewers (see later discussion) =20 Reviewer specializations? Mark will show what we have so far =20 What kind of review are we asking for? Very specific topics, or more = general? =20 Need to accommodate specialist reviewers and generalist reviewers =20 A little history is a good thing... =20 - Subjective criteria for admittance to the reviewer pool? =20 Start by doing real work before you're admitted? AD can just decide?=20 =20 Requests are published publicly, so one could establish a track record =20 Sponsorship by an AD? Definition of "good" for an AD is "what I would = have figured out anyway"? =20 Volunteering and accepting/rejecting should be in secret =20 If "n" (reviews) is sufficiently large, that flushes out a lot of the = idiots =20 "Sorry, but your reviews don't seem useful" - as a teaching tool - but = need to keep from turning ADs into fulltime mentors - are reviewers = expected to be willing to serve as mentors? Could this be this an area = directorate responsibility (defined broadly)? =20 EDU team should (obviously) be willing to help here =20 Could have a separate pool of mentors, set up independently =20 Are we doing early or late reviews? ICAR is focused on early review - = the hope is that this helps with late reviews as well =20 WG needs some assurance that they are likely to get a good review from = anyone in the pool =20 Nothing but early review can reduce late surprises - this needs to be = part of the success criteria for ICAR =20 Some set of reviews plus AD sponsorship?=20 =20 ADs can sponsor anyone, but recommend looking at previous reviews? We = trust ADs to do a lot more than make choices like this without adult = supervision =20 (Objective - it's "two or more" RFCs, not "two") =20 - How do we measure success/failure of ICAR experiments? =20 Some types of failure are easy to detect, but success is harder - look = for absence of failures and keep going? =20 Objective criteria for failure, subjective criteria for success =20 Helped or hindered in RFC production? Experiment will be over long = before we get an RFC out through this mechanism =20 Ask the chairs, not the document editors, for success, but ask the = document editors for feedback as well =20 - How are people removed from the pool, or is this process needed at = all? =20 At least remove yourself! =20 Let bad reviewers rot on the vine? =20 Need a minimal "periodic survey" for self-removal? People should = "time-out" after six months, or something =20 Removal on the basis of quality - one person's crank is another person's = helpful reviewer =20 If you qualify and time out, you still qualify when you ask to be added = back =20 ADs need a removal-for-cause mechanism - we do remove WG chairs from = time to time =20 If reviewers don't provide reviews - performance criteria for remaining = in the pool? =20 This is a voluntary process anyway - why do we need to throw people out = of the pool? What's the worst case? Is it as bad as what a WG chair can = do? We can throw them out now... =20 Are we talking one percent bad reviewers or ten percent bad reviewers? = We don't need to create an expulsion mechanism for the experiments =20 We pretty much agree we need a timeout =20 What about a reviewer with an ax to grind? Is "serve at the pleasure" = language good enough? Is "the grapevine" good enough? Bad reviewers are = worse for new working groups with new chairs.=20 =20 Why is this worse than a bozo appearing at IETF last call? The pool is = supposed to have more clue, and the working group is supposed to respond =20 What about reviewers that don't review? Just timing out is good enough =20 Underburden bad reviewers, ignore them, or overburden them? We already = have three mechanisms =20 There is a difference between a process that's seldom used and one that = doesn't exist =20 We don't have consensus here - take this question to the list =20 How does AD removal map to pre-qualified participants? To the list... =20 - What can we hope for from the IESG? How do reviewers, prospective = reviewers, and working groups interact with IESG? =20 IESG has promised Harald they will provide sacrificial working groups = for the experiment, and are concerned about providing names of people = who don't have time to review documents for ADs now =20 "Perhaps your working group should get an ICAR review" =20 We're asking "help us find working groups, help us find reviewers, = sponsor reviewers" - IESG is mumbling on the middle answer =20 - Other comments? =20 Non-performance will be a problem - need backup reviewers - yes, and WGs = can handle this however they want to - we could learn this in an = experiment, instead of noodling about it =20 We are depending on volunteers - dates and plans to follow =20 3. the ICAR web page and soliciting for volunteers - Mark Allman =20 Web page available from ICAR charter page =20 Describes the proposal for an experiment =20 ADs can request reviews for non-WG documents, too =20 Pool members send reviews to icar mailing address =20 WG considers comments and suggestions included in the reviews =20 Reviewer attributes - expertise, reviews, ideal queue depth, current = queue depth, plus arbitrary bio =20 Reviews and responses available from reviewer page =20 Could be sliced by WG, by draft, etc. =20 Logging time when information about requests and reviews are entered = into the system =20 Feedback? Codify expertise? At least sort by expertise? Grade expertise? = What about subjective grade inflation? What about "wireless", "XML", = etc. =20 4. experience with the general area review team - Harald Alvestrand =20 Gen-ART is late review - one week before IESG processing - and general = review, including readability =20 IETF chair unable to get responsible and timely reviews, and other = people had directorates and review teams - so asked for help =20 Called an experiment, getting experience with fast reviews, identify = what turns out to be problems, and actually publish reviews =20 One-area pilot, so Harald didn't have to ask for agreement, and Harald = already had a web server and mail server =20 Review guidelines stolen from SIR, driven by IESG mechanics =20 Reviewers assigned round-robin, reviews and names are public, started = January 22 =20 Reviewers recruited word-of-mouth, "known to the AD", but also asked for = volunteers on ICAR, no reviewers have crumbled yet =20 One-person, one-document? Don't double up, because we have 20 documents = per telechat and less than 40 reviewers =20 10 reviewers, 135 reviews, lots of concerns, 5 DISCUSSes (at least) =20 How did this compare to SIR experiment? This is review at a different = level =20 Load on AD: down. Community involvement: Up =20 Lessons learned - one week is a short time, reviewers need a discussion = forum, ten people isn't enough, it needs to be a team, procedures are = necessary, IT support is optional but nice. =20 Trying to move standards-track review to IETF Last Call time =20 Would a single discussion forum scale to 200 reviewers? =20 Unresolved issues: Follow-up is important. Procedures for pointing at = information is important. Getting the WG's attention for non-critical = issues isn't a solved problem. =20 Not sure who to send reviews to (ADs, WG chairs, editors, or some = combination). =20 Reviews need to be more visible to the community (you can find them now, = if you're psychic). =20 5. General Discussion =20 What sort of expectations for reviewer responsiveness on committing to a = review? =20 Hard deadlines help us focus... =20 Short timeframes are good, when we're asking for a commitment - Dave = will suggest a number on the mailing list ------=_NextPart_000_002E_01C47A47.319EE4A0 Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable

These are my notes = from our=20 session today (and I apologize in advance for HTML),

 

Spencr

 

Improved = Cross-Area Review WG=20 (icar)

 

Wednesday, August = 4 at=20 1530-1730

=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D

 

CHAIRS: Mark = Allman=20 <mallman@icir.org>

       =20 Joel Halpern=20 <joel@stevecrocker.com>

 

AGENDA:

 

1.  intro/agenda bashing -=20 co-chairs

2.  discussion of=20 draft-ietf-icar-experiment-early-review-00.txt -

    David Partain

 

Draft to capture = the state of=20 WG discussions =96 supposed to capture WG mindset

 

Open issues:

 

- guidelines, if = any, for the=20 form of the review?

 

From TSV =96 ask = for=20 important/crucial issues at the top of the review, as a summary of the=20 review

 

From ITU =96 major = technical,=20 minor technical, editorial categories

 

This isn=92t about = binding, it=92s=20 about demonstrating to ADs that =93major technical=94 items have been = considered

 

Working groups = have to=20 consider comments, can=92t just ignore them

 

What is =93must = respond=94?=20 Working group is free to say =93we=92re right=94, not free to say = =93we=92ve never heard=20 this comment before=94

 

- information = about potential=20 reviewers? How many requests, how many reviews ...

 

Number of reviews = is also a=20 recognition item for reviewers

 

Average time to = complete a=20 review per page of document

 

Need to be = collecting raw=20 information as we go

 

Need to avoid = =93top 10=94 lists=20 of reviewers, etc.

 

Don=92t set the = threshold to=20 become a reviewer too high =96 need to be bringing in new reviewers (see = later=20 discussion)

 

Reviewer = specializations? Mark=20 will show what we have so far

 

What kind of = review are we=20 asking for? Very specific topics, or more general?

 

Need to = accommodate specialist=20 reviewers and generalist reviewers

 

A little history = is a good=20 thing...

 

- Subjective = criteria for=20 admittance to the reviewer pool?

 

Start by doing = real work=20 before you=92re admitted? AD can just decide?

 

Requests are = published=20 publicly, so one could establish a track record

 

Sponsorship by an = AD?=20 Definition of =93good=94 for an AD is =93what I would have figured out = anyway=94?

 

Volunteering and=20 accepting/rejecting should be in secret

 

If =93n=94 = (reviews) is=20 sufficiently large, that flushes out a lot of the idiots

 

=93Sorry, but your = reviews don=92t=20 seem useful=94 =96 as a teaching tool =96 but need to keep from turning = ADs into=20 fulltime mentors =96 are reviewers expected to be willing to serve as = mentors?=20 Could this be this an area directorate responsibility (defined = broadly)?

 

EDU team should = (obviously) be=20 willing to help here

 

Could have a = separate pool of=20 mentors, set up independently

 

Are we doing early = or late=20 reviews? ICAR is focused on early review =96 the hope is that this helps = with late=20 reviews as well

 

WG needs some = assurance that=20 they are likely to get a good review from anyone in the pool

 

Nothing but early = review can=20 reduce late surprises =96 this needs to be part of the success criteria = for=20 ICAR

 

Some set of = reviews plus AD=20 sponsorship?

 

ADs can sponsor = anyone, but=20 recommend looking at previous reviews? We trust ADs to do a lot more = than make=20 choices like this without adult supervision

 

(Objective =96 = it=92s =93two or=20 more=94 RFCs, not =93two=94)

 

- How do we = measure=20 success/failure of ICAR experiments?

 

Some types of = failure are easy=20 to detect, but success is harder =96 look for absence of failures and = keep=20 going?

 

Objective criteria = for=20 failure, subjective criteria for success

 

Helped or hindered = in RFC=20 production? Experiment will be over long before we get an RFC out = through this=20 mechanism

 

Ask the chairs, = not the=20 document editors, for success, but ask the document editors for feedback = as=20 well

 

- How are people = removed from=20 the pool, or is this process needed at all?

 

At least remove = yourself!

 

Let bad reviewers = rot on the=20 vine?

 

Need a minimal = =93periodic=20 survey=94 for self-removal? People should =93time-out=94 after six = months, or=20 something

 

Removal on the = basis of=20 quality =96 one person=92s crank is another person=92s helpful = reviewer

 

If you qualify and = time out,=20 you still qualify when you ask to be added back

 

ADs need a = removal-for-cause=20 mechanism =96 we do remove WG chairs from time to time

 

If reviewers = don=92t provide=20 reviews =96 performance criteria for remaining in the pool?

 

This is a = voluntary process=20 anyway =96 why do we need to throw people out of the pool? What=92s the = worst case?=20 Is it as bad as what a WG chair can do? We can throw them out now...

 

Are we talking one = percent bad=20 reviewers or ten percent bad reviewers? We don=92t need to create an = expulsion=20 mechanism for the experiments

 

We pretty much = agree we need a=20 timeout

 

What about a = reviewer with an=20 ax to grind? Is =93serve at the pleasure=94 language good enough? Is = =93the grapevine=94=20 good enough? Bad reviewers are worse for new working groups with new = chairs.=20

 

Why is this worse = than a bozo=20 appearing at IETF last call? The pool is supposed to have more clue, and = the=20 working group is supposed to respond

 

What about = reviewers that=20 don=92t review? Just timing out is good enough

 

Underburden bad = reviewers,=20 ignore them, or overburden them? We already have three mechanisms

 

There is a = difference between=20 a process that=92s seldom used and one that doesn=92t exist

 

We don=92t have = consensus here =96=20 take this question to the list

 

How does AD = removal map to=20 pre-qualified participants? To the list...

 

- What can we hope = for from=20 the IESG? How do reviewers, prospective reviewers, and working groups = interact=20 with IESG?

 

IESG has promised = Harald they=20 will provide sacrificial working groups for the experiment, and are = concerned=20 about providing names of people who don=92t have time to review = documents for ADs=20 now

 

=93Perhaps your = working group=20 should get an ICAR review=94

 

We=92re asking = =93help us find=20 working groups, help us find reviewers, sponsor reviewers=94 =96 IESG is = mumbling on=20 the middle answer

 

- Other = comments?

 

Non-performance = will be a=20 problem =96 need backup reviewers =96 yes, and WGs can handle this = however they want=20 to =96 we could learn this in an experiment, instead of noodling about = it

 

We are depending = on volunteers=20 =96 dates and plans to follow

 

3.  the ICAR web page and = soliciting for=20 volunteers - Mark Allman

 

Web page available = from ICAR=20 charter page

 

Describes the = proposal for an=20 experiment

 

ADs can request = reviews for=20 non-WG documents, too

 

Pool members send = reviews to=20 icar mailing address

 

WG considers = comments and=20 suggestions included in the reviews

 

Reviewer = attributes =96=20 expertise, reviews, ideal queue depth, current queue depth, plus = arbitrary=20 bio

 

Reviews and = responses=20 available from reviewer page

 

Could be sliced by = WG, by=20 draft, etc.

 

Logging time when = information=20 about requests and reviews are entered into the system

 

Feedback? Codify = expertise? At=20 least sort by expertise? Grade expertise? What about subjective grade = inflation?=20 What about =93wireless=94, =93XML=94, etc.

 

4.  experience with the general = area review=20 team - Harald Alvestrand

 

Gen-ART is late = review =96 one=20 week before IESG processing =96 and general review, including = readability

 

IETF chair unable = to get=20 responsible and timely reviews, and other people had directorates and = review=20 teams =96 so asked for help

 

Called an = experiment, getting=20 experience with fast reviews, identify what turns out to be problems, = and=20 actually publish reviews

 

One-area pilot, so = Harald=20 didn=92t have to ask for agreement, and Harald already had a web server = and mail=20 server

 

Review guidelines = stolen from=20 SIR, driven by IESG mechanics

 

Reviewers assigned = round-robin, reviews and names are public, started January 22

 

Reviewers = recruited=20 word-of-mouth, =93known to the AD=94, but also asked for volunteers on = ICAR, no=20 reviewers have crumbled yet

 

One-person, = one-document?=20 Don=92t double up, because we have 20 documents per telechat and less = than 40=20 reviewers

 

10 reviewers, 135 = reviews,=20 lots of concerns, 5 DISCUSSes (at least)

 

How did this = compare to SIR=20 experiment? This is review at a different level

 

Load on AD: down. = Community=20 involvement: Up

 

Lessons learned = =96 one week is=20 a short time, reviewers need a discussion forum, ten people isn=92t = enough, it=20 needs to be a team, procedures are necessary, IT support is optional but = nice.

 

Trying to move = standards-track=20 review to IETF Last Call time

 

Would a single = discussion=20 forum scale to 200 reviewers?

 

Unresolved issues: = Follow-up=20 is important. Procedures for pointing at information is important. = Getting the=20 WG=92s attention for non-critical issues isn=92t a solved problem.

 

Not sure who to = send reviews=20 to (ADs, WG chairs, editors, or some combination).

 

Reviews need to be = more=20 visible to the community (you can find them now, if you=92re = psychic).

 

5.   General Discussion

 

What sort of = expectations for=20 reviewer responsiveness on committing to a review?

 

Hard deadlines = help us=20 focus...

 

Short timeframes = are good,=20 when we=92re asking for a commitment =96 Dave will suggest a number on = the mailing=20 list

------=_NextPart_000_002E_01C47A47.319EE4A0-- --===============0205075687== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline Content-Transfer-Encoding: 7bit _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar --===============0205075687==-- From icar-bounces@ietf.org Wed Aug 11 12:37:42 2004 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1Buvvn-0006lY-Uw for icar-web-archive@megatron.ietf.org; Wed, 11 Aug 2004 12:26:28 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA16812 for ; Wed, 11 Aug 2004 12:26:25 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1Buw0a-0002Ur-Bw for icar-web-archive@ietf.org; Wed, 11 Aug 2004 12:31:24 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1BuvrS-0005p5-DE; Wed, 11 Aug 2004 12:21:58 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1BuvWO-0000Ef-IZ for icar@megatron.ietf.org; Wed, 11 Aug 2004 12:00:12 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA14804 for ; Wed, 11 Aug 2004 12:00:09 -0400 (EDT) Received: from wyvern.icir.org ([192.150.187.14]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1Buvb9-0001wM-FF for icar@ietf.org; Wed, 11 Aug 2004 12:05:08 -0400 Received: from guns.icir.org (adsl-68-76-113-50.dsl.bcvloh.ameritech.net [68.76.113.50]) by wyvern.icir.org (8.12.9p1/8.12.8) with ESMTP id i7BG06Ne090901; Wed, 11 Aug 2004 09:00:06 -0700 (PDT) (envelope-from mallman@guns.icir.org) Received: from guns.icir.org (localhost [127.0.0.1]) by guns.icir.org (Postfix) with ESMTP id 5A4E977AED4; Wed, 11 Aug 2004 12:00:02 -0400 (EDT) To: Pekka Savola From: Mark Allman Subject: Re: [Icar] icar-experiment-early-review-00 comments In-Reply-To: Organization: ICSI Center for Internet Research (ICIR) Song-of-the-Day: Born to Run MIME-Version: 1.0 Date: Wed, 11 Aug 2004 12:00:02 -0400 Message-Id: <20040811160002.5A4E977AED4@guns.icir.org> X-Spam-Score: 0.0 (/) X-Scan-Signature: f4c2cf0bccc868e4cc88dace71fb3f44 Cc: icar@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: mallman@icir.org List-Id: Improved Cross-Area Review List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============1397428808==" Sender: icar-bounces@ietf.org Errors-To: icar-bounces@ietf.org X-Spam-Score: 0.0 (/) X-Scan-Signature: 0fa76816851382eb71b0a882ccdc29ac Status: O --===============1397428808== Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha1; protocol="application/pgp-signature" --=-=-= Content-Type: text/plain; charset=us-ascii (WG chair hat off) > 5.1 Who requests a review > > ==> this doesn't specify what happens to the review requests which do > not come from WG chairs/secretaries or ADs. In particular, if a doc > author wants to get review of his shiny -00 submission nobody else has > bothered to comment..... If the shiny -00 is a WG document or a WG is thinking about it then a WG chair / secratary can handle it. If it is to be submitted to IESG independent of a WG then an AD can handle it. Then there are the others. Two thoughts... * It could just get to be a mess if an author is requesting reviews and a WG chair is also requesting reviews because they each think they should. (E.g., on an independent submission that is within the sphere of a WG, but not technically a WG item.) In principle, this could be accounted for if the automated system was savvy enough, I suppose. * But, really... IMO, nobody has convinced me that I need to care about these documents. If an author can't get enough buy-in from a WG (/AD) to get that WG to request review as part of the "should we take this on as a WG item" process then why are we bothering to expend reviewer cycles? I just don't follow this. (WG chair hat dusted off and put back on) It'd be nice to hear more comments on this issue. allman -- Mark Allman -- ICIR -- http://www.icir.org/mallman/ --=-=-= Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.3 (FreeBSD) iD8DBQFBGkKAWyrrWs4yIs4RAvVjAKCIai4r7TI2hqDNutvxJHlHVewTIwCdFZiG 4vQnalOVGtKlal459kN3MKY= =RKMi -----END PGP SIGNATURE----- --=-=-=-- --===============1397428808== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline Content-Transfer-Encoding: 7bit _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar --===============1397428808==-- From icar-bounces@ietf.org Wed Aug 11 13:33:33 2004 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1BuwoE-000186-CS for icar-web-archive@megatron.ietf.org; Wed, 11 Aug 2004 13:22:42 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA20459 for ; Wed, 11 Aug 2004 13:22:41 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1Buwsy-0003UG-Hw for icar-web-archive@ietf.org; Wed, 11 Aug 2004 13:27:39 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1Buwnp-0000tl-5E; Wed, 11 Aug 2004 13:22:17 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1BuweP-00080P-Uu for icar@megatron.ietf.org; Wed, 11 Aug 2004 13:12:33 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA19700 for ; Wed, 11 Aug 2004 13:12:31 -0400 (EDT) Received: from sj-iport-2-in.cisco.com ([171.71.176.71] helo=sj-iport-2.cisco.com) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1BuwjC-0003Hj-0O for icar@ietf.org; Wed, 11 Aug 2004 13:17:31 -0400 Received: from sj-core-1.cisco.com (171.71.177.237) by sj-iport-2.cisco.com with ESMTP; 11 Aug 2004 10:14:21 -0700 Received: from mira-sjc5-c.cisco.com (IDENT:mirapoint@mira-sjc5-c.cisco.com [171.71.163.17]) by sj-core-1.cisco.com (8.12.10/8.12.6) with ESMTP id i7BHBxVD025202; Wed, 11 Aug 2004 10:11:59 -0700 (PDT) Received: from cisco.com ([10.25.65.179]) by mira-sjc5-c.cisco.com (MOS 3.4.5-GR) with SMTP id AXY01000; Wed, 11 Aug 2004 10:11:58 -0700 (PDT) Date: Wed, 11 Aug 2004 13:11:36 -0400 Subject: Re: [Icar] icar-experiment-early-review-00 comments Content-Type: text/plain; charset=US-ASCII; format=flowed Mime-Version: 1.0 (Apple Message framework v553) To: mallman@icir.org From: Melinda Shore In-Reply-To: <20040811160002.5A4E977AED4@guns.icir.org> Message-Id: <80C51526-EBB9-11D8-923B-000A95E35274@cisco.com> Content-Transfer-Encoding: 7bit X-Mailer: Apple Mail (2.553) X-Spam-Score: 0.0 (/) X-Scan-Signature: 7655788c23eb79e336f5f8ba8bce7906 Content-Transfer-Encoding: 7bit Cc: icar@ietf.org, Pekka Savola X-BeenThere: icar@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Improved Cross-Area Review List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: icar-bounces@ietf.org Errors-To: icar-bounces@ietf.org X-Spam-Score: 0.0 (/) X-Scan-Signature: 0bc60ec82efc80c84b8d02f4b0e4de22 Content-Transfer-Encoding: 7bit Status: O On Wednesday, August 11, 2004, at 12:00 PM, Mark Allman wrote: > If the shiny -00 is a WG document or a WG is thinking about it then a > WG > chair / secratary can handle it. For early review I think I'd prefer that the wg chair or secretary handle it, as well. If the AD thinks there's an issue, he or she can let the wg chair know that there's a need for a review. > If it is to be submitted to IESG independent of a WG then an AD can > handle it. I agree. What I really don't want to see is the reviewer pool, which I think needs to be regarded as a scarce and precious resource, being used to review drafts at the request of an author. That can be handled through the usual mechanisms (posting a "hey, look at this" message to the appropriate mailing list, contacting interested people out-of- band, etc.). Melinda _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From icar-bounces@ietf.org Wed Aug 11 14:06:31 2004 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1BuxP4-0007Hc-7L for icar-web-archive@megatron.ietf.org; Wed, 11 Aug 2004 14:00:46 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA22869 for ; Wed, 11 Aug 2004 14:00:45 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1BuxTq-00048x-SS for icar-web-archive@ietf.org; Wed, 11 Aug 2004 14:05:44 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1BuxL2-0006q2-J1; Wed, 11 Aug 2004 13:56:36 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1BuxFB-00058w-QO for icar@megatron.ietf.org; Wed, 11 Aug 2004 13:50:33 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA22241 for ; Wed, 11 Aug 2004 13:50:32 -0400 (EDT) Received: from netcore.fi ([193.94.160.1]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1BuxJx-0003xh-Tr for icar@ietf.org; Wed, 11 Aug 2004 13:55:31 -0400 Received: from localhost (pekkas@localhost) by netcore.fi (8.11.6/8.11.6) with ESMTP id i7BHnOt31082; Wed, 11 Aug 2004 20:49:24 +0300 Date: Wed, 11 Aug 2004 20:49:24 +0300 (EEST) From: Pekka Savola To: Melinda Shore Subject: Re: [Icar] icar-experiment-early-review-00 comments In-Reply-To: <80C51526-EBB9-11D8-923B-000A95E35274@cisco.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-Spam-Score: 0.0 (/) X-Scan-Signature: de4f315c9369b71d7dd5909b42224370 Cc: icar@ietf.org, mallman@icir.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Improved Cross-Area Review List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: icar-bounces@ietf.org Errors-To: icar-bounces@ietf.org X-Spam-Score: 0.0 (/) X-Scan-Signature: 798b2e660f1819ae38035ac1d8d5e3ab Status: O On Wed, 11 Aug 2004, Melinda Shore wrote: > I agree. What I really don't want to see is the reviewer pool, > which I think needs to be regarded as a scarce and precious > resource, being used to review drafts at the request of an author. > That can be handled through the usual mechanisms (posting a "hey, > look at this" message to the appropriate mailing list, contacting > interested people out-of- band, etc.). This was precisely my point -- I think the doc or the process needs to have a good understanding how it will handle the authors who request reviews of their half-baked ideas (when they shouldn't). -- Pekka Savola "You each name yourselves king, yet the Netcore Oy kingdom bleeds." Systems. Networks. Security. -- George R.R. Martin: A Clash of Kings _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From icar-bounces@ietf.org Wed Aug 11 22:54:14 2004 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1Bv5fh-0007P3-B0 for icar-web-archive@megatron.ietf.org; Wed, 11 Aug 2004 22:50:29 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id WAA10313 for ; Wed, 11 Aug 2004 22:50:27 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1Bv5kZ-00019Q-Oa for icar-web-archive@ietf.org; Wed, 11 Aug 2004 22:55:31 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1Bv5fM-0007Cm-JV; Wed, 11 Aug 2004 22:50:08 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1Bv5Zh-000675-GY for icar@megatron.ietf.org; Wed, 11 Aug 2004 22:44:17 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id WAA09972 for ; Wed, 11 Aug 2004 22:44:15 -0400 (EDT) Received: from joy.songbird.com ([208.184.79.7]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1Bv5eZ-00012B-Ef for icar@ietf.org; Wed, 11 Aug 2004 22:49:20 -0400 Received: from bbprime (jay.songbird.com [208.184.79.253]) by joy.songbird.com (8.11.6/8.11.6) with ESMTP id i7C2hjj04627 for ; Wed, 11 Aug 2004 19:43:45 -0700 Date: Wed, 11 Aug 2004 19:43:38 -0700 From: Dave Crocker Organization: Brandenburg InternetWorking X-Priority: 3 (Normal) Message-ID: <516932251.20040811194338@brandenburg.com> To: icar@ietf.org Subject: Re: [Icar] icar-experiment-early-review-00 comments In-Reply-To: References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Spam-Score: 0.8 (/) X-Scan-Signature: 769a46790fb42fbb0b0cc700c82f7081 Content-Transfer-Encoding: 7bit X-BeenThere: icar@ietf.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Dave Crocker List-Id: Improved Cross-Area Review List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: icar-bounces@ietf.org Errors-To: icar-bounces@ietf.org X-Spam-Score: 0.8 (/) X-Scan-Signature: 538aad3a3c4f01d8b6a6477ca4248793 Content-Transfer-Encoding: 7bit Status: O Folks, PS> 5.1 Who requests a review This is the sort of section that nicely demonstrates some core problems in IETF process. The problem that this sub-section is trying to solve is real. If there is no coordination in making requests, then things really are a mess. The question is whether we have made things better by micro-managing working group internal operations in this document. Not surprisingly, I think we have made them worse. If the document says something like: "it is the the working group's responsibility to obtain necessary reviews", then the document has said all it needs to. It has made clear the responsibility and has left the details to the folks who have to do the work. Think of all the other micro-events that we do not -- and should not -- dictate in the life of a working group. One of the benefits of a well-run IETF working group is its freedom to organize in a way that works for that particular topic and that particular set of participants. Let's not take that away. (A nice side-effect of such simplification of the icar spec is that the icar working group gets done quicker.) PS> This was precisely my point -- I think the doc or the process needs to PS> have a good understanding how it will handle the authors who request PS> reviews of their half-baked ideas (when they shouldn't). along the lines of Melinda's comment: the icar mechanism really is tailored for formal working group efforts, and so requests come formally from the working group. that should provide enough of a filter, no? d/ -- Dave Crocker Brandenburg InternetWorking Sunnyvale, CA USA , _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From icar-bounces@ietf.org Thu Aug 12 04:40:20 2004 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1BvB7U-0006sU-5f for icar-web-archive@megatron.ietf.org; Thu, 12 Aug 2004 04:39:32 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id EAA13190 for ; Thu, 12 Aug 2004 04:39:30 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1BvBCP-00076a-MG for icar-web-archive@ietf.org; Thu, 12 Aug 2004 04:44:37 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1BvB47-0006Qf-CB; Thu, 12 Aug 2004 04:36:03 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1BvB2q-0006Io-JL for icar@megatron.ietf.org; Thu, 12 Aug 2004 04:34:44 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id EAA12975 for ; Thu, 12 Aug 2004 04:34:42 -0400 (EDT) Received: from eikenes.alvestrand.no ([158.38.152.233]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1BvB7k-000728-RF for icar@ietf.org; Thu, 12 Aug 2004 04:39:50 -0400 Received: from localhost (localhost.localdomain [127.0.0.1]) by eikenes.alvestrand.no (Postfix) with ESMTP id A126861B9D; Thu, 12 Aug 2004 10:34:11 +0200 (CEST) Received: from eikenes.alvestrand.no ([127.0.0.1]) by localhost (eikenes.alvestrand.no [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 18530-06; Thu, 12 Aug 2004 10:34:10 +0200 (CEST) Received: from halvestr-w2k02.emea.cisco.com (localhost.localdomain [127.0.0.1]) by eikenes.alvestrand.no (Postfix) with ESMTP id 277BA61B8E; Thu, 12 Aug 2004 10:34:10 +0200 (CEST) Date: Thu, 12 Aug 2004 10:34:09 +0200 From: Harald Tveit Alvestrand To: Melinda Shore Subject: Re: [Icar] icar-experiment-early-review-00 comments Message-ID: <09DBDE432925E7F06745323B@[192.168.1.43]> In-Reply-To: <80C51526-EBB9-11D8-923B-000A95E35274@cisco.com> References: <80C51526-EBB9-11D8-923B-000A95E35274@cisco.com> X-Mailer: Mulberry/3.1.5 (Win32) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline X-Virus-Scanned: by amavisd-new at alvestrand.no X-Spam-Score: 0.0 (/) X-Scan-Signature: 69a74e02bbee44ab4f8eafdbcedd94a1 Content-Transfer-Encoding: 7bit Cc: icar@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Improved Cross-Area Review List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: icar-bounces@ietf.org Errors-To: icar-bounces@ietf.org X-Spam-Score: 0.0 (/) X-Scan-Signature: 9ed51c9d1356100bce94f1ae4ec616a9 Content-Transfer-Encoding: 7bit Status: O --On 11. august 2004 13:11 -0400 Melinda Shore wrote: >> If it is to be submitted to IESG independent of a WG then an AD can >> handle it. > > I agree. What I really don't want to see is the reviewer pool, which I > think needs to be regarded as a scarce and precious resource, > being used to review drafts at the request of an author. That can be > handled through the usual mechanisms (posting a "hey, look at this" > message > to the appropriate mailing list, contacting interested people out-of- > band, etc.). When I first read the suggestion of "AD can handle it", my AD-load alarms went off - we have problems with getting cycles to handle independent submissions for the standards track that the author thinks is ready today, and doing it earlier in the process wouldn't make it easier (methinks). But after thinking for a few minutes, it looks more reasonable; if the NORMAL path for a non-WG submission for review is through a WG chair of a relevant WG (it doesn't have to be a WG document), it's OK to say that "if an AD feels the need to have a document reviewed, he/she can ask for it". So we get a statement saying: - A WG chair can request review for a document he/she finds relevant to the WG - An AD can request review of any document - Neither one is obliged to request a review. It's their call to do so. Makes sense? harald _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From icar-bounces@ietf.org Thu Aug 12 09:57:20 2004 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1BvFkx-0000mr-Tl for icar-web-archive@megatron.ietf.org; Thu, 12 Aug 2004 09:36:35 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id JAA01393 for ; Thu, 12 Aug 2004 09:36:34 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1BvFpt-00044A-P9 for icar-web-archive@ietf.org; Thu, 12 Aug 2004 09:41:44 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1BvFkX-0000Tv-Ty; Thu, 12 Aug 2004 09:36:09 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1BvFYT-0006I3-R9 for icar@megatron.ietf.org; Thu, 12 Aug 2004 09:23:41 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id JAA00640 for ; Thu, 12 Aug 2004 09:23:40 -0400 (EDT) Received: from mtagate4.de.ibm.com ([195.212.29.153]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1BvFdQ-0003o0-Ig for icar@ietf.org; Thu, 12 Aug 2004 09:28:50 -0400 Received: from d12nrmr1607.megacenter.de.ibm.com (d12nrmr1607.megacenter.de.ibm.com [9.149.167.49]) by mtagate4.de.ibm.com (8.12.10/8.12.10) with ESMTP id i7CDN869027350 for ; Thu, 12 Aug 2004 13:23:08 GMT Received: from sihl.zurich.ibm.com (sihl.zurich.ibm.com [9.4.16.232]) by d12nrmr1607.megacenter.de.ibm.com (8.12.10/NCO/VER6.6) with ESMTP id i7CDN8js053458 for ; Thu, 12 Aug 2004 15:23:08 +0200 Received: from zurich.ibm.com (sig-9-145-133-10.de.ibm.com [9.145.133.10]) by sihl.zurich.ibm.com (AIX4.3/8.9.3p2/8.9.3) with ESMTP id PAA81356 for ; Thu, 12 Aug 2004 15:23:08 +0200 Message-ID: <411B6F3A.8090509@zurich.ibm.com> Date: Thu, 12 Aug 2004 15:23:06 +0200 From: Brian E Carpenter Organization: IBM User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.6) Gecko/20040113 X-Accept-Language: en, fr, de MIME-Version: 1.0 To: icar@ietf.org Subject: Re: [Icar] icar-experiment-early-review-00 comments References: <80C51526-EBB9-11D8-923B-000A95E35274@cisco.com> <09DBDE432925E7F06745323B@[192.168.1.43]> In-Reply-To: <09DBDE432925E7F06745323B@[192.168.1.43]> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Score: 0.0 (/) X-Scan-Signature: 8b431ad66d60be2d47c7bfeb879db82c Content-Transfer-Encoding: 7bit X-BeenThere: icar@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Improved Cross-Area Review List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: icar-bounces@ietf.org Errors-To: icar-bounces@ietf.org X-Spam-Score: 0.0 (/) X-Scan-Signature: 82c9bddb247d9ba4471160a9a865a5f3 Content-Transfer-Encoding: 7bit Status: O Harald Tveit Alvestrand wrote: > > > --On 11. august 2004 13:11 -0400 Melinda Shore wrote: > >>> If it is to be submitted to IESG independent of a WG then an AD can >>> handle it. >> >> >> I agree. What I really don't want to see is the reviewer pool, which I >> think needs to be regarded as a scarce and precious resource, >> being used to review drafts at the request of an author. That can be >> handled through the usual mechanisms (posting a "hey, look at this" >> message >> to the appropriate mailing list, contacting interested people out-of- >> band, etc.). > > > When I first read the suggestion of "AD can handle it", my AD-load > alarms went off - we have problems with getting cycles to handle > independent submissions for the standards track that the author thinks > is ready today, and doing it earlier in the process wouldn't make it > easier (methinks). > > But after thinking for a few minutes, it looks more reasonable; if the > NORMAL path for a non-WG submission for review is through a WG chair of > a relevant WG (it doesn't have to be a WG document), it's OK to say that > "if an AD feels the need to have a document reviewed, he/she can ask for > it". > > So we get a statement saying: > > - A WG chair can request review for a document he/she finds relevant to > the WG > - An AD can request review of any document > - Neither one is obliged to request a review. It's their call to do so. > > Makes sense? > I think so. Of course, there is nothing to stop an author requesting reviews as a private matter, but having WG chairs and ADs as gatekeepers for the icar process seems appropriate. Of course, an author of a piece of cruft who can't get past any WG chair or AD may then go and annoy the RFC Editor. Ultimately, the RFC Editor relies on the same pool of potential reviewers as icar does, so the Principle of Conservation of Review Work applies. Since that is a law of Nature, we can't do much about it. Brian _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From icar-bounces@ietf.org Thu Aug 12 11:56:30 2004 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1BvHoc-0001pG-Qu for icar-web-archive@megatron.ietf.org; Thu, 12 Aug 2004 11:48:30 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA10122 for ; Thu, 12 Aug 2004 11:48:28 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1BvHtb-0006RF-7M for icar-web-archive@ietf.org; Thu, 12 Aug 2004 11:53:40 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1BvHmJ-0001B6-OH; Thu, 12 Aug 2004 11:46:07 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1BvHi6-0000Fx-I2 for icar@megatron.ietf.org; Thu, 12 Aug 2004 11:41:46 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA09729 for ; Thu, 12 Aug 2004 11:41:44 -0400 (EDT) Received: from joy.songbird.com ([208.184.79.7]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1BvHn5-0006JY-Fq for icar@ietf.org; Thu, 12 Aug 2004 11:46:56 -0400 Received: from bbprime (jay.songbird.com [208.184.79.253]) by joy.songbird.com (8.11.6/8.11.6) with ESMTP id i7CFfCj23736; Thu, 12 Aug 2004 08:41:12 -0700 Date: Thu, 12 Aug 2004 08:41:06 -0700 From: Dave Crocker Organization: Brandenburg InternetWorking X-Priority: 3 (Normal) Message-ID: <1186621368.20040812084106@brandenburg.com> To: Harald Tveit Alvestrand Subject: Re: [Icar] icar-experiment-early-review-00 comments In-Reply-To: <09DBDE432925E7F06745323B@[192\.168\.1\.43]> References: <80C51526-EBB9-11D8-923B-000A95E35274@cisco.com> <09DBDE432925E7F06745323B@[192.168.1.43]> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Spam-Score: 0.8 (/) X-Scan-Signature: e5ba305d0e64821bf3d8bc5d3bb07228 Content-Transfer-Encoding: 7bit Cc: icar@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Dave Crocker List-Id: Improved Cross-Area Review List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: icar-bounces@ietf.org Errors-To: icar-bounces@ietf.org X-Spam-Score: 0.8 (/) X-Scan-Signature: 52e1467c2184c31006318542db5614d5 Content-Transfer-Encoding: 7bit Status: O Harald, HTA> So we get a statement saying: HTA> - A WG chair can request review for a document he/she finds relevant to the HTA> WG HTA> - An AD can request review of any document HTA> - Neither one is obliged to request a review. It's their call to do so. Yes, this sounds reasonable. Here's why it isn't: Reviews are part of the document development process. It is not an AD's job to do the work of document development. That is the job of a working group or an independent submitter. An AD needs to facilitate the process of getting work done, but that is different than getting it done themselves. An example of facilitating would be to make sure that the responsible person understands what will help the process and how to get it done. Before assigning a task to an AD, ask whether it is essential, or whether the task can reasonably be done elsewhere. d/ -- Dave Crocker Brandenburg InternetWorking Sunnyvale, CA USA , _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From icar-bounces@ietf.org Wed Aug 25 12:47:53 2004 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA00130 for ; Wed, 25 Aug 2004 12:47:53 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1C00wy-0005Zu-On for icar-web-archive@ietf.org; Wed, 25 Aug 2004 12:48:44 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1C00ke-0004qU-1J; Wed, 25 Aug 2004 12:35:56 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1C00Ln-0000CB-Se for icar@megatron.ietf.org; Wed, 25 Aug 2004 12:10:15 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA27059 for ; Wed, 25 Aug 2004 12:10:08 -0400 (EDT) Received: from wyvern.icir.org ([192.150.187.14]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1C00MR-0004uB-Kc for icar@ietf.org; Wed, 25 Aug 2004 12:10:58 -0400 Received: from guns.icir.org (adsl-68-76-113-50.dsl.bcvloh.ameritech.net [68.76.113.50]) by wyvern.icir.org (8.12.9p1/8.12.8) with ESMTP id i7PGA6ZN034909 for ; Wed, 25 Aug 2004 09:10:06 -0700 (PDT) (envelope-from mallman@guns.icir.org) Received: from guns.icir.org (localhost [127.0.0.1]) by guns.icir.org (Postfix) with ESMTP id 091A077AA5C for ; Wed, 25 Aug 2004 12:10:05 -0400 (EDT) To: icar@ietf.org From: Mark Allman Organization: ICSI Center for Internet Research (ICIR) Song-of-the-Day: Sweet Home Alabama MIME-Version: 1.0 Date: Wed, 25 Aug 2004 12:10:05 -0400 Message-Id: <20040825161005.091A077AA5C@guns.icir.org> X-Spam-Score: 0.0 (/) X-Scan-Signature: 16a2b98d831858659c646b3dec9ed22b Subject: [Icar] draft minutes from San Diego X-BeenThere: icar@ietf.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: mallman@icir.org List-Id: Improved Cross-Area Review List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============1010746666==" Sender: icar-bounces@ietf.org Errors-To: icar-bounces@ietf.org X-Spam-Score: 0.0 (/) X-Scan-Signature: 6b519fb0ef66258f34533f52ff46aedf --===============1010746666== Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha1; protocol="application/pgp-signature" --=-=-= Content-Type: text/plain; charset=us-ascii The following are draft minutes from the ICAR meeting in San Diego. Thanks to Spencer for taking the minutes. Please send corrections. allman Improved Cross-Area Review WG (icar) Wednesday, August 4 at 1530-1730 ================================ CHAIRS: Mark Allman Joel Halpern NOTE-TAKER: Spencer Dawkins AGENDA: 1. intro/agenda bashing - co-chairs 2. discussion of draft-ietf-icar-experiment-early-review-00.txt - David Partain Draft to capture the state of WG discussions - supposed to capture WG mindset Open issues: - guidelines, if any, for the form of the review? From TSV - ask for important/crucial issues at the top of the review, as a summary of the review From ITU - major technical, minor technical, editorial categories This isn't about binding, it's about demonstrating to ADs that "major technical" items have been considered Working groups have to consider comments, can't just ignore them What is "must respond"? Working group is free to say "we're right", not free to say "we've never heard this comment before" - information about potential reviewers? How many requests, how many reviews ... Number of reviews is also a recognition item for reviewers Average time to complete a review per page of document Need to be collecting raw information as we go Need to avoid "top 10" lists of reviewers, etc. Don't set the threshold to become a reviewer too high - need to be bringing in new reviewers (see later discussion) Reviewer specializations? Mark will show what we have so far What kind of review are we asking for? Very specific topics, or more general? Need to accommodate specialist reviewers and generalist reviewers - Subjective criteria for admittance to the reviewer pool? (This discussion is in the context of the WG having agreed to a set of objective criteria, by which folks are eligible for the pool without additional approval.) Start by doing real work before you're admitted? AD can just decide? Requests are published publicly, so one could establish a track record Sponsorship by an AD? Definition of "good" for an AD is "what I would have figured out anyway"? Volunteering for reviewer pool and accepting/rejecting by an AD should be in secret If "n" (reviews) is sufficiently large, that flushes out a lot of the idiots "Sorry, but your reviews don't seem useful" - as a teaching tool - but need to keep from turning ADs into fulltime mentors - are reviewers expected to be willing to serve as mentors? Could this be this an area directorate responsibility (defined broadly)? EDU team should (obviously) be willing to help here Could have a separate pool of mentors, set up independently Are we doing early or late reviews? This document is focused on early review - the hope is that this helps with late reviews as well WG needs some assurance that they are likely to get a good review from anyone in the pool Nothing but early review can reduce late surprises - this needs to be part of the success criteria for ICAR Some set of reviews plus AD sponsorship? ADs can sponsor anyone, but recommend looking at previous reviews? We trust ADs to do a lot more than make choices like this without adult supervision (Objective - it's "two or more" RFCs, not "two") - How do we measure success/failure of ICAR experiments? Some types of failure are easy to detect, but success is harder - look for absence of failures and keep going? Objective criteria for failure, subjective criteria for success Helped or hindered in RFC production? Experiment will be over long before we get an RFC out through this mechanism Ask the chairs, not the document editors, for success, but ask the document editors for feedback as well - How are people removed from the pool, or is this process needed at all? At least remove yourself! Let bad reviewers rot on the vine? Need a minimal "periodic survey" for self-removal? People should "time-out" after six months, or something Removal on the basis of quality - one person's crank is another person's helpful reviewer If you qualify and time out, you still qualify when you ask to be added back ADs need a removal-for-cause mechanism - we do remove WG chairs from time to time If reviewers don't provide reviews - performance criteria for remaining in the pool? This is a voluntary process anyway - why do we need to throw people out of the pool? What's the worst case? Is it as bad as what a WG chair can do? We can throw them out now... Are we talking one percent bad reviewers or ten percent bad reviewers? We don't need to create an expulsion mechanism for the experiments We pretty much agree we need a timeout What about a reviewer with an ax to grind? Is "serve at the pleasure" language good enough? Is "the grapevine" good enough? Bad reviewers are worse for new working groups with new chairs. Why is this worse than a bozo appearing at IETF last call? The pool is supposed to have more clue, and the working group is supposed to respond What about reviewers that don't review? Just timing out is good enough Underburden bad reviewers, ignore them, or overburden them? We already have these mechanisms There is a difference between a process that's seldom used and one that doesn't exist We don't have consensus here - take this question to the list How does AD removal map to pre-qualified participants? To the list... - What can we hope for from the IESG? How do reviewers, prospective reviewers, and working groups interact with IESG? IESG has promised Harald they will provide sacrificial working groups for the experiment, and are concerned about providing names of people who don't have time to review documents for ADs now "Perhaps your working group should get an ICAR review" We're asking "help us find working groups, help us find reviewers, sponsor reviewers" - IESG is mumbling on the middle answer - Other comments? Non-performance will be a problem - need backup reviewers - yes, and WGs can handle this however they want to - we could learn this in an experiment, instead of noodling about it We are depending on volunteers - dates and plans to follow 3. the ICAR web page and soliciting for volunteers - Mark Allman Web page available from ICAR charter page Describes the proposal for an experiment ADs can request reviews for non-WG documents, too Pool members send reviews to icar mailing address WG considers comments and suggestions included in the reviews Reviewer attributes - expertise, reviews, ideal queue depth, current queue depth, plus arbitrary bio Reviews and responses available from reviewer page Could be sliced by WG, by draft, etc. Logging time when information about requests and reviews are entered into the system Feedback? Codify expertise? At least sort by expertise? Grade expertise? What about subjective grade inflation? What about "wireless", "XML", etc. 4. experience with the general area review team - Harald Alvestrand Gen-ART is late review - one week before IESG processing - and general review, including readability IETF chair unable to get responsible and timely reviews, and other people had directorates and review teams - so asked for help Called an experiment, getting experience with fast reviews, identify what turns out to be problems, and actually publish reviews One-area pilot, so Harald didn't have to ask for agreement, and Harald already had a web server and mail server Review guidelines stolen from SIR, driven by IESG mechanics Reviewers assigned round-robin, reviews and names are public, started January 22 Reviewers recruited word-of-mouth, "known to the AD", but also asked for volunteers on ICAR, no reviewers have crumbled yet One-person, one-document? Don't double up, because we have 20 documents per telechat and less than 40 reviewers 10 reviewers, 135 reviews, lots of concerns, 5 DISCUSSes (at least) How did this compare to SIR experiment? This is review at a different level Load on AD: down. Community involvement: Up Lessons learned - one week is a short time, reviewers need a discussion forum, ten people isn't enough, it needs to be a team, procedures are necessary, IT support is optional but nice. Trying to move standards-track review to IETF Last Call time Would a single discussion forum scale to 200 reviewers? Unresolved issues: Follow-up is important. Procedures for pointing at information is important. Getting the WG's attention for non-critical issues isn't a solved problem. Not sure who to send reviews to (ADs, WG chairs, editors, or some combination). Reviews need to be more visible to the community (you can find them now, if you're psychic). 5. General Discussion What sort of expectations for reviewer responsiveness on committing to a review? Hard deadlines help us focus... Short timeframes are good, when we're asking for a commitment - Dave will suggest a number on the mailing list --=-=-= Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.3 (FreeBSD) iD8DBQFBLLndWyrrWs4yIs4RAiTtAKCOD7NDcvZwaRNpYCaTTneYLQwUIACeKDoF hKMIFf0QzG4Ax8uU6SFOqyk= =Ib0n -----END PGP SIGNATURE----- --=-=-=-- --===============1010746666== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline Content-Transfer-Encoding: 7bit _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar --===============1010746666==-- From icar-bounces@ietf.org Wed Aug 25 13:11:34 2004 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA01675 for ; Wed, 25 Aug 2004 13:11:34 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1C01Jw-00060Z-3q for icar-web-archive@ietf.org; Wed, 25 Aug 2004 13:12:25 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1C00nb-0005wv-EI; Wed, 25 Aug 2004 12:38:59 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1C00ff-0003eA-Kc for icar@megatron.ietf.org; Wed, 25 Aug 2004 12:30:47 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA28634 for ; Wed, 25 Aug 2004 12:30:40 -0400 (EDT) Received: from wyvern.icir.org ([192.150.187.14]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1C00gL-0005F1-QS for icar@ietf.org; Wed, 25 Aug 2004 12:31:30 -0400 Received: from guns.icir.org (adsl-68-76-113-50.dsl.bcvloh.ameritech.net [68.76.113.50]) by wyvern.icir.org (8.12.9p1/8.12.8) with ESMTP id i7PGUeZN035230 for ; Wed, 25 Aug 2004 09:30:40 -0700 (PDT) (envelope-from mallman@guns.icir.org) Received: from guns.icir.org (localhost [127.0.0.1]) by guns.icir.org (Postfix) with ESMTP id 54C8D77AA5C for ; Wed, 25 Aug 2004 12:30:39 -0400 (EDT) To: icar@ietf.org From: Mark Allman Organization: ICSI Center for Internet Research (ICIR) Song-of-the-Day: Sweet Home Alabama MIME-Version: 1.0 Date: Wed, 25 Aug 2004 12:30:39 -0400 Message-Id: <20040825163039.54C8D77AA5C@guns.icir.org> X-Spam-Score: 0.0 (/) X-Scan-Signature: c1c65599517f9ac32519d043c37c5336 Subject: [Icar] pool entry; take N X-BeenThere: icar@ietf.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: mallman@icir.org List-Id: Improved Cross-Area Review List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============0585779641==" Sender: icar-bounces@ietf.org Errors-To: icar-bounces@ietf.org X-Spam-Score: 0.0 (/) X-Scan-Signature: b4a0a5f5992e2a4954405484e7717d8c --===============0585779641== Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha1; protocol="application/pgp-signature" --=-=-= Content-Type: text/plain; charset=us-ascii With my WG co-chair hat on (and, after having consulted with the holder of the other such hat) .... Based on the discussion on the list and the discussion at the San Diego IETF meeting I'd like to propose that we have rough consensus that subjective entry into the ICAR reviewer pool be based on AD sponsorship. And, further, we encourage folks to review a variety of documents that ICAR has been requested to review and present those to the ADs at the time of requesting sponsorship (but, that there is no magic N that is required and that sponsorship is not tied to only these reviews). I understand that some folks don't agree with this approach. However, it seems that we do have agreement to have *some* subjective criteria. And, further, we need to get this experiment going. I think we have rough consensus that the above is a reasonable first approach (that can, as with everything else, be tuned during the experiment). If you feel this is unreasonable please yell soon (especially if you have not voice an opinion on this issue in the past). We'd like to start some reviewing shortly. Thanks! allman -- Mark Allman -- ICIR -- http://www.icir.org/mallman/ --=-=-= Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.3 (FreeBSD) iD8DBQFBLL6vWyrrWs4yIs4RApLPAJ9BQXo0zfkZSIQSmpwqo6Y2WR/TdwCeJ9h8 3r/TNGz8FfcINR3THPd/sO0= =wC9E -----END PGP SIGNATURE----- --=-=-=-- --===============0585779641== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline Content-Transfer-Encoding: 7bit _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar --===============0585779641==-- From icar-bounces@ietf.org Wed Aug 25 18:35:52 2004 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id SAA03312 for ; Wed, 25 Aug 2004 18:35:52 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1C06Nm-0006HB-7r for icar-web-archive@ietf.org; Wed, 25 Aug 2004 18:36:45 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1C05w8-0000vB-82; Wed, 25 Aug 2004 18:08:08 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1C04i5-0001zn-7D for icar@megatron.ietf.org; Wed, 25 Aug 2004 16:49:33 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA18907 for ; Wed, 25 Aug 2004 16:49:26 -0400 (EDT) Received: from joy.songbird.com ([208.184.79.7]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1C04in-0001xw-Hh for icar@ietf.org; Wed, 25 Aug 2004 16:50:18 -0400 Received: from bbprime (jay.songbird.com [208.184.79.253]) by joy.songbird.com (8.11.6/8.11.6) with ESMTP id i7PKmrY05220; Wed, 25 Aug 2004 13:48:53 -0700 Date: Wed, 25 Aug 2004 13:48:48 -0700 From: Dave Crocker Organization: Brandenburg InternetWorking X-Priority: 3 (Normal) Message-ID: <84310048.20040825134848@brandenburg.com> To: Mark Allman Subject: Re: [Icar] pool entry; take N In-Reply-To: <20040825163039.54C8D77AA5C@guns.icir.org> References: <20040825163039.54C8D77AA5C@guns.icir.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Spam-Score: 0.8 (/) X-Scan-Signature: e5ba305d0e64821bf3d8bc5d3bb07228 Content-Transfer-Encoding: 7bit Cc: icar@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Dave Crocker List-Id: Improved Cross-Area Review List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: icar-bounces@ietf.org Errors-To: icar-bounces@ietf.org X-Spam-Score: 0.8 (/) X-Scan-Signature: 52e1467c2184c31006318542db5614d5 Content-Transfer-Encoding: 7bit Mark, MA> Based on the discussion on the list and the discussion at the San Diego MA> IETF meeting I'd like to propose that we have rough consensus that MA> subjective entry into the ICAR reviewer pool be based on AD sponsorship. ... MA> If you feel this is unreasonable please yell soon (especially if you I'm unclear whether the sponsorship mechanism is a way into the pool that is IN ADDITION to an objective, performance-based mechanism (a la SIRS) or whether it is the sole way. If it is in addition, then dandy. I'm all for it. If it is a sole way, it is a fundamental mistake, because it guarantees that the review process will be dominated by the ADs. However wonderful an AD is, or not, problems with IESG process have been consistently identified as a source of on-going IETF frustration. Making this technical review activity be completely subservient to ADs just adds it to the problem space, rather than letting it operate with a pure focus on technical issues. yours, continuing to bang my one note, d/ -- Dave Crocker Brandenburg InternetWorking Sunnyvale, CA USA _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From icar-bounces@ietf.org Wed Aug 25 20:23:51 2004 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id UAA09932 for ; Wed, 25 Aug 2004 20:23:51 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1C084I-0008A0-R6 for icar-web-archive@ietf.org; Wed, 25 Aug 2004 20:24:43 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1C07nj-0006wp-FA; Wed, 25 Aug 2004 20:07:35 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1C07br-0004qa-KK for icar@megatron.ietf.org; Wed, 25 Aug 2004 19:55:19 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id TAA08324 for ; Wed, 25 Aug 2004 19:55:03 -0400 (EDT) Received: from ns.execdsl.net ([208.184.15.238] helo=EXECDSL.COM) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1C07cR-0007fQ-Rr for icar@ietf.org; Wed, 25 Aug 2004 19:55:56 -0400 Received: from [69.170.3.232] (HELO JLaptop.stevecrocker.com) by EXECDSL.COM (CommuniGate Pro SMTP 3.3) with ESMTP id 7456036; Wed, 25 Aug 2004 19:54:34 -0400 Message-Id: <5.1.0.14.0.20040825195347.02417230@localhost> X-Sender: joel@stevecrocker.com@localhost X-Mailer: QUALCOMM Windows Eudora Version 5.1 Date: Wed, 25 Aug 2004 19:54:48 -0400 To: Dave Crocker From: "Joel M. Halpern" Subject: Re: [Icar] pool entry; take N In-Reply-To: <84310048.20040825134848@brandenburg.com> References: <20040825163039.54C8D77AA5C@guns.icir.org> <20040825163039.54C8D77AA5C@guns.icir.org> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed X-Spam-Score: 0.1 (/) X-Scan-Signature: cab78e1e39c4b328567edb48482b6a69 Cc: icar@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Improved Cross-Area Review List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: icar-bounces@ietf.org Errors-To: icar-bounces@ietf.org X-Spam-Score: 0.1 (/) X-Scan-Signature: 0ddefe323dd869ab027dbfff7eff0465 This is in addition to the objective criteria, not instead of them. THere had been an expressed wish by the working group for an additional, subjective, mechanism for getting people into the pool. This is our best read on the rough consensus for such a mechanism. Yours, Joel At 01:48 PM 8/25/2004 -0700, Dave Crocker wrote: >Mark, > >MA> Based on the discussion on the list and the discussion at the San Diego >MA> IETF meeting I'd like to propose that we have rough consensus that >MA> subjective entry into the ICAR reviewer pool be based on AD sponsorship. >... >MA> If you feel this is unreasonable please yell soon (especially if you > >I'm unclear whether the sponsorship mechanism is a way into the pool >that is IN ADDITION to an objective, performance-based mechanism (a la >SIRS) or whether it is the sole way. > >If it is in addition, then dandy. I'm all for it. > >If it is a sole way, it is a fundamental mistake, because it guarantees >that the review process will be dominated by the ADs. > >However wonderful an AD is, or not, problems with IESG process have been >consistently identified as a source of on-going IETF frustration. Making >this technical review activity be completely subservient to ADs just >adds it to the problem space, rather than letting it operate with a pure >focus on technical issues. > >yours, continuing to bang my one note, > >d/ >-- > Dave Crocker > Brandenburg InternetWorking > Sunnyvale, CA USA > > >_______________________________________________ >Icar mailing list >Icar@ietf.org >https://www1.ietf.org/mailman/listinfo/icar _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From icar-bounces@ietf.org Wed Aug 25 20:29:49 2004 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id UAA10179 for ; Wed, 25 Aug 2004 20:29:49 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1C08A5-0008Et-Pg for icar-web-archive@ietf.org; Wed, 25 Aug 2004 20:30:43 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1C082v-0001Te-Eu; Wed, 25 Aug 2004 20:23:17 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1C07l3-0006iU-Kx for icar@megatron.ietf.org; Wed, 25 Aug 2004 20:04:49 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id UAA08780 for ; Wed, 25 Aug 2004 20:04:43 -0400 (EDT) Received: from joy.songbird.com ([208.184.79.7]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1C07lm-0007ol-O8 for icar@ietf.org; Wed, 25 Aug 2004 20:05:36 -0400 Received: from bbprime (jay.songbird.com [208.184.79.253]) by joy.songbird.com (8.11.6/8.11.6) with ESMTP id i7Q047Y15550; Wed, 25 Aug 2004 17:04:07 -0700 Date: Wed, 25 Aug 2004 17:03:32 -0700 From: Dave Crocker Organization: Brandenburg InternetWorking X-Priority: 3 (Normal) Message-ID: <1157021322.20040825170332@brandenburg.com> To: "Joel M. Halpern" Subject: Re: [Icar] pool entry; take N In-Reply-To: <5.1.0.14.0.20040825195347.02417230@localhost> References: <20040825163039.54C8D77AA5C@guns.icir.org> <20040825163039.54C8D77AA5C@guns.icir.org> <5.1.0.14.0.20040825195347.02417230@localhost> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Spam-Score: 0.8 (/) X-Scan-Signature: cf4fa59384e76e63313391b70cd0dd25 Content-Transfer-Encoding: 7bit Cc: icar@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Dave Crocker List-Id: Improved Cross-Area Review List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: icar-bounces@ietf.org Errors-To: icar-bounces@ietf.org X-Spam-Score: 0.8 (/) X-Scan-Signature: 7d33c50f3756db14428398e2bdedd581 Content-Transfer-Encoding: 7bit Joel, JMH> This is in addition to the objective criteria, not instead of them. thanks for the clarification. more is better. please ignore my rant on 'only'. d/ -- Dave Crocker Brandenburg InternetWorking Sunnyvale, CA USA _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From icar-bounces@ietf.org Thu Aug 26 10:17:49 2004 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id KAA03027 for ; Thu, 26 Aug 2004 10:17:49 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1C0L5U-0004tC-Pm for icar-web-archive@ietf.org; Thu, 26 Aug 2004 10:18:50 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1C0EWp-0007Ie-JL; Thu, 26 Aug 2004 03:18:35 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1C0EIG-0004z6-5h for icar@megatron.ietf.org; Thu, 26 Aug 2004 03:03:32 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id DAA12359 for ; Thu, 26 Aug 2004 03:03:25 -0400 (EDT) Received: from mtagate3.de.ibm.com ([195.212.29.152]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1C0EJ1-0005tX-Ny for icar@ietf.org; Thu, 26 Aug 2004 03:04:22 -0400 Received: from d12nrmr1607.megacenter.de.ibm.com (d12nrmr1607.megacenter.de.ibm.com [9.149.167.49]) by mtagate3.de.ibm.com (8.12.10/8.12.10) with ESMTP id i7Q72LGm137194; Thu, 26 Aug 2004 07:02:21 GMT Received: from sihl.zurich.ibm.com (sihl.zurich.ibm.com [9.4.16.232]) by d12nrmr1607.megacenter.de.ibm.com (8.12.10/NCO/VER6.6) with ESMTP id i7Q72HIA089702; Thu, 26 Aug 2004 09:02:17 +0200 Received: from zurich.ibm.com (sig-9-146-229-158.de.ibm.com [9.146.229.158]) by sihl.zurich.ibm.com (AIX4.3/8.9.3p2/8.9.3) with ESMTP id JAA57574; Thu, 26 Aug 2004 09:02:16 +0200 Message-ID: <412D8AF7.7060700@zurich.ibm.com> Date: Thu, 26 Aug 2004 09:02:15 +0200 From: Brian E Carpenter Organization: IBM User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.6) Gecko/20040113 X-Accept-Language: en, fr, de MIME-Version: 1.0 To: "Joel M. Halpern" Subject: Re: [Icar] pool entry; take N References: <20040825163039.54C8D77AA5C@guns.icir.org> <20040825163039.54C8D77AA5C@guns.icir.org> <5.1.0.14.0.20040825195347.02417230@localhost> In-Reply-To: <5.1.0.14.0.20040825195347.02417230@localhost> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Score: 0.0 (/) X-Scan-Signature: 02ec665d00de228c50c93ed6b5e4fc1a Content-Transfer-Encoding: 7bit Cc: Dave Crocker , icar@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Improved Cross-Area Review List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: icar-bounces@ietf.org Errors-To: icar-bounces@ietf.org X-Spam-Score: 0.0 (/) X-Scan-Signature: b4a0a5f5992e2a4954405484e7717d8c Content-Transfer-Encoding: 7bit I'm happy with this, but maybe the co-chairs should check informally with the IESG that they will be willing to do this. Brian Joel M. Halpern wrote: > This is in addition to the objective criteria, not instead of them. > THere had been an expressed wish by the working group for an additional, > subjective, mechanism for getting people into the pool. > This is our best read on the rough consensus for such a mechanism. > Yours, > Joel > > At 01:48 PM 8/25/2004 -0700, Dave Crocker wrote: > >> Mark, >> >> MA> Based on the discussion on the list and the discussion at the San >> Diego >> MA> IETF meeting I'd like to propose that we have rough consensus that >> MA> subjective entry into the ICAR reviewer pool be based on AD >> sponsorship. >> ... >> MA> If you feel this is unreasonable please yell soon (especially if you >> >> I'm unclear whether the sponsorship mechanism is a way into the pool >> that is IN ADDITION to an objective, performance-based mechanism (a la >> SIRS) or whether it is the sole way. >> >> If it is in addition, then dandy. I'm all for it. >> >> If it is a sole way, it is a fundamental mistake, because it guarantees >> that the review process will be dominated by the ADs. >> >> However wonderful an AD is, or not, problems with IESG process have been >> consistently identified as a source of on-going IETF frustration. Making >> this technical review activity be completely subservient to ADs just >> adds it to the problem space, rather than letting it operate with a pure >> focus on technical issues. >> >> yours, continuing to bang my one note, >> >> d/ >> -- >> Dave Crocker >> Brandenburg InternetWorking >> Sunnyvale, CA USA >> >> >> _______________________________________________ >> Icar mailing list >> Icar@ietf.org >> https://www1.ietf.org/mailman/listinfo/icar > > > > _______________________________________________ > Icar mailing list > Icar@ietf.org > https://www1.ietf.org/mailman/listinfo/icar > _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From icar-bounces@ietf.org Thu Aug 26 10:54:43 2004 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id KAA07641 for ; Thu, 26 Aug 2004 10:54:43 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1C0LfF-00067u-6X for icar-web-archive@ietf.org; Thu, 26 Aug 2004 10:55:45 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1C0LU7-0007vv-D1; Thu, 26 Aug 2004 10:44:15 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1C0L8X-0002sa-Nq for icar@megatron.ietf.org; Thu, 26 Aug 2004 10:21:57 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id KAA03683 for ; Thu, 26 Aug 2004 10:21:56 -0400 (EDT) Received: from wyvern.icir.org ([192.150.187.14]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1C0L9T-00050k-Og for icar@ietf.org; Thu, 26 Aug 2004 10:22:57 -0400 Received: from guns.icir.org (adsl-68-76-113-50.dsl.bcvloh.ameritech.net [68.76.113.50]) by wyvern.icir.org (8.12.9p1/8.12.8) with ESMTP id i7QELsZN049698; Thu, 26 Aug 2004 07:21:54 -0700 (PDT) (envelope-from mallman@guns.icir.org) Received: from guns.icir.org (localhost [127.0.0.1]) by guns.icir.org (Postfix) with ESMTP id 807F377A6DD; Thu, 26 Aug 2004 10:21:53 -0400 (EDT) To: Brian E Carpenter From: Mark Allman Subject: Re: [Icar] pool entry; take N In-Reply-To: <412D8AF7.7060700@zurich.ibm.com> Organization: ICSI Center for Internet Research (ICIR) Song-of-the-Day: Blue on Black MIME-Version: 1.0 Date: Thu, 26 Aug 2004 10:21:53 -0400 Message-Id: <20040826142153.807F377A6DD@guns.icir.org> X-Spam-Score: 0.0 (/) X-Scan-Signature: e5ba305d0e64821bf3d8bc5d3bb07228 Cc: "Joel M. Halpern" , Dave Crocker , icar@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: mallman@icir.org List-Id: Improved Cross-Area Review List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============0205206184==" Sender: icar-bounces@ietf.org Errors-To: icar-bounces@ietf.org X-Spam-Score: 0.0 (/) X-Scan-Signature: 0ddefe323dd869ab027dbfff7eff0465 --===============0205206184== Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha1; protocol="application/pgp-signature" --=-=-= Content-Type: text/plain; charset=us-ascii > I'm happy with this, but maybe the co-chairs should check informally > with the IESG that they will be willing to do this. We did so, and in the ICAR meeting Harald indicated that this seems OK (after chatting with the IESG, after Joel and I asked him about it a few weeks prior to the San Diego meeting). allman -- Mark Allman -- ICIR -- http://www.icir.org/mallman/ --=-=-= Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.3 (FreeBSD) iD8DBQFBLfIBWyrrWs4yIs4RAmSZAJ9P36Xt20BLspd2303SvwtrF5oOPwCfcVFI zRmKq/NcyvC8wtMAUJ+e8qA= =Nu0r -----END PGP SIGNATURE----- --=-=-=-- --===============0205206184== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline Content-Transfer-Encoding: 7bit _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar --===============0205206184==-- From icar-bounces@ietf.org Thu Aug 26 11:29:30 2004 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA11228 for ; Thu, 26 Aug 2004 11:29:30 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1C0MCt-0007Dl-Ee for icar-web-archive@ietf.org; Thu, 26 Aug 2004 11:30:32 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1C0LzE-0007xG-OU; Thu, 26 Aug 2004 11:16:24 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1C0Lcq-0002Ro-7c for icar@megatron.ietf.org; Thu, 26 Aug 2004 10:53:16 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id KAA07448 for ; Thu, 26 Aug 2004 10:53:14 -0400 (EDT) Received: from eikenes.alvestrand.no ([158.38.152.233]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1C0Ldm-00063w-Km for icar@ietf.org; Thu, 26 Aug 2004 10:54:15 -0400 Received: from localhost (localhost.localdomain [127.0.0.1]) by eikenes.alvestrand.no (Postfix) with ESMTP id 7B5B161B9F; Thu, 26 Aug 2004 16:52:43 +0200 (CEST) Received: from eikenes.alvestrand.no ([127.0.0.1]) by localhost (eikenes.alvestrand.no [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 27165-07; Thu, 26 Aug 2004 16:52:42 +0200 (CEST) Received: from halvestr-w2k02.emea.cisco.com (localhost.localdomain [127.0.0.1]) by eikenes.alvestrand.no (Postfix) with ESMTP id DC78561B98; Thu, 26 Aug 2004 16:52:41 +0200 (CEST) Date: Thu, 26 Aug 2004 16:52:40 +0200 From: Harald Tveit Alvestrand To: Brian E Carpenter , "Joel M. Halpern" Subject: Re: [Icar] pool entry; take N Message-ID: <8F32DC8C857CD719D677D6D1@[192.168.1.225]> In-Reply-To: <412D8AF7.7060700@zurich.ibm.com> References: <20040825163039.54C8D77AA5C@guns.icir.org> <20040825163039.54C8D77AA5C@guns.icir.org> <5.1.0.14.0.20040825195347.02417230@localhost> <412D8AF7.7060700@zurich.ibm.com> X-Mailer: Mulberry/3.1.5 (Win32) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline X-Virus-Scanned: by amavisd-new at alvestrand.no X-Spam-Score: 0.0 (/) X-Scan-Signature: 1ac7cc0a4cd376402b85bc1961a86ac2 Content-Transfer-Encoding: 7bit Cc: icar@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Improved Cross-Area Review List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: icar-bounces@ietf.org Errors-To: icar-bounces@ietf.org X-Spam-Score: 0.0 (/) X-Scan-Signature: d6b246023072368de71562c0ab503126 Content-Transfer-Encoding: 7bit --On 26. august 2004 09:02 +0200 Brian E Carpenter wrote: > I'm happy with this, but maybe the co-chairs should check informally > with the IESG that they will be willing to do this. Did so in San Diego. As long as they can decide based on reviews, and the normal case is the "objective" metrics, they're OK with it. _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From icar-bounces@ietf.org Fri Aug 27 03:44:36 2004 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id DAA06863 for ; Fri, 27 Aug 2004 03:44:36 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1C0bQd-0002um-Vq for icar-web-archive@ietf.org; Fri, 27 Aug 2004 03:45:47 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1C0bKd-000266-9l; Fri, 27 Aug 2004 03:39:31 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1C0bJB-0001cT-Uk for icar@megatron.ietf.org; Fri, 27 Aug 2004 03:38:01 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id DAA06276 for ; Fri, 27 Aug 2004 03:37:59 -0400 (EDT) Received: from mtagate2.de.ibm.com ([195.212.29.151]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1C0bKH-0002jv-51 for icar@ietf.org; Fri, 27 Aug 2004 03:39:10 -0400 Received: from d12nrmr1607.megacenter.de.ibm.com (d12nrmr1607.megacenter.de.ibm.com [9.149.167.49]) by mtagate2.de.ibm.com (8.12.10/8.12.10) with ESMTP id i7R7bRFW140530 for ; Fri, 27 Aug 2004 07:37:27 GMT Received: from sihl.zurich.ibm.com (sihl.zurich.ibm.com [9.4.16.232]) by d12nrmr1607.megacenter.de.ibm.com (8.12.10/NCO/VER6.6) with ESMTP id i7R7bRB7089702 for ; Fri, 27 Aug 2004 09:37:27 +0200 Received: from zurich.ibm.com (dyn-9-13-126-72.ge.ch.ibm.com [9.13.126.72]) by sihl.zurich.ibm.com (AIX4.3/8.9.3p2/8.9.3) with ESMTP id JAA48228 for ; Fri, 27 Aug 2004 09:37:26 +0200 Message-ID: <412EE4BA.7010406@zurich.ibm.com> Date: Fri, 27 Aug 2004 09:37:30 +0200 From: Brian E Carpenter Organization: IBM User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.6) Gecko/20040113 X-Accept-Language: en, fr, de MIME-Version: 1.0 To: icar@ietf.org References: <20040825161005.091A077AA5C@guns.icir.org> In-Reply-To: <20040825161005.091A077AA5C@guns.icir.org> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Score: 0.0 (/) X-Scan-Signature: 9182cfff02fae4f1b6e9349e01d62f32 Content-Transfer-Encoding: 7bit Subject: [Icar] early, middle, or late X-BeenThere: icar@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Improved Cross-Area Review List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: icar-bounces@ietf.org Errors-To: icar-bounces@ietf.org X-Spam-Score: 0.0 (/) X-Scan-Signature: ea4ac80f790299f943f0a53be7e1a21a Content-Transfer-Encoding: 7bit The meeting minutes say: > Are we doing early or late reviews? This document is focused > on early review - the hope is that this helps with late > reviews as well I'd like to add that I've thought for some time that we need early, middle and late reviews: early: to detect major conceptual or architectural problems middle: to check that things are coming together well (e.g. that the security considerations have influenced the design rather than being an afterthought, that comments made in the early review have been addressed, etc.) late: that technical issues have been resolved, that additional items like IANA considerations have been dealt with, and general form and content issues. (needless to say, the middle and late reviews should include a sanity check that the previous reviews didn't miss any big problems.) Brian _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar