From exim@www1.ietf.org Mon Mar 1 00:20:00 2004 Received: from optimus.ietf.org (optimus.ietf.org [132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id AAA05011 for ; Mon, 1 Mar 2004 00:20:00 -0500 (EST) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1Axfq0-0004DT-QW for icar-archive@odin.ietf.org; Mon, 01 Mar 2004 00:19:32 -0500 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i215JWYI016205 for icar-archive@odin.ietf.org; Mon, 1 Mar 2004 00:19:32 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1Axfq0-0004DI-MB for icar-web-archive@optimus.ietf.org; Mon, 01 Mar 2004 00:19:32 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id AAA04950 for ; Mon, 1 Mar 2004 00:19:29 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1Axfpy-0001I6-00 for icar-web-archive@ietf.org; Mon, 01 Mar 2004 00:19:30 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1Axfp0-0001CW-00 for icar-web-archive@ietf.org; Mon, 01 Mar 2004 00:18:30 -0500 Received: from optimus.ietf.org ([132.151.1.19]) by ietf-mx with esmtp (Exim 4.12) id 1AxfoX-00017E-00 for icar-web-archive@ietf.org; Mon, 01 Mar 2004 00:18:01 -0500 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AxfoY-00044t-Cw; Mon, 01 Mar 2004 00:18:02 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1Axfo2-00043t-Dl for icar@optimus.ietf.org; Mon, 01 Mar 2004 00:17:30 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id AAA04837 for ; Mon, 1 Mar 2004 00:17:27 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1Axfo0-000165-00 for icar@ietf.org; Mon, 01 Mar 2004 00:17:28 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1Axfn9-000115-00 for icar@ietf.org; Mon, 01 Mar 2004 00:16:35 -0500 Received: from sccrmhc11.comcast.net ([204.127.202.55]) by ietf-mx with esmtp (Exim 4.12) id 1Axfmi-0000ty-00 for icar@ietf.org; Mon, 01 Mar 2004 00:16:08 -0500 Received: from dfnjgl21 (dfnjgl21.dhcp.ietf59.or.kr[218.37.225.240]) by comcast.net (sccrmhc11) with SMTP id <2004030105153801100m77die> (Authid: sdawkins@comcast.net); Mon, 1 Mar 2004 05:15:39 +0000 Message-ID: <01d301c3ff4c$3d6d3370$f0e125da@DFNJGL21> Reply-To: "Spencer Dawkins" From: "Spencer Dawkins" To: "ICAR Mailing List" Subject: Fw: [Icar] Moving on with ICAR: WG formation and charter Date: Mon, 1 Mar 2004 14:15:37 +0900 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2800.1158 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1165 Content-Transfer-Encoding: 7bit Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.1 required=5.0 tests=AWL autolearn=no version=2.60 Content-Transfer-Encoding: 7bit Content-Transfer-Encoding: 7bit Dear co-chairs, The attached e-mail may have been before your time (Alex was still kicking ICAR forward at the time), but I am curious about your thoughts on it. I won't be able to attend the session (as of now), but am hoping that we come up with some level of indirection, for everyone's benefit... Spencer From: "Alex Zinin" To: "Spencer Dawkins" Cc: Sent: Thursday, December 18, 2003 5:56 AM Subject: Re: [Icar] Moving on with ICAR: WG formation and charter > Spencer, > [deleted down to] > > > I know Alex is the routing AD - are we trying to move from a broadcast > > LAN in a working group to a routed network of reviewers? I've been > > following manet for a while, and think self-organizing routed networks > > is still a research topic :-} Actually, if this is what you are trying > > to do, it's worth saying so explicitly, because if this is what you're > > trying to do, we tend to rely on hierarchies for scalability :-{ > > Nice one ;) > > Again, seems the new thread on review types should help. > Using your analogy though, I think the goal should be to encourage > paying more attention to the traffic on the broadcast segment, > AND improve scalability of the router network so we can push > more traffic through it. > > My frank opinion regarding improving the former (broadcast or > unstructured review) is that it cuts into the area of motivation and > incentive, and we as the organization can only do so much here by > making sure people find required information easier, maybe give more > credits, etc. I personally do not know how to make sure more people > "care about the Internet" more... > > Alex _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Mon Mar 1 01:39:55 2004 Received: from optimus.ietf.org (optimus.ietf.org [132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id BAA09038 for ; Mon, 1 Mar 2004 01:39:55 -0500 (EST) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1Axh5K-0005Ap-Pq for icar-archive@odin.ietf.org; Mon, 01 Mar 2004 01:39:26 -0500 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i216dQTT019880 for icar-archive@odin.ietf.org; Mon, 1 Mar 2004 01:39:26 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1Axh5K-0005AM-AC for icar-web-archive@optimus.ietf.org; Mon, 01 Mar 2004 01:39:26 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id BAA09034 for ; Mon, 1 Mar 2004 01:39:24 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1Axh5H-0001SZ-00 for icar-web-archive@ietf.org; Mon, 01 Mar 2004 01:39:23 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1Axh4L-0001Oj-00 for icar-web-archive@ietf.org; Mon, 01 Mar 2004 01:38:25 -0500 Received: from optimus.ietf.org ([132.151.1.19]) by ietf-mx with esmtp (Exim 4.12) id 1Axh3v-0001Kx-00 for icar-web-archive@ietf.org; Mon, 01 Mar 2004 01:37:59 -0500 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1Axh3x-0004wE-LO; Mon, 01 Mar 2004 01:38:01 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1Axh3Q-0004kM-RJ for icar@optimus.ietf.org; Mon, 01 Mar 2004 01:37:28 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id BAA08975 for ; Mon, 1 Mar 2004 01:37:26 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1Axh3N-0001KU-00 for icar@ietf.org; Mon, 01 Mar 2004 01:37:25 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1Axh2T-0001GZ-00 for icar@ietf.org; Mon, 01 Mar 2004 01:36:29 -0500 Received: from ns.execdsl.net ([208.184.15.238] helo=EXECDSL.COM) by ietf-mx with esmtp (Exim 4.12) id 1Axh2G-0001CA-00 for icar@ietf.org; Mon, 01 Mar 2004 01:36:16 -0500 Received: from [210.93.162.110] (HELO JLaptop.stevecrocker.com) by EXECDSL.COM (CommuniGate Pro SMTP 3.3) with ESMTP id 6674319 for icar@ietf.org; Mon, 01 Mar 2004 01:36:16 -0500 Message-Id: <5.1.0.14.0.20040301013010.019448a0@localhost> X-Sender: joel@stevecrocker.com@localhost X-Mailer: QUALCOMM Windows Eudora Version 5.1 Date: Mon, 01 Mar 2004 01:36:06 -0500 To: icar@ietf.org From: "Joel M. Halpern" Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed Subject: [Icar] Agenda revisions Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.0 required=5.0 tests=none autolearn=no version=2.60 1) Introduction, brief review of Charter - 5 Minutes, Joel Halpern 2) Agenda Bash - short 3) Existing Experience: SIRS lessons - 10 minutes MIB Doctors lessons - 10 minutes Bert Wijnen Security Directorate experience - 10 minutes Steve Bellovin Routing Area Directorate - 10 minutes Alex Zinin 4) What constitutes a good review - 20 minutes (number of reviews? skill set of reviewers? focus? depth? breadth?)? 5) What should an early review mechanism look like? - 30 minutes draft-carpenter-solutions-sirs-02.txt draft-allman-problem-wg-revcomm-00.txt draft-ietf-ops-mib-review-guidelines-02.txt 6) What incentives can the organization put into place to encourage early review (both soliciting reviews and providing reviews)? Do we need incentive? - 20 minutes 7) What structures make sense of providing off-loading late / quasi-formal reviews? What properties should such a structure have?- 30 minutes draft-zinin-early-review-00.txt draft-iesg-alvestrand-twolevel (section 4.2) _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Mon Mar 1 03:50:16 2004 Received: from optimus.ietf.org (optimus.ietf.org [132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id DAA02913 for ; Mon, 1 Mar 2004 03:50:16 -0500 (EST) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1Axj7S-0002OG-RZ for icar-archive@odin.ietf.org; Mon, 01 Mar 2004 03:49:47 -0500 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i218nkTi009187 for icar-archive@odin.ietf.org; Mon, 1 Mar 2004 03:49:46 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1Axj7S-0002O6-L8 for icar-web-archive@optimus.ietf.org; Mon, 01 Mar 2004 03:49:46 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id DAA02861 for ; Mon, 1 Mar 2004 03:49:45 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1Axj7Q-000011-00 for icar-web-archive@ietf.org; Mon, 01 Mar 2004 03:49:44 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1Axj6U-0007hX-00 for icar-web-archive@ietf.org; Mon, 01 Mar 2004 03:48:47 -0500 Received: from optimus.ietf.org ([132.151.1.19]) by ietf-mx with esmtp (Exim 4.12) id 1Axj5m-0007av-00 for icar-web-archive@ietf.org; Mon, 01 Mar 2004 03:48:02 -0500 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1Axj5m-000248-Qa; Mon, 01 Mar 2004 03:48:02 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1Axisf-0000tA-36 for icar@optimus.ietf.org; Mon, 01 Mar 2004 03:34:29 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id DAA02177 for ; Mon, 1 Mar 2004 03:34:27 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1Axisc-0006Ii-00 for icar@ietf.org; Mon, 01 Mar 2004 03:34:26 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1Axire-0006Bo-00 for icar@ietf.org; Mon, 01 Mar 2004 03:33:26 -0500 Received: from joy.songbird.com ([208.184.79.7]) by ietf-mx with esmtp (Exim 4.12) id 1Axiqh-00062D-00 for icar@ietf.org; Mon, 01 Mar 2004 03:32:27 -0500 Received: from BBFUJIP.brandenburg.com (jay.songbird.com [208.184.79.253]) by joy.songbird.com (8.11.6/8.11.6) with ESMTP id i218ecd24693; Mon, 1 Mar 2004 00:40:38 -0800 Date: Mon, 1 Mar 2004 17:31:44 +0900 From: Dave Crocker Organization: Brandenburg InternetWorking X-Priority: 3 (Normal) Message-ID: <1097854518.20040301173144@brandenburg.com> To: Mark Allman CC: icar@ietf.org Subject: Re: [Icar] agenda In-Reply-To: <20040301035811.5700010D065@lawyers.icir.org> References: <20040301035811.5700010D065@lawyers.icir.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Content-Transfer-Encoding: 7bit Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.8 required=5.0 tests=AWL,PRIORITY_NO_NAME autolearn=no version=2.60 Content-Transfer-Encoding: 7bit Content-Transfer-Encoding: 7bit Mark, MA> The WG meeting will tackle the following questions: The focus of the agenda is on types of reviews and overcoming the hurdle of getting working groups to get reviews done. An orthogonal issue is the structure of the review "group". Should it be under the IESG? Should there be one for each area? Should it be independent of other IETF management groups (albeit with liaison relationships, of course)? Related, of course, are the qualifications for being a reviewer. d/ -- Dave Crocker Brandenburg InternetWorking Sunnyvale, CA USA _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Mon Mar 1 06:06:14 2004 Received: from optimus.ietf.org (optimus.ietf.org [132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id GAA10085 for ; Mon, 1 Mar 2004 06:06:14 -0500 (EST) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AxlF5-0006fn-1e for icar-archive@odin.ietf.org; Mon, 01 Mar 2004 06:05:47 -0500 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i21B5l4b025647 for icar-archive@odin.ietf.org; Mon, 1 Mar 2004 06:05:47 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AxlF4-0006fa-S9 for icar-web-archive@optimus.ietf.org; Mon, 01 Mar 2004 06:05:46 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id GAA10058 for ; Mon, 1 Mar 2004 06:05:43 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1AxlF1-0006Mo-00 for icar-web-archive@ietf.org; Mon, 01 Mar 2004 06:05:43 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1AxlE3-0006Gz-00 for icar-web-archive@ietf.org; Mon, 01 Mar 2004 06:04:44 -0500 Received: from optimus.ietf.org ([132.151.1.19]) by ietf-mx with esmtp (Exim 4.12) id 1AxlDK-0006Be-00 for icar-web-archive@ietf.org; Mon, 01 Mar 2004 06:03:58 -0500 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AxlDN-0006MG-4o; Mon, 01 Mar 2004 06:04:01 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AxlCz-0006Kr-Ck for icar@optimus.ietf.org; Mon, 01 Mar 2004 06:03:37 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id GAA09921 for ; Mon, 1 Mar 2004 06:03:34 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1AxlCv-00069D-00 for icar@ietf.org; Mon, 01 Mar 2004 06:03:33 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1AxlC2-00063O-00 for icar@ietf.org; Mon, 01 Mar 2004 06:02:38 -0500 Received: from joy.songbird.com ([208.184.79.7]) by ietf-mx with esmtp (Exim 4.12) id 1AxlBC-0005t4-00 for icar@ietf.org; Mon, 01 Mar 2004 06:01:46 -0500 Received: from BBFUJIP.brandenburg.com (jay.songbird.com [208.184.79.253]) by joy.songbird.com (8.11.6/8.11.6) with ESMTP id i21BA4d01360; Mon, 1 Mar 2004 03:10:04 -0800 Date: Mon, 1 Mar 2004 20:01:16 +0900 From: Dave Crocker Reply-To: Dave Crocker Organization: Brandenburg InternetWorking X-Priority: 3 (Normal) Message-ID: <359897983.20040301200116@brandenburg.com> To: "Joel M. Halpern" CC: icar@ietf.org Subject: Re: [Icar] agenda In-Reply-To: <5.1.0.14.0.20040229234325.01a25d98@localhost> References: <20040301035811.5700010D065@lawyers.icir.org> <5.1.0.14.0.20040229234325.01a25d98@localhost> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Content-Transfer-Encoding: 7bit Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.7 required=5.0 tests=AWL,PRIORITY_NO_NAME autolearn=no version=2.60 Content-Transfer-Encoding: 7bit Content-Transfer-Encoding: 7bit Joel, JMH> I have received some private feedback about SIRS that I would like to get JMH> to the group, but I am still looking for someone to start off a discussion JMH> on it. I'm triple-booked for that time slot, but will be glad to do a sir summary. d/ -- Dave Crocker Brandenburg InternetWorking Sunnyvale, CA USA _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Mon Mar 1 06:07:13 2004 Received: from optimus.ietf.org (optimus.ietf.org [132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id GAA10151 for ; Mon, 1 Mar 2004 06:07:12 -0500 (EST) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AxlG1-0006jk-MZ for icar-archive@odin.ietf.org; Mon, 01 Mar 2004 06:06:45 -0500 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i21B6jvO025890 for icar-archive@odin.ietf.org; Mon, 1 Mar 2004 06:06:45 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AxlG1-0006jV-Gk for icar-web-archive@optimus.ietf.org; Mon, 01 Mar 2004 06:06:45 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id GAA10125 for ; Mon, 1 Mar 2004 06:06:42 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1AxlFx-0006TZ-00 for icar-web-archive@ietf.org; Mon, 01 Mar 2004 06:06:41 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1AxlF6-0006NU-00 for icar-web-archive@ietf.org; Mon, 01 Mar 2004 06:05:48 -0500 Received: from optimus.ietf.org ([132.151.1.19]) by ietf-mx with esmtp (Exim 4.12) id 1AxlEJ-0006Hm-00 for icar-web-archive@ietf.org; Mon, 01 Mar 2004 06:04:59 -0500 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AxlEM-0006WO-Hd; Mon, 01 Mar 2004 06:05:02 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AxlDv-0006UR-32 for icar@optimus.ietf.org; Mon, 01 Mar 2004 06:04:35 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id GAA09964 for ; Mon, 1 Mar 2004 06:04:31 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1AxlDr-0006FB-00 for icar@ietf.org; Mon, 01 Mar 2004 06:04:31 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1AxlCu-00068z-00 for icar@ietf.org; Mon, 01 Mar 2004 06:03:32 -0500 Received: from joy.songbird.com ([208.184.79.7]) by ietf-mx with esmtp (Exim 4.12) id 1AxlBz-0005yn-00 for icar@ietf.org; Mon, 01 Mar 2004 06:02:35 -0500 Received: from BBFUJIP.brandenburg.com (jay.songbird.com [208.184.79.253]) by joy.songbird.com (8.11.6/8.11.6) with ESMTP id i21BArd01413; Mon, 1 Mar 2004 03:10:53 -0800 Date: Mon, 1 Mar 2004 20:02:03 +0900 From: Dave Crocker Reply-To: Dave Crocker Organization: Brandenburg InternetWorking X-Priority: 3 (Normal) Message-ID: <1062267814.20040301200203@brandenburg.com> To: Mark Allman CC: icar@ietf.org Subject: Re: [Icar] agenda In-Reply-To: <20040301035811.5700010D065@lawyers.icir.org> References: <20040301035811.5700010D065@lawyers.icir.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Content-Transfer-Encoding: 7bit Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.7 required=5.0 tests=AWL,PRIORITY_NO_NAME autolearn=no version=2.60 Content-Transfer-Encoding: 7bit Content-Transfer-Encoding: 7bit Mark, MA> The WG meeting will tackle the following questions: The focus of the agenda is on types of reviews and overcoming the hurdle of getting working groups to get reviews done. An orthogonal issue is the structure of the review "group". Should it be under the IESG? Should there be one for each area? Should it be independent of other IETF management groups (albeit with liaison relationships, of course)? Related, of course, are the qualifications for being a reviewer. d/ -- Dave Crocker Brandenburg InternetWorking Sunnyvale, CA USA _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Mon Mar 1 06:07:15 2004 Received: from optimus.ietf.org (optimus.ietf.org [132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id GAA10168 for ; Mon, 1 Mar 2004 06:07:15 -0500 (EST) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AxlG4-0006k7-MU for icar-archive@odin.ietf.org; Mon, 01 Mar 2004 06:06:48 -0500 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i21B6mkv025913 for icar-archive@odin.ietf.org; Mon, 1 Mar 2004 06:06:48 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AxlG4-0006js-Gz for icar-web-archive@optimus.ietf.org; Mon, 01 Mar 2004 06:06:48 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id GAA10132 for ; Mon, 1 Mar 2004 06:06:45 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1AxlG0-0006U0-00 for icar-web-archive@ietf.org; Mon, 01 Mar 2004 06:06:44 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1AxlF7-0006Np-00 for icar-web-archive@ietf.org; Mon, 01 Mar 2004 06:05:50 -0500 Received: from optimus.ietf.org ([132.151.1.19]) by ietf-mx with esmtp (Exim 4.12) id 1AxlEK-0006Hq-00 for icar-web-archive@ietf.org; Mon, 01 Mar 2004 06:05:00 -0500 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AxlEN-0006XU-Hl; Mon, 01 Mar 2004 06:05:03 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AxlDz-0006Ul-UT for icar@optimus.ietf.org; Mon, 01 Mar 2004 06:04:39 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id GAA09975 for ; Mon, 1 Mar 2004 06:04:36 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1AxlDw-0006Fu-00 for icar@ietf.org; Mon, 01 Mar 2004 06:04:36 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1AxlD6-0006Am-00 for icar@ietf.org; Mon, 01 Mar 2004 06:03:44 -0500 Received: from joy.songbird.com ([208.184.79.7]) by ietf-mx with esmtp (Exim 4.12) id 1AxlCH-0005z0-00 for icar@ietf.org; Mon, 01 Mar 2004 06:02:53 -0500 Received: from BBFUJIP.brandenburg.com (jay.songbird.com [208.184.79.253]) by joy.songbird.com (8.11.6/8.11.6) with ESMTP id i21BBBd01421; Mon, 1 Mar 2004 03:11:11 -0800 Date: Mon, 1 Mar 2004 20:02:21 +0900 From: Dave Crocker Reply-To: Dave Crocker Organization: Brandenburg InternetWorking X-Priority: 3 (Normal) Message-ID: <901255565.20040301200221@brandenburg.com> To: Mark Allman CC: icar@ietf.org Subject: Re: [Icar] agenda In-Reply-To: <20040301035811.5700010D065@lawyers.icir.org> References: <20040301035811.5700010D065@lawyers.icir.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Content-Transfer-Encoding: 7bit Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.7 required=5.0 tests=AWL,PRIORITY_NO_NAME autolearn=no version=2.60 Content-Transfer-Encoding: 7bit Content-Transfer-Encoding: 7bit Mark, MA> The WG meeting will tackle the following questions: The focus of the agenda is on types of reviews and overcoming the hurdle of getting working groups to get reviews done. An orthogonal issue is the structure of the review "group". Should it be under the IESG? Should there be one for each area? Should it be independent of other IETF management groups (albeit with liaison relationships, of course)? Related, of course, are the qualifications for being a reviewer. d/ -- Dave Crocker Brandenburg InternetWorking Sunnyvale, CA USA _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Mon Mar 8 19:49:30 2004 Received: from optimus.ietf.org (optimus.ietf.org [132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id TAA14175 for ; Mon, 8 Mar 2004 19:49:30 -0500 (EST) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B0VQb-0004cY-Gt for icar-archive@odin.ietf.org; Mon, 08 Mar 2004 19:49:02 -0500 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i290n1YO017758 for icar-archive@odin.ietf.org; Mon, 8 Mar 2004 19:49:01 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B0VQb-0004cJ-Bs for icar-web-archive@optimus.ietf.org; Mon, 08 Mar 2004 19:49:01 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id TAA14097 for ; Mon, 8 Mar 2004 19:48:59 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1B0VQZ-0007W5-00 for icar-web-archive@ietf.org; Mon, 08 Mar 2004 19:48:59 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1B0VPa-0007KY-00 for icar-web-archive@ietf.org; Mon, 08 Mar 2004 19:47:58 -0500 Received: from optimus.ietf.org ([132.151.1.19]) by ietf-mx with esmtp (Exim 4.12) id 1B0VOf-0007Az-00 for icar-web-archive@ietf.org; Mon, 08 Mar 2004 19:47:01 -0500 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B0VOf-0004Wx-8s; Mon, 08 Mar 2004 19:47:01 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B0VNh-0004Ux-GN for icar@optimus.ietf.org; Mon, 08 Mar 2004 19:46:01 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id TAA14004 for ; Mon, 8 Mar 2004 19:45:59 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1B0VNf-00071Q-00 for icar@ietf.org; Mon, 08 Mar 2004 19:45:59 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1B0VMl-0006rq-00 for icar@ietf.org; Mon, 08 Mar 2004 19:45:04 -0500 Received: from joy.songbird.com ([208.184.79.7]) by ietf-mx with esmtp (Exim 4.12) id 1B0VLp-0006ZP-00 for icar@ietf.org; Mon, 08 Mar 2004 19:44:06 -0500 Received: from bbprime (jay.songbird.com [208.184.79.253]) by joy.songbird.com (8.11.6/8.11.6) with ESMTP id i290qUd16097 for ; Mon, 8 Mar 2004 16:52:30 -0800 Date: Mon, 8 Mar 2004 16:43:30 -0800 From: Dave Crocker Reply-To: Dave Crocker Organization: Brandenburg InternetWorking X-Priority: 3 (Normal) Message-ID: <1221060422.20040308164330@brandenburg.com> To: icar@ietf.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Content-Transfer-Encoding: 7bit Subject: [Icar] independence of reviews; variability Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.7 required=5.0 tests=AWL,PRIORITY_NO_NAME autolearn=no version=2.60 Content-Transfer-Encoding: 7bit Content-Transfer-Encoding: 7bit Folks, At the icar meeting, last week, Harald and I had an exchange about the independence of review effort. I was being channeled through jabber, so I'm relying on the jabber log for the details. I think the exchange highlights some very basic issues, both about reviewing and about IETF culture and organization. It would be extremely helpful to pursue this further. Here are the relevant parts of the jabber logs. I'm hoping I have retained all the relevant text, which was interspersed with other threads. I've also included some of the log from Harald's presentation, which touches on variability of reviews. This message is just the relevant text. I'll post comments separately. d/ --- (edited) JABBER LOG --- dcrocker says: I strongly suggest that the review function be independent of the IETF administrative management team. The reviewing stuff is not about helping an "area" or dealing with any aspect of working group process. It is strictly for technical commentary. hta: every time dave says something, I have to say the opposite. if you want to create a body with power and responsibility outside the IETF management structure with power and responsibility, be careful what you ask for dcrocker says: what "power" does a review group have and what "benefit" is there in making such a purely technical effort subject to the varied goals of ietf management? hta: if review is to make any sense, it has to influence the quality, relevance, timeliness of ourput. the managemt exists to influence the quality, relevance, timeliness of ietf output Why. Of course I don't see the value of independence in the review other than to get people who will see clarity problems. [JJJ]: Are you saying we can already ignore reviewers? hta: If you have reviewers who make a difference, they have power dcrocker says: working group / iesg conflict is a fact of life. an independent review process provides _independent_ input to such conflict. Next: harald alvestrand on draft-ietf-alvestrand-twolevel-00.txt (if I got this right) ... note document proposes giving review groups ability to approve documents [did I get this right]; that is out of scope for here [Dave Thaler]: how is it determined which team a document is reviewed by? hta: throw dice, except that when a group of docs have strong internal refrerences, they should be treated by the same review team [Dave Thaler]: concern I have with this is that not all review teams will be equal, not just in terms of quality, but in terms of what direction one team may say "this is a good idea" while another wants to go another direction who has ultimate responsibility? central management important can't "shop around" for a team that is friendly to what you want to do. need some way to counter that hta: if you're going to have more than one group, you need to expect and deal with variations and judgement. need to have someplace in the system that can take a review team and say "you're out of line" i believe in central authoritty (we have no kings, but we have central authority) dcrocker says: shopping around is fine, if the reviews must be credible on their own right, rather than by virtue of some organizational affiliation. If a wg gets only friendly reviews, they have not done their job. [Dave Thaler]: if the team is chosen randomly, and there are differences between teams, then WG may not know how to construct a document that will pass. f it is not random, there may be forum shopping\ hta: if we have multiple teams, we may have more guidelines greg daily(?): when doc goes to ietf last call, there can be a dynamic selection of reviewers : differences between sets of reviews - this is goodness, not badness. the job of a wg is to resolve differences and considerations. if reviews do not differ, they probably are not adding much benefit. -- Dave Crocker Brandenburg InternetWorking Sunnyvale, CA USA _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Mon Mar 8 20:50:33 2004 Received: from optimus.ietf.org (optimus.ietf.org [132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id UAA16208 for ; Mon, 8 Mar 2004 20:50:33 -0500 (EST) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B0WNh-0008AC-PO for icar-archive@odin.ietf.org; Mon, 08 Mar 2004 20:50:05 -0500 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i291o5Xt031374 for icar-archive@odin.ietf.org; Mon, 8 Mar 2004 20:50:05 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B0WNh-00089x-Jw for icar-web-archive@optimus.ietf.org; Mon, 08 Mar 2004 20:50:05 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id UAA16194 for ; Mon, 8 Mar 2004 20:50:02 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1B0WNf-0001cA-00 for icar-web-archive@ietf.org; Mon, 08 Mar 2004 20:50:03 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1B0WMf-0001Sk-00 for icar-web-archive@ietf.org; Mon, 08 Mar 2004 20:49:02 -0500 Received: from optimus.ietf.org ([132.151.1.19]) by ietf-mx with esmtp (Exim 4.12) id 1B0WLg-0001JT-00 for icar-web-archive@ietf.org; Mon, 08 Mar 2004 20:48:00 -0500 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B0WLh-00084x-3P; Mon, 08 Mar 2004 20:48:01 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B0WKy-00080a-Uk for icar@optimus.ietf.org; Mon, 08 Mar 2004 20:47:16 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id UAA16129 for ; Mon, 8 Mar 2004 20:47:14 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1B0WKw-0001As-00 for icar@ietf.org; Mon, 08 Mar 2004 20:47:14 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1B0WK2-00011B-00 for icar@ietf.org; Mon, 08 Mar 2004 20:46:18 -0500 Received: from joy.songbird.com ([208.184.79.7]) by ietf-mx with esmtp (Exim 4.12) id 1B0WJP-0000qL-00 for icar@ietf.org; Mon, 08 Mar 2004 20:45:40 -0500 Received: from bbprime (jay.songbird.com [208.184.79.253]) by joy.songbird.com (8.11.6/8.11.6) with ESMTP id i291s9d20229 for ; Mon, 8 Mar 2004 17:54:09 -0800 Date: Mon, 8 Mar 2004 17:45:08 -0800 From: Dave Crocker Reply-To: dcrocker@brandenburg.com Organization: Brandenburg InternetWorking X-Priority: 3 (Normal) Message-ID: <1609599601.20040308174508@brandenburg.com> To: icar@ietf.org Subject: Re: [Icar] independence of reviews; variability In-Reply-To: <1221060422.20040308164330@brandenburg.com> References: <1221060422.20040308164330@brandenburg.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Content-Transfer-Encoding: 7bit Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.7 required=5.0 tests=AWL,PRIORITY_NO_NAME autolearn=no version=2.60 Content-Transfer-Encoding: 7bit Content-Transfer-Encoding: 7bit So, now some comments and questions: DC> hta: if you want to create a body with power and responsibility DC> outside the IETF management structure with power and responsibility, DC> be careful what you ask for The caution is warranted. When organizing power centers, be very careful. And fully centralized power is particularly subject to problems, particularly for groups with highly diverse, volunteer participation, like the IETF. In fact, the history of the IETF has tended towards particularly DE-centralized activities. This is exactly what has changed over recent years and could reasonably be interpreted as one of the sources of current problems. Note, for example, that the effort to exert arbitrary, centralized power was what triggered the 1992 Kobe situation. The real work of the organization is initiated by, and performed by, self-motivated volunteers. At base, this is a grass-roots organization. Highly centralized control works against a grass-roots sense of empowerment and responsibility. (Lest one think this means that there does not need to be coordination and quality control, it doesn't.) Reviews are supposed to be technical analyses, rather than sources of design recommendations. The latter always get added, because anyone good enough to be a reviewer will also have thoughts on how to improve the thing being reviewed. But the core benefit of a review is the independent perspective on how well or poorly the thing being reviewed will work. DC> hta: if review is to make any sense, it has to influence the quality, DC> relevance, timeliness of ourput. the managemt exists to influence the DC> quality, relevance, timeliness of ietf output The fact that this activity is being translated into a question of power challenges the basic credibility of these volunteer contributions, since it suggests that the reviewers need to be controlled. This is at odds with the IETF's history of collaboration rather than of leverage through power. Such a requirement deprives reviews of the independence needed to ensure the essential variability of perspective that a reviews are supposed to have. If we view the IESG as "being in charge" of the IETF and really being the "authority" for IETF decisions, then perhaps is it is useful to give the IESG control over qualification and selection of reviewers. If, instead, we view the IETF as being a collaborative organization that seeks to build broad-based support for its decisions -- with the IESG coordinating and assessing that support -- then we need to be organized and operated in a way that encourages the diversity and independence that make claims of broad-based support credible. d/ -- Dave Crocker Brandenburg InternetWorking Sunnyvale, CA USA _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Mon Mar 8 22:50:32 2004 Received: from optimus.ietf.org (optimus.ietf.org [132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id WAA19153 for ; Mon, 8 Mar 2004 22:50:32 -0500 (EST) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B0YFq-0005dZ-Tk for icar-archive@odin.ietf.org; Mon, 08 Mar 2004 22:50:06 -0500 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i293o6s0021663 for icar-archive@odin.ietf.org; Mon, 8 Mar 2004 22:50:06 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B0YFq-0005dK-PL for icar-web-archive@optimus.ietf.org; Mon, 08 Mar 2004 22:50:06 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id WAA19131 for ; Mon, 8 Mar 2004 22:50:02 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1B0YFn-0004HY-00 for icar-web-archive@ietf.org; Mon, 08 Mar 2004 22:50:03 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1B0YEo-00048U-00 for icar-web-archive@ietf.org; Mon, 08 Mar 2004 22:49:03 -0500 Received: from optimus.ietf.org ([132.151.1.19]) by ietf-mx with esmtp (Exim 4.12) id 1B0YDp-0003x6-00 for icar-web-archive@ietf.org; Mon, 08 Mar 2004 22:48:01 -0500 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B0YDp-0005ZW-Ka; Mon, 08 Mar 2004 22:48:01 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B0YD5-0005Yj-8C for icar@optimus.ietf.org; Mon, 08 Mar 2004 22:47:15 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id WAA19064 for ; Mon, 8 Mar 2004 22:47:10 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1B0YD1-0003rb-00 for icar@ietf.org; Mon, 08 Mar 2004 22:47:11 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1B0YCA-0003id-00 for icar@ietf.org; Mon, 08 Mar 2004 22:46:18 -0500 Received: from rwcrmhc12.comcast.net ([216.148.227.85]) by ietf-mx with esmtp (Exim 4.12) id 1B0YBX-0003Z5-00 for icar@ietf.org; Mon, 08 Mar 2004 22:45:40 -0500 Received: from dfnjgl21 (c-24-1-97-129.client.comcast.net[24.1.97.129]) by comcast.net (rwcrmhc12) with SMTP id <20040309034409014006sia5e> (Authid: sdawkins@comcast.net); Tue, 9 Mar 2004 03:44:10 +0000 Message-ID: <035201c40588$c86cc840$0400a8c0@DFNJGL21> Reply-To: "Spencer Dawkins" From: "Spencer Dawkins" To: References: <1221060422.20040308164330@brandenburg.com> Subject: Re: [Icar] independence of reviews; variability Date: Mon, 8 Mar 2004 21:44:08 -0600 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2800.1158 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1165 Content-Transfer-Encoding: 7bit Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.0 required=5.0 tests=AWL autolearn=no version=2.60 Content-Transfer-Encoding: 7bit Content-Transfer-Encoding: 7bit FWIW, my understanding of the current "IETF Last Call" mechanics is that reviewers today are, in the final analysis, "uncontrolled". A scathing review correctly pointing out technical flaws is always admissible at any point in the review cycle, all the way through publication by the RFC Editor, no matter who wrote it or why. If we don't think anyone except area directors reads documents any more, discussion about "reviewers inside/outside the management structure" might make sense. I had hoped that we would not be starting down that road. Having a Internet Review Board that is entirely independent of current management structures works for me. Having an Internet Review Board that "reports" to the IAB works for me. We may end up with Area Review Teams, but that is *not* the only reasonable answer. Spencer From: "Dave Crocker" > --- (edited) JABBER LOG --- > > dcrocker says: I strongly suggest that the review function be > independent of the IETF administrative management team. The reviewing > stuff is not about helping an "area" or dealing with any aspect of > working group process. It is strictly for technical commentary. > > hta: every time dave says something, I have to say the opposite. if you > want to create a body with power and responsibility outside the IETF > management structure with power and responsibility, be careful what you > ask for _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Mon Mar 8 23:10:43 2004 Received: from optimus.ietf.org (optimus.ietf.org [132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id XAA19920 for ; Mon, 8 Mar 2004 23:10:43 -0500 (EST) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B0YZN-0006yQ-B2 for icar-archive@odin.ietf.org; Mon, 08 Mar 2004 23:10:17 -0500 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i294AHZt026805 for icar-archive@odin.ietf.org; Mon, 8 Mar 2004 23:10:17 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B0YZM-0006yG-Ri for icar-web-archive@optimus.ietf.org; Mon, 08 Mar 2004 23:10:16 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id XAA19910 for ; Mon, 8 Mar 2004 23:10:11 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1B0YZI-0007UU-00 for icar-web-archive@ietf.org; Mon, 08 Mar 2004 23:10:12 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1B0YYC-0007E7-00 for icar-web-archive@ietf.org; Mon, 08 Mar 2004 23:09:06 -0500 Received: from optimus.ietf.org ([132.151.1.19]) by ietf-mx with esmtp (Exim 4.12) id 1B0YX7-00073s-00 for icar-web-archive@ietf.org; Mon, 08 Mar 2004 23:07:57 -0500 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B0YXA-0006jB-85; Mon, 08 Mar 2004 23:08:00 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B0YWW-0006YC-G2 for icar@optimus.ietf.org; Mon, 08 Mar 2004 23:07:20 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id XAA19838 for ; Mon, 8 Mar 2004 23:07:15 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1B0YWR-00072J-00 for icar@ietf.org; Mon, 08 Mar 2004 23:07:15 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1B0YVM-0006tM-00 for icar@ietf.org; Mon, 08 Mar 2004 23:06:10 -0500 Received: from adsl-68-76-113-50.dsl.bcvloh.ameritech.net ([68.76.113.50] helo=guns.icir.org) by ietf-mx with esmtp (Exim 4.12) id 1B0YUK-0006cU-00 for icar@ietf.org; Mon, 08 Mar 2004 23:05:04 -0500 Received: from lawyers.icir.org (adsl-68-76-113-50.dsl.bcvloh.ameritech.net [68.76.113.50]) by guns.icir.org (Postfix) with ESMTP id 04F7F77AA3D; Mon, 8 Mar 2004 23:04:33 -0500 (EST) Received: from lawyers.icir.org (localhost [127.0.0.1]) by lawyers.icir.org (Postfix) with ESMTP id 175D510F427; Mon, 8 Mar 2004 23:04:24 -0500 (EST) To: "Spencer Dawkins" From: Mark Allman Reply-To: mallman@icir.org Cc: icar@ietf.org Subject: Re: [Icar] independence of reviews; variability In-Reply-To: <035201c40588$c86cc840$0400a8c0@DFNJGL21> Organization: ICSI Center for Internet Research (ICIR) Song-of-the-Day: Moondance MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha1; protocol="application/pgp-signature" Date: Mon, 08 Mar 2004 23:04:23 -0500 Message-Id: <20040309040424.175D510F427@lawyers.icir.org> Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.0 required=5.0 tests=AWL autolearn=no version=2.60 --=-=-= > If we don't think anyone except area directors reads documents any > more, discussion about "reviewers inside/outside the management > structure" might make sense. I had hoped that we would not be starting > down that road. I believe the spirit of the ICAR charter is that we should think more along the lines of *early* review -- which my mind thinks of as "way before the IESG sees a document". I was unclear about whether Dave's comments were about early review or late review or just plain review. But, I am pretty sure the above is about later review. Am I off on the spirit of ICAR? And, if we're scoped on early review does that help the independence issue? allman (w/o co-chair hat) -- Mark Allman -- ICIR -- http://www.icir.org/mallman/ --=-=-= Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.3 (Darwin) iD8DBQFATUJHWyrrWs4yIs4RAi9FAJ4njNl2pwnKMGX4ocXlYNQ3+fug5ACgncew Lb3Ol0Iw/fbFHQoQaPJ5LBY= =Kkh2 -----END PGP SIGNATURE----- --=-=-=-- _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Tue Mar 9 02:20:53 2004 Received: from optimus.ietf.org (optimus.ietf.org [132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id CAA11251 for ; Tue, 9 Mar 2004 02:20:53 -0500 (EST) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B0bXM-0002F8-PW for icar-archive@odin.ietf.org; Tue, 09 Mar 2004 02:20:25 -0500 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i297KOrw008595 for icar-archive@odin.ietf.org; Tue, 9 Mar 2004 02:20:24 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B0bXK-0002EY-MK for icar-web-archive@optimus.ietf.org; Tue, 09 Mar 2004 02:20:22 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id CAA11213 for ; Tue, 9 Mar 2004 02:20:19 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1B0bXH-0000EV-00 for icar-web-archive@ietf.org; Tue, 09 Mar 2004 02:20:19 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1B0bWR-00005w-00 for icar-web-archive@ietf.org; Tue, 09 Mar 2004 02:19:27 -0500 Received: from optimus.ietf.org ([132.151.1.19]) by ietf-mx with esmtp (Exim 4.12) id 1B0bW0-0007kM-00 for icar-web-archive@ietf.org; Tue, 09 Mar 2004 02:19:00 -0500 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B0bW1-0001hp-LL; Tue, 09 Mar 2004 02:19:01 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B0bVN-0001Tf-PS for icar@optimus.ietf.org; Tue, 09 Mar 2004 02:18:21 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id CAA11075 for ; Tue, 9 Mar 2004 02:18:19 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1B0bVK-0007js-00 for icar@ietf.org; Tue, 09 Mar 2004 02:18:18 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1B0bUP-0007aq-00 for icar@ietf.org; Tue, 09 Mar 2004 02:17:21 -0500 Received: from joy.songbird.com ([208.184.79.7]) by ietf-mx with esmtp (Exim 4.12) id 1B0bTp-0007QY-00 for icar@ietf.org; Tue, 09 Mar 2004 02:16:45 -0500 Received: from bbprime (jay.songbird.com [208.184.79.253]) by joy.songbird.com (8.11.6/8.11.6) with ESMTP id i297PFd07254; Mon, 8 Mar 2004 23:25:15 -0800 Date: Mon, 8 Mar 2004 23:16:14 -0800 From: Dave Crocker Reply-To: Dave Crocker Organization: Brandenburg InternetWorking X-Priority: 3 (Normal) Message-ID: <44347062.20040308231614@brandenburg.com> To: Mark Allman CC: icar@ietf.org Subject: Re: [Icar] independence of reviews; variability In-Reply-To: <20040309042849.2E00310F4AE@lawyers.icir.org> References: <17710675617.20040308201158@brandenburg.com> <20040309042849.2E00310F4AE@lawyers.icir.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Content-Transfer-Encoding: 7bit Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.7 required=5.0 tests=AWL,PRIORITY_NO_NAME autolearn=no version=2.60 Content-Transfer-Encoding: 7bit Content-Transfer-Encoding: 7bit Mark, MA> One would really hope that swaying an area director would be roughly the MA> same as swaying the community. ... MA> Of course, I am sure that maybe some ADs have hot-button issues that do MA> not enjoy IETF consensus. Since you are asking me a direct question: Having independent reviews is in line with a community, collaborative process. It makes the primary force of decision-making come from an aggregation of comments, and an aggregation of reactions, rather than from any one person's views or preferences (or even one particular group's.) Yes, that might be relevant to the peculiarities of one AD or another, but it is equally relevant to anyone else's peculiarities. d/ -- Dave Crocker Brandenburg InternetWorking Sunnyvale, CA USA _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Tue Mar 9 10:49:56 2004 Received: from optimus.ietf.org (optimus.ietf.org [132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id KAA03804 for ; Tue, 9 Mar 2004 10:49:56 -0500 (EST) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B0jU1-00023f-Ve for icar-archive@odin.ietf.org; Tue, 09 Mar 2004 10:49:30 -0500 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i29FnTin007906 for icar-archive@odin.ietf.org; Tue, 9 Mar 2004 10:49:29 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B0jU1-00023R-NO for icar-web-archive@optimus.ietf.org; Tue, 09 Mar 2004 10:49:29 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id KAA03651 for ; Tue, 9 Mar 2004 10:49:25 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1B0jTy-0004p4-00 for icar-web-archive@ietf.org; Tue, 09 Mar 2004 10:49:26 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1B0jPk-0003W6-00 for icar-web-archive@ietf.org; Tue, 09 Mar 2004 10:45:05 -0500 Received: from optimus.ietf.org ([132.151.1.19]) by ietf-mx with esmtp (Exim 4.12) id 1B0jMf-0002et-00 for icar-web-archive@ietf.org; Tue, 09 Mar 2004 10:41:53 -0500 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B0jHm-0004BD-Q1; Tue, 09 Mar 2004 10:36:50 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B0hZB-0003mN-Cv for icar@optimus.ietf.org; Tue, 09 Mar 2004 08:46:41 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id IAA27198 for ; Tue, 9 Mar 2004 08:46:38 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1B0hZ9-00068v-00 for icar@ietf.org; Tue, 09 Mar 2004 08:46:39 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1B0hYF-0005zC-00 for icar@ietf.org; Tue, 09 Mar 2004 08:45:43 -0500 Received: from adsl-68-76-113-50.dsl.bcvloh.ameritech.net ([68.76.113.50] helo=guns.icir.org) by ietf-mx with esmtp (Exim 4.12) id 1B0hXi-0005oS-00 for icar@ietf.org; Tue, 09 Mar 2004 08:45:10 -0500 Received: from guns.icir.org (localhost [127.0.0.1]) by guns.icir.org (Postfix) with ESMTP id CACC477A6D5; Tue, 9 Mar 2004 08:44:38 -0500 (EST) To: icar@ietf.org Cc: joel@stevecrocker.com From: Mark Allman Reply-To: mallman@icir.org Organization: ICSI Center for Internet Research (ICIR) Song-of-the-Day: Wild Horses MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha1; protocol="application/pgp-signature" Date: Tue, 09 Mar 2004 08:44:38 -0500 Message-Id: <20040309134438.CACC477A6D5@guns.icir.org> Subject: [Icar] call for proposals Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.0 required=5.0 tests=AWL autolearn=no version=2.60 --=-=-= Hi folks! Co-chair hat on .... We'd really like to see some concrete review proposals on the table that we can discuss. To help us manage such a discussion, we'd like to request that folks update their I-Ds with their latest thinking or pull out the relavent bits from larger documents and submit them on their own or invent new proposals and submit them to the I-D archives. Such documents will give us a solid basis for a conversation, rather talking in abstract terms (and, requiring context from other mailing lists, etc. to grok what the current state of the proposal is). Please use the naming convention: draft-lastname-icar-proposalname-00.txt. We'd like to see this happen fairly quickly. Also, if you intend to submit something, please send a note to Joel and I right now indicating this, so that we have an idea about what to expect. Thanks a ton! allman -- Mark Allman -- ICIR -- http://www.icir.org/mallman/ --=-=-= Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.3 (FreeBSD) iD8DBQFATcpGWyrrWs4yIs4RAuzqAKCE+sP/2HJOZQ/m587Ux/AikLBkBQCgm/88 9vlhditShIx3Zw+hQgxO7GU= =v00j -----END PGP SIGNATURE----- --=-=-=-- _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Tue Mar 9 13:35:24 2004 Received: from optimus.ietf.org (optimus.ietf.org [132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA14131 for ; Tue, 9 Mar 2004 13:35:24 -0500 (EST) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B0m47-0007uW-UR for icar-archive@odin.ietf.org; Tue, 09 Mar 2004 13:34:56 -0500 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i29IYt34030407 for icar-archive@odin.ietf.org; Tue, 9 Mar 2004 13:34:55 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B0m47-0007uL-QY for icar-web-archive@optimus.ietf.org; Tue, 09 Mar 2004 13:34:55 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA14122 for ; Tue, 9 Mar 2004 13:34:53 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1B0m45-0000q2-00 for icar-web-archive@ietf.org; Tue, 09 Mar 2004 13:34:53 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1B0m3H-0000g5-00 for icar-web-archive@ietf.org; Tue, 09 Mar 2004 13:34:04 -0500 Received: from optimus.ietf.org ([132.151.1.19]) by ietf-mx with esmtp (Exim 4.12) id 1B0m2H-0000VC-00 for icar-web-archive@ietf.org; Tue, 09 Mar 2004 13:33:01 -0500 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B0m2H-0007qJ-C3; Tue, 09 Mar 2004 13:33:01 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B0m23-0007q0-CA for icar@optimus.ietf.org; Tue, 09 Mar 2004 13:32:47 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA14014 for ; Tue, 9 Mar 2004 13:32:45 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1B0m21-0000SK-00 for icar@ietf.org; Tue, 09 Mar 2004 13:32:45 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1B0m18-0000IJ-00 for icar@ietf.org; Tue, 09 Mar 2004 13:31:51 -0500 Received: from eikenes.alvestrand.no ([158.38.152.233]) by ietf-mx with esmtp (Exim 4.12) id 1B0m0J-0007kz-00 for icar@ietf.org; Tue, 09 Mar 2004 13:30:59 -0500 Received: from halvestr-w2k1 (localhost.localdomain [127.0.0.1]) by eikenes.alvestrand.no (Postfix) with ESMTP id AC50A621B8; Tue, 9 Mar 2004 19:30:26 +0100 (CET) Date: Tue, 09 Mar 2004 10:30:09 -0800 From: Harald Tveit Alvestrand To: Spencer Dawkins , icar@ietf.org Subject: Late review management (Re: [Icar] independence of reviews; variability) Message-ID: <227129254.1078828209@localhost> In-Reply-To: <035201c40588$c86cc840$0400a8c0@DFNJGL21> References: <1221060422.20040308164330@brandenburg.com> <035201c40588$c86cc840$0400a8c0@DFNJGL21> X-Mailer: Mulberry/3.1.0 (Win32) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline Content-Transfer-Encoding: 7bit Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.0 required=5.0 tests=AWL autolearn=no version=2.60 Content-Transfer-Encoding: 7bit Content-Transfer-Encoding: 7bit [[[NOTE: This note is ONLY about LATE review. Early review is a different kettle of fish.....]]] Spencer: there are two ways to view reviews, I think: - As input to further work ("early review") - As a check that this is good enough to be declared finished (what I usually refer to as "late review") In the first category, I think getting serious thought applied is more important than consistency - and structures need to reflect that. In the second category, I think consistency is important - and in two directions, not one: - The "floor" - that at least SOME review is carried out in ALL cases - The "ceiling" - that product not be held to an unreasonably high standard, that stuff that is really nits is not declared show-stoppers, and that one show respect for well justified technical choices, even when one fundamentally disagrees with them. (The last one is actually something most IESG members have spent time learning - not what issues to find, but what issues are NOT worth raising, because the time lost is more costly than the value add of the improvement) Adding more structured late reviews is, in a sense, "raising the floor". (In -twolevel, it is also intended to get away from the idea that a single small group has to do *all* the reviews for the IETF. That's nonscalable.) As was raised in the ICAR meeting, if we have multiple groups reviewing documents, consistency (of both "floor" and "ceiling") isn't automatic, and we should make sure we have mechanisms where someone sees if we have consistency, and can take steps to increase consistency if the divergence becomes big enough to be harmful. So - someone has to manage it. If management of consistency of late reviews is completely independent from other management structures, we set ourselves up for another "late surprise" problem - that the standards groups have been told to apply during development is not the standards they will be held to in the final review. So - I think that having *one* management structure for the IETF makes sense. I don't particularly care what the shape of that management structure is, or what it is called - I care that it works. Harald --On 8. mars 2004 21:44 -0600 Spencer Dawkins wrote: > FWIW, my understanding of the current "IETF Last Call" mechanics is > that reviewers today are, in the final analysis, "uncontrolled". A > scathing review correctly pointing out technical flaws is always > admissible at any point in the review cycle, all the way through > publication by the RFC Editor, no matter who wrote it or why. > > If we don't think anyone except area directors reads documents any > more, discussion about "reviewers inside/outside the management > structure" might make sense. I had hoped that we would not be starting > down that road. > > Having a Internet Review Board that is entirely independent of current > management structures works for me. Having an Internet Review Board > that "reports" to the IAB works for me. We may end up with Area Review > Teams, but that is *not* the only reasonable answer. _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Tue Mar 9 14:19:20 2004 Received: from optimus.ietf.org (optimus.ietf.org [132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA15951 for ; Tue, 9 Mar 2004 14:19:20 -0500 (EST) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B0mkf-0002PT-5V for icar-archive@odin.ietf.org; Tue, 09 Mar 2004 14:18:53 -0500 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i29JIrog009257 for icar-archive@odin.ietf.org; Tue, 9 Mar 2004 14:18:53 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B0mkf-0002PE-0q for icar-web-archive@optimus.ietf.org; Tue, 09 Mar 2004 14:18:53 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA15939 for ; Tue, 9 Mar 2004 14:18:50 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1B0mkc-0000vj-00 for icar-web-archive@ietf.org; Tue, 09 Mar 2004 14:18:50 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1B0mjh-0000li-00 for icar-web-archive@ietf.org; Tue, 09 Mar 2004 14:17:54 -0500 Received: from optimus.ietf.org ([132.151.1.19]) by ietf-mx with esmtp (Exim 4.12) id 1B0mio-0000cL-00 for icar-web-archive@ietf.org; Tue, 09 Mar 2004 14:16:58 -0500 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B0miq-0002MT-J2; Tue, 09 Mar 2004 14:17:00 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B0mht-0002KX-R3 for icar@optimus.ietf.org; Tue, 09 Mar 2004 14:16:01 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA15837 for ; Tue, 9 Mar 2004 14:15:59 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1B0mhr-0000S5-00 for icar@ietf.org; Tue, 09 Mar 2004 14:15:59 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1B0mgy-0000Hh-00 for icar@ietf.org; Tue, 09 Mar 2004 14:15:05 -0500 Received: from smtp.exodus.net ([66.35.230.237] helo=smtp02-w.exodus.net) by ietf-mx with esmtp (Exim 4.12) id 1B0mgJ-00004A-00 for icar@ietf.org; Tue, 09 Mar 2004 14:14:23 -0500 Received: from ms101.mail1.com (ms101.mail1.com [209.1.5.174]) by smtp02-w.exodus.net (8.12.8/8.12.8) with ESMTP id i29GMJEv012391 for ; Tue, 9 Mar 2004 10:22:19 -0600 Received: from [192.168.2.2] (unverified [207.31.248.169]) by accounting.espmail.com (Rockliffe SMTPRA 5.2.5) with ESMTP id for ; Tue, 9 Mar 2004 11:13:53 -0800 Mime-Version: 1.0 X-Sender: margaret@thingmagic.com@ms101.mail1.com Message-Id: In-Reply-To: <1221060422.20040308164330@brandenburg.com> References: <1221060422.20040308164330@brandenburg.com> Date: Tue, 9 Mar 2004 14:11:07 -0500 To: icar@ietf.org From: Margaret Wasserman Subject: Re: [Icar] independence of reviews; variability Content-Type: text/plain; charset="us-ascii" ; format="flowed" Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.0 required=5.0 tests=AWL autolearn=no version=2.60 Although Dave and Harald are both using the term "review" in this thread, I think that the they may be talking about two very different things... In my mind, I would call these things review and evaluation. IMO, review is a purely informative activity. The reviewer reads the document and provides information about its quality, clarity, feasibility, etc. A review generally includes both positive and negative feedback. The WG can then use that review to inform their decisions about the document -- whether to accept it as a WG work item, how to modify it over time, whether or not to submit it for publication. The reviewer is not making a decision about the document, but is instead providing information to the WG that can be used to make a decision. I agree with Dave that this activity should be independent of the IESG/IAB leadership and that this can best be accomplished as a widely-distributed grass roots function. There is also another concept that I will call "evaluation". In today's process, there are two evaluations done on every IETF document: an AD evaluation and an IESG evaluation. These evaluations involves reading the document and _deciding_ whether or not it is suitable for publication. In general, the AD and the IESG do not provide reviews (as I used the term above) as part of this evaluation process. If we decide that a document is not suitable for publication in its current form, we do tell the WG why, but we do not attempt (nor, IMO, should we) to provide a complete, balanced review of the document. IMO, both review and evaluation are necessary parts of the document process, but they don't need to be done by the same people, nor are they done at the same time in the process. There are a number of things that I believe that this WG could do to support, encourage and improve reviews within the IETF, including: - Defining a mechanism for how WG chairs/editors can get reviewers assigned to their documents (perhaps building from experience with MIB Doctors, SIRs, etc.) - Setting guidelines for the type and quantity of review that should be obtained at each stage in the WG process (before accepting as WG work item, before sending to WG last call, before submitting for publication, etc.). - Establish review criteria for each stage in the WG process, perhaps including some type of review forms. - Provide input to the EDU team, so that we can educate IETF participants on how to provide useful and productive reviews. This group might also look at how we can make better use of reviews to inform and accelerate the evaluation process. Margaret _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Tue Mar 9 16:23:29 2004 Received: from optimus.ietf.org (optimus.ietf.org [132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA27316 for ; Tue, 9 Mar 2004 16:23:29 -0500 (EST) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B0ogn-0004xv-Nw for icar-archive@odin.ietf.org; Tue, 09 Mar 2004 16:23:01 -0500 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i29LN1YY019081 for icar-archive@odin.ietf.org; Tue, 9 Mar 2004 16:23:01 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B0ogm-0004xe-Ux for icar-web-archive@optimus.ietf.org; Tue, 09 Mar 2004 16:23:00 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA27225 for ; Tue, 9 Mar 2004 16:22:58 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1B0ogl-0001JW-00 for icar-web-archive@ietf.org; Tue, 09 Mar 2004 16:22:59 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1B0odu-0000Sw-00 for icar-web-archive@ietf.org; Tue, 09 Mar 2004 16:20:03 -0500 Received: from [65.246.255.50] (helo=mx2.foretec.com) by ietf-mx with esmtp (Exim 4.12) id 1B0ocO-00002l-03 for icar-web-archive@ietf.org; Tue, 09 Mar 2004 16:18:29 -0500 Received: from optimus.ietf.org ([132.151.1.19]) by mx2.foretec.com with esmtp (Exim 4.24) id 1B0oQQ-0000yX-Sj for icar-web-archive@ietf.org; Tue, 09 Mar 2004 16:06:06 -0500 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B0oQM-00049u-4h; Tue, 09 Mar 2004 16:06:02 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B0oPU-00049M-KH for icar@optimus.ietf.org; Tue, 09 Mar 2004 16:05:08 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA24323 for ; Tue, 9 Mar 2004 16:05:06 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1B0oPT-0004H5-00 for icar@ietf.org; Tue, 09 Mar 2004 16:05:07 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1B0oOV-00042q-00 for icar@ietf.org; Tue, 09 Mar 2004 16:04:07 -0500 Received: from joy.songbird.com ([208.184.79.7]) by ietf-mx with esmtp (Exim 4.12) id 1B0oNa-0003ff-00 for icar@ietf.org; Tue, 09 Mar 2004 16:03:11 -0500 Received: from bbprime (jay.songbird.com [208.184.79.253]) by joy.songbird.com (8.11.6/8.11.6) with ESMTP id i29LBad32229; Tue, 9 Mar 2004 13:11:37 -0800 Date: Tue, 9 Mar 2004 13:02:31 -0800 From: Dave Crocker Organization: Brandenburg InternetWorking X-Priority: 3 (Normal) Message-ID: <31173904.20040309130231@brandenburg.com> To: Margaret Wasserman CC: icar@ietf.org Subject: Re: [Icar] independence of reviews; variability In-Reply-To: References: <1221060422.20040308164330@brandenburg.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Content-Transfer-Encoding: 7bit Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=1.9 required=5.0 tests=AWL,OPT_HEADER, PRIORITY_NO_NAME autolearn=no version=2.60 Content-Transfer-Encoding: 7bit Content-Transfer-Encoding: 7bit Margaret, MW> This group might also look at how we can make better use of reviews MW> to inform and accelerate the evaluation process. Very helpful -- and accurate -- distinction and goal. Thanks. d/ -- Dave Crocker Brandenburg InternetWorking Sunnyvale, CA USA _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Wed Mar 10 16:24:41 2004 Received: from optimus.ietf.org (optimus.ietf.org [132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA07094 for ; Wed, 10 Mar 2004 16:24:41 -0500 (EST) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B1BBU-00006V-QX for icar-archive@odin.ietf.org; Wed, 10 Mar 2004 16:24:13 -0500 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i2ALOCp6000399 for icar-archive@odin.ietf.org; Wed, 10 Mar 2004 16:24:12 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B1BBU-00006M-L5 for icar-web-archive@optimus.ietf.org; Wed, 10 Mar 2004 16:24:12 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA07056 for ; Wed, 10 Mar 2004 16:24:10 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1B1BBS-0005li-00 for icar-web-archive@ietf.org; Wed, 10 Mar 2004 16:24:11 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1B1B9s-0005ID-00 for icar-web-archive@ietf.org; Wed, 10 Mar 2004 16:22:33 -0500 Received: from optimus.ietf.org ([132.151.1.19]) by ietf-mx with esmtp (Exim 4.12) id 1B1B8S-0004y0-00 for icar-web-archive@ietf.org; Wed, 10 Mar 2004 16:21:04 -0500 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B1B8S-00080i-Oh; Wed, 10 Mar 2004 16:21:04 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B1B7V-0007wr-F1 for icar@optimus.ietf.org; Wed, 10 Mar 2004 16:20:05 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA06728 for ; Wed, 10 Mar 2004 16:20:03 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1B1B7T-0004kz-00 for icar@ietf.org; Wed, 10 Mar 2004 16:20:03 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1B1B6O-0004Oh-00 for icar@ietf.org; Wed, 10 Mar 2004 16:18:56 -0500 Received: from joy.songbird.com ([208.184.79.7]) by ietf-mx with esmtp (Exim 4.12) id 1B1B4M-0003j7-00 for icar@ietf.org; Wed, 10 Mar 2004 16:16:50 -0500 Received: from bbprime (jay.songbird.com [208.184.79.253]) by joy.songbird.com (8.11.6/8.11.6) with ESMTP id i2ALPEd26661; Wed, 10 Mar 2004 13:25:14 -0800 Date: Wed, 10 Mar 2004 13:16:12 -0800 From: Dave Crocker Organization: Brandenburg InternetWorking X-Priority: 3 (Normal) Message-ID: <161118984.20040310131612@brandenburg.com> To: Harald Tveit Alvestrand CC: Spencer Dawkins , icar@ietf.org Subject: Re: Late review management (Re: [Icar] independence of reviews; variability) In-Reply-To: <227129254.1078828209@localhost> References: <1221060422.20040308164330@brandenburg.com> <035201c40588$c86cc840$0400a8c0@DFNJGL21> <227129254.1078828209@localhost> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Content-Transfer-Encoding: 7bit Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.7 required=5.0 tests=AWL,PRIORITY_NO_NAME autolearn=no version=2.60 Content-Transfer-Encoding: 7bit Content-Transfer-Encoding: 7bit Harald, HTA> If management of consistency of late reviews is completely independent from HTA> other management structures, we set ourselves up for another "late HTA> surprise" problem - that the standards groups have been told to apply HTA> during development is not the standards they will be held to in the final HTA> review. So - I think that having *one* management structure for the IETF HTA> makes sense. An interesting question is whether this potential disparity is a feature or a bug? The sort of rigid control you are promoting can be its own trap, for failing to detect problems. If a collection of independent reviewers holds an assessment that is markedly different that of one or more ADs, it might indicate a problem with some of the reviewers or with the ADs holding objections. Once again, diversity of perspectives is our friend... that is, if we believe in community rough consensus, rather than hierarchical assertion of authority. The sort of homogeneity that you are espousing could easily encourage production of specifications that are nicely in line with some AD's desires, but not the desires of the community that must adopt and use the specification. d/ -- Dave Crocker Brandenburg InternetWorking Sunnyvale, CA USA _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Wed Mar 10 16:47:26 2004 Received: from optimus.ietf.org (optimus.ietf.org [132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA08688 for ; Wed, 10 Mar 2004 16:47:26 -0500 (EST) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B1BXW-00029U-Gp for icar-archive@odin.ietf.org; Wed, 10 Mar 2004 16:46:58 -0500 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i2ALkwoZ008271 for icar-archive@odin.ietf.org; Wed, 10 Mar 2004 16:46:58 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B1BXW-00029K-CQ for icar-web-archive@optimus.ietf.org; Wed, 10 Mar 2004 16:46:58 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA08655 for ; Wed, 10 Mar 2004 16:46:55 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1B1BXU-0001yW-00 for icar-web-archive@ietf.org; Wed, 10 Mar 2004 16:46:56 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1B1BWX-0001pT-00 for icar-web-archive@ietf.org; Wed, 10 Mar 2004 16:45:58 -0500 Received: from optimus.ietf.org ([132.151.1.19]) by ietf-mx with esmtp (Exim 4.12) id 1B1BVe-0001h0-00 for icar-web-archive@ietf.org; Wed, 10 Mar 2004 16:45:02 -0500 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B1BVf-000204-Cr; Wed, 10 Mar 2004 16:45:03 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B1BVb-0001yR-TD for icar@optimus.ietf.org; Wed, 10 Mar 2004 16:44:59 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA08579 for ; Wed, 10 Mar 2004 16:44:57 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1B1BVZ-0001gB-00 for icar@ietf.org; Wed, 10 Mar 2004 16:44:58 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1B1BUb-0001Wv-00 for icar@ietf.org; Wed, 10 Mar 2004 16:43:58 -0500 Received: from smtp.exodus.net ([66.35.230.236]) by ietf-mx with esmtp (Exim 4.12) id 1B1BTi-0001F5-00 for icar@ietf.org; Wed, 10 Mar 2004 16:43:02 -0500 Received: from ms101.mail1.com (ms101.mail1.com [209.1.5.174]) by smtp.exodus.net (8.12.8/8.12.8) with ESMTP id i2ANKRw3005851 for ; Wed, 10 Mar 2004 15:20:27 -0800 Received: from [192.168.2.2] (unverified [207.31.248.169]) by accounting.espmail.com (Rockliffe SMTPRA 5.2.5) with ESMTP id ; Wed, 10 Mar 2004 13:42:32 -0800 Mime-Version: 1.0 X-Sender: margaret@thingmagic.com@ms101.mail1.com Message-Id: In-Reply-To: <161118984.20040310131612@brandenburg.com> References: <1221060422.20040308164330@brandenburg.com> <035201c40588$c86cc840$0400a8c0@DFNJGL21> <227129254.1078828209@localhost> <161118984.20040310131612@brandenburg.com> Date: Wed, 10 Mar 2004 16:42:28 -0500 To: Dave Crocker , Harald Tveit Alvestrand From: Margaret Wasserman Subject: Re: Late review management (Re: [Icar] independence of reviews; variability) Cc: Spencer Dawkins , icar@ietf.org Content-Type: text/plain; charset="us-ascii" ; format="flowed" Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.0 required=5.0 tests=AWL autolearn=no version=2.60 >Once again, diversity of perspectives is our friend... that is, if we >believe in community rough consensus, rather than hierarchical assertion >of authority. This touches on a thought I've had off-and-on for the past couple of years... The current IETF structures, policies and processes are based on some fundamental assumptions two of which are (1) that making decisions by rough consensus, reached through an open process will achieve good results, and (2) we can select leaders (ADs, WG chairs, etc.) and trust them to fairly solicit, judge and act upon rough community consensus. Sometimes, though, we seem to question those assumption, and we end up adding structures, policies or processes that are intended to protect us from the results of our own consensus-driven decisions. I think that is (and has been) a mistake. While there is no decision making process (hierarchical, democratic, I just get to decide...) that achieves good results all of the time, the consensus-driven process has worked well for the IETF over the years. By failing to trust it, we don't actually move to another effective decision making process, we just break the one that we have. I think that this same line of thinking applies to review... We should put into place the mechanisms, tools, training, etc. to improve the community's capacity to provide quality review. Perhaps we should provide mechanisms that help WGs' find and recruit reviewers. We could even develop some guidelines about what type and quantity of review makes sense at each level. But, ultimately, I think that we should trust that our WGs (and our WG chairs, document editors, etc.) actually _want_ to produce good quality, well-reviewed work. We can improve their ability to do that by giving them better tools, but we won't achieve anything by trying to enforce quality through "better" rules. If we decide, as a community, that we can no longer trust the consensus process and/or the motivations of our ADs, WG chairs or document editors, we have a _much_ bigger problem than can be solved by changing our review processes. Margaret _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Wed Mar 10 18:04:28 2004 Received: from optimus.ietf.org (optimus.ietf.org [132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id SAA11981 for ; Wed, 10 Mar 2004 18:04:28 -0500 (EST) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B1Ck5-0004Rb-8d for icar-archive@odin.ietf.org; Wed, 10 Mar 2004 18:04:01 -0500 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i2AN414G017080 for icar-archive@odin.ietf.org; Wed, 10 Mar 2004 18:04:01 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B1Ck5-0004RJ-2m for icar-web-archive@optimus.ietf.org; Wed, 10 Mar 2004 18:04:01 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id SAA11933 for ; Wed, 10 Mar 2004 18:03:57 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1B1Ck2-0006El-00 for icar-web-archive@ietf.org; Wed, 10 Mar 2004 18:03:58 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1B1Cj6-00065y-00 for icar-web-archive@ietf.org; Wed, 10 Mar 2004 18:03:01 -0500 Received: from optimus.ietf.org ([132.151.1.19]) by ietf-mx with esmtp (Exim 4.12) id 1B1Ci8-0005wx-00 for icar-web-archive@ietf.org; Wed, 10 Mar 2004 18:02:00 -0500 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B1CiA-0002oJ-4o; Wed, 10 Mar 2004 18:02:02 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B1ChB-00027e-GP for icar@optimus.ietf.org; Wed, 10 Mar 2004 18:01:01 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id SAA11848 for ; Wed, 10 Mar 2004 18:00:57 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1B1Ch8-0005nW-00 for icar@ietf.org; Wed, 10 Mar 2004 18:00:58 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1B1CgD-0005f5-00 for icar@ietf.org; Wed, 10 Mar 2004 18:00:02 -0500 Received: from m106.maoz.com ([205.167.76.9]) by ietf-mx with esmtp (Exim 4.12) id 1B1CfP-0005NM-00 for icar@ietf.org; Wed, 10 Mar 2004 17:59:11 -0500 Received: from m106.maoz.com (localhost.localdomain [127.0.0.1]) by m106.maoz.com (8.12.11/8.12.11) with ESMTP id i2AMwdR2018170; Wed, 10 Mar 2004 14:58:39 -0800 Received: (from dmm@localhost) by m106.maoz.com (8.12.11/8.12.10/Submit) id i2AMwd5t018169; Wed, 10 Mar 2004 14:58:39 -0800 X-Authentication-Warning: m106.maoz.com: dmm set sender to dmm@1-4-5.net using -f Date: Wed, 10 Mar 2004 14:58:39 -0800 From: David Meyer To: Margaret Wasserman Cc: Dave Crocker , Harald Tveit Alvestrand , Spencer Dawkins , icar@ietf.org Subject: Re: Late review management (Re: [Icar] independence of reviews; variability) Message-ID: <20040310225839.GA17999@1-4-5.net> References: <1221060422.20040308164330@brandenburg.com> <035201c40588$c86cc840$0400a8c0@DFNJGL21> <227129254.1078828209@localhost> <161118984.20040310131612@brandenburg.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.1i X-public-key: http://www.1-4-5.net/~dmm/public-key.asc X-philosophy: "I just had to let it go" -- John Lennon Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.0 required=5.0 tests=AWL autolearn=no version=2.60 Margaret, >> >Once again, diversity of perspectives is our friend... that is, if we >> >believe in community rough consensus, rather than hierarchical assertion >> >of authority. >> >> This touches on a thought I've had off-and-on for the past couple of >> years... >> >> The current IETF structures, policies and processes are based on some >> fundamental assumptions two of which are (1) that making decisions by >> rough consensus, reached through an open process will achieve good >> results, and (2) we can select leaders (ADs, WG chairs, etc.) and >> trust them to fairly solicit, judge and act upon rough community >> consensus. Sometimes, though, we seem to question those assumption, >> and we end up adding structures, policies or processes that are >> intended to protect us from the results of our own consensus-driven >> decisions. I think that is (and has been) a mistake. I couldn't really parse the last sentence in this paragraph, i.e., what do you think is the mistake, questioning the assumptions or adding structures, policies, or processes? That being said, honestly questioning one's assumptions with an eye toward improvement is never wrong. However, the latter (adding structure, etc) clearly can. So maybe its not really so much about the questioning (that much is healthy for our or any organization); maybe it is more about what we do in few those cases (if any) that we might find in which our common assumptions don't hold. In the case of the IETF, it is pretty clear that there are at most a few cases in which our consensus based approach is less efficient than we might like; it clearly works well in the vast majority of cases (nuff said on that one; the record speaks for itself). >> While there is no decision making process (hierarchical, democratic, >> I just get to decide...) that achieves good results all of the time, >> the consensus-driven process has worked well for the IETF over the >> years. By failing to trust it, we don't actually move to another >> effective decision making process, we just break the one that we have. Maybe there is be something more (less?) granular here, like trusting the "Rough consensus and running code" isn't a binary thing? That is, it works extremely well for us most of the time, however, we do need to be vigilant and watch out for those few instances in which it doesn't. So maybe it is less about trust, and possibly more about trying to make sure those processes that have been so successful for us continue to serve us well. That goal would seem (minimally) to require continual reevaluation as we, our technology, and our industry evolve. At least that is how I see this issue. >> I think that this same line of thinking applies to review... We >> should put into place the mechanisms, tools, training, etc. to >> improve the community's capacity to provide quality review. Perhaps >> we should provide mechanisms that help WGs' find and recruit >> reviewers. We could even develop some guidelines about what type and >> quantity of review makes sense at each level. But, ultimately, I >> think that we should trust that our WGs (and our WG chairs, document >> editors, etc.) actually _want_ to produce good quality, >> well-reviewed work. We can improve their ability to do that by >> giving them better tools, but we won't achieve anything by trying to >> enforce quality through "better" rules. Completely agree. >> If we decide, as a community, that we can no longer trust the >> consensus process and/or the motivations of our ADs, WG chairs or >> document editors, we have a _much_ bigger problem than can be solved >> by changing our review processes. Again, completely agree. Dave _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Wed Mar 10 18:07:24 2004 Received: from optimus.ietf.org (optimus.ietf.org [132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id SAA12255 for ; Wed, 10 Mar 2004 18:07:24 -0500 (EST) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B1Cmv-0005Bl-Tf for icar-archive@odin.ietf.org; Wed, 10 Mar 2004 18:06:57 -0500 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i2AN6v3S019934 for icar-archive@odin.ietf.org; Wed, 10 Mar 2004 18:06:57 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B1Cmv-0005BP-NV for icar-web-archive@optimus.ietf.org; Wed, 10 Mar 2004 18:06:57 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id SAA12209 for ; Wed, 10 Mar 2004 18:06:54 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1B1Cmt-0006de-00 for icar-web-archive@ietf.org; Wed, 10 Mar 2004 18:06:55 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1B1Clx-0006VT-00 for icar-web-archive@ietf.org; Wed, 10 Mar 2004 18:05:58 -0500 Received: from optimus.ietf.org ([132.151.1.19]) by ietf-mx with esmtp (Exim 4.12) id 1B1Cl1-0006Nb-00 for icar-web-archive@ietf.org; Wed, 10 Mar 2004 18:04:59 -0500 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B1Cl3-0004g9-Cs; Wed, 10 Mar 2004 18:05:01 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B1Ck5-0004Rj-NH for icar@optimus.ietf.org; Wed, 10 Mar 2004 18:04:01 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id SAA11936 for ; Wed, 10 Mar 2004 18:03:58 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1B1Ck3-0006Es-00 for icar@ietf.org; Wed, 10 Mar 2004 18:03:59 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1B1CjA-00066a-00 for icar@ietf.org; Wed, 10 Mar 2004 18:03:05 -0500 Received: from m106.maoz.com ([205.167.76.9]) by ietf-mx with esmtp (Exim 4.12) id 1B1CiF-0005p3-00 for icar@ietf.org; Wed, 10 Mar 2004 18:02:08 -0500 Received: from m106.maoz.com (localhost.localdomain [127.0.0.1]) by m106.maoz.com (8.12.11/8.12.11) with ESMTP id i2AN1afh018285; Wed, 10 Mar 2004 15:01:36 -0800 Received: (from dmm@localhost) by m106.maoz.com (8.12.11/8.12.10/Submit) id i2AN1apN018284; Wed, 10 Mar 2004 15:01:36 -0800 X-Authentication-Warning: m106.maoz.com: dmm set sender to dmm@1-4-5.net using -f Date: Wed, 10 Mar 2004 15:01:36 -0800 From: David Meyer To: Margaret Wasserman Cc: Dave Crocker , Harald Tveit Alvestrand , Spencer Dawkins , icar@ietf.org Subject: Re: Late review management (Re: [Icar] independence of reviews; variability) Message-ID: <20040310230136.GA18259@1-4-5.net> References: <1221060422.20040308164330@brandenburg.com> <035201c40588$c86cc840$0400a8c0@DFNJGL21> <227129254.1078828209@localhost> <161118984.20040310131612@brandenburg.com> <20040310225839.GA17999@1-4-5.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20040310225839.GA17999@1-4-5.net> User-Agent: Mutt/1.4.1i X-public-key: http://www.1-4-5.net/~dmm/public-key.asc X-philosophy: "I just had to let it go" -- John Lennon Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.0 required=5.0 tests=AWL autolearn=no version=2.60 On Wed, Mar 10, 2004 at 02:58:39PM -0800, David Meyer wrote: >> Margaret, >> >> >> >Once again, diversity of perspectives is our friend... that is, if we >> >> >believe in community rough consensus, rather than hierarchical assertion >> >> >of authority. >> >> >> >> This touches on a thought I've had off-and-on for the past couple of >> >> years... >> >> >> >> The current IETF structures, policies and processes are based on some >> >> fundamental assumptions two of which are (1) that making decisions by >> >> rough consensus, reached through an open process will achieve good >> >> results, and (2) we can select leaders (ADs, WG chairs, etc.) and >> >> trust them to fairly solicit, judge and act upon rough community >> >> consensus. Sometimes, though, we seem to question those assumption, >> >> and we end up adding structures, policies or processes that are >> >> intended to protect us from the results of our own consensus-driven >> >> decisions. I think that is (and has been) a mistake. >> >> I couldn't really parse the last sentence in this >> paragraph, i.e., what do you think is the mistake, >> questioning the assumptions or adding structures, >> policies, or processes? >> >> That being said, honestly questioning one's assumptions >> with an eye toward improvement is never wrong. However, >> the latter (adding structure, etc) clearly can. So maybe s/clearly can./clearly can be. Sorry about that. Dave >> its not really so much about the questioning (that much >> is healthy for our or any organization); maybe it is more >> about what we do in few those cases (if any) that we >> might find in which our common assumptions don't hold. In >> the case of the IETF, it is pretty clear that there are >> at most a few cases in which our consensus based approach >> is less efficient than we might like; it clearly works >> well in the vast majority of cases (nuff said on that >> one; the record speaks for itself). >> >> >> While there is no decision making process (hierarchical, democratic, >> >> I just get to decide...) that achieves good results all of the time, >> >> the consensus-driven process has worked well for the IETF over the >> >> years. By failing to trust it, we don't actually move to another >> >> effective decision making process, we just break the one that we have. >> >> Maybe there is be something more (less?) granular here, >> like trusting the "Rough consensus and running code" >> isn't a binary thing? That is, it works extremely well >> for us most of the time, however, we do need to be >> vigilant and watch out for those few instances in which >> it doesn't. So maybe it is less about trust, and possibly >> more about trying to make sure those processes that have >> been so successful for us continue to serve us well. That >> goal would seem (minimally) to require continual >> reevaluation as we, our technology, and our industry >> evolve. At least that is how I see this issue. >> >> >> I think that this same line of thinking applies to review... We >> >> should put into place the mechanisms, tools, training, etc. to >> >> improve the community's capacity to provide quality review. Perhaps >> >> we should provide mechanisms that help WGs' find and recruit >> >> reviewers. We could even develop some guidelines about what type and >> >> quantity of review makes sense at each level. But, ultimately, I >> >> think that we should trust that our WGs (and our WG chairs, document >> >> editors, etc.) actually _want_ to produce good quality, >> >> well-reviewed work. We can improve their ability to do that by >> >> giving them better tools, but we won't achieve anything by trying to >> >> enforce quality through "better" rules. >> >> Completely agree. >> >> >> If we decide, as a community, that we can no longer trust the >> >> consensus process and/or the motivations of our ADs, WG chairs or >> >> document editors, we have a _much_ bigger problem than can be solved >> >> by changing our review processes. >> >> Again, completely agree. >> >> Dave _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Wed Mar 10 18:59:26 2004 Received: from optimus.ietf.org (optimus.ietf.org [132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id SAA14965 for ; Wed, 10 Mar 2004 18:59:26 -0500 (EST) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B1DbI-0003rF-20 for icar-archive@odin.ietf.org; Wed, 10 Mar 2004 18:59:00 -0500 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i2ANx0XW014828 for icar-archive@odin.ietf.org; Wed, 10 Mar 2004 18:59:00 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B1DbH-0003r5-PO for icar-web-archive@optimus.ietf.org; Wed, 10 Mar 2004 18:58:59 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id SAA14947 for ; Wed, 10 Mar 2004 18:58:55 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1B1DbE-0006TM-00 for icar-web-archive@ietf.org; Wed, 10 Mar 2004 18:58:56 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1B1DaH-0006L2-00 for icar-web-archive@ietf.org; Wed, 10 Mar 2004 18:57:58 -0500 Received: from optimus.ietf.org ([132.151.1.19]) by ietf-mx with esmtp (Exim 4.12) id 1B1DZK-0006Cv-00 for icar-web-archive@ietf.org; Wed, 10 Mar 2004 18:56:58 -0500 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B1DZM-0003kY-Vp; Wed, 10 Mar 2004 18:57:00 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B1DYR-0003jV-Fk for icar@optimus.ietf.org; Wed, 10 Mar 2004 18:56:03 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id SAA14835 for ; Wed, 10 Mar 2004 18:55:59 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1B1DYO-00064o-00 for icar@ietf.org; Wed, 10 Mar 2004 18:56:00 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1B1DXW-0005wV-00 for icar@ietf.org; Wed, 10 Mar 2004 18:55:06 -0500 Received: from eikenes.alvestrand.no ([158.38.152.233]) by ietf-mx with esmtp (Exim 4.12) id 1B1DX6-0005nv-00 for icar@ietf.org; Wed, 10 Mar 2004 18:54:41 -0500 Received: from halvestr-w2k1 (localhost.localdomain [127.0.0.1]) by eikenes.alvestrand.no (Postfix) with ESMTP id D621C6223F; Thu, 11 Mar 2004 00:54:08 +0100 (CET) Date: Wed, 10 Mar 2004 15:53:32 -0800 From: Harald Tveit Alvestrand To: Dave Crocker Cc: icar@ietf.org Subject: Re: Late review management (Re: [Icar] independence of reviews; variability) Message-ID: <332932651.1078934012@localhost> In-Reply-To: <161118984.20040310131612@brandenburg.com> References: <1221060422.20040308164330@brandenburg.com> <035201c40588$c86cc840$0400a8c0@DFNJGL21> <227129254.1078828209@localhost> <161118984.20040310131612@brandenburg.com> X-Mailer: Mulberry/3.1.0 (Win32) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline Content-Transfer-Encoding: 7bit Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.0 required=5.0 tests=AWL autolearn=no version=2.60 Content-Transfer-Encoding: 7bit Content-Transfer-Encoding: 7bit Dave, --On 10. mars 2004 13:16 -0800 Dave Crocker wrote: > An interesting question is whether this potential disparity is a feature > or a bug? The sort of rigid control you are promoting can be its own > trap, for failing to detect problems. > > If a collection of independent reviewers holds an assessment that is > markedly different that of one or more ADs, it might indicate a problem > with some of the reviewers or with the ADs holding objections. you heard it from me: The right way to deal with ADs that are markedly at odds with IETF consensus is to correct them, reeducate them or fire them. Not building baroque structures of multiple incompatible control functions in order to minimize the damage they can do. Nit: I never used the word "rigid". That was not an accident. Harald _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Wed Mar 10 19:32:39 2004 Received: from optimus.ietf.org (optimus.ietf.org [132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id TAA16639 for ; Wed, 10 Mar 2004 19:32:39 -0500 (EST) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B1E7M-0000UD-02 for icar-archive@odin.ietf.org; Wed, 10 Mar 2004 19:32:10 -0500 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i2B0W79H001868 for icar-archive@odin.ietf.org; Wed, 10 Mar 2004 19:32:07 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B1E7L-0000U3-Pd for icar-web-archive@optimus.ietf.org; Wed, 10 Mar 2004 19:32:07 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id TAA16560 for ; Wed, 10 Mar 2004 19:32:06 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1B1E7K-0003z2-00 for icar-web-archive@ietf.org; Wed, 10 Mar 2004 19:32:06 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1B1E6I-0003mN-00 for icar-web-archive@ietf.org; Wed, 10 Mar 2004 19:31:03 -0500 Received: from optimus.ietf.org ([132.151.1.19]) by ietf-mx with esmtp (Exim 4.12) id 1B1E5L-0003c5-00 for icar-web-archive@ietf.org; Wed, 10 Mar 2004 19:30:03 -0500 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B1E5K-0000QV-C9; Wed, 10 Mar 2004 19:30:02 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B1E4S-0000PK-5X for icar@optimus.ietf.org; Wed, 10 Mar 2004 19:29:08 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id TAA16440 for ; Wed, 10 Mar 2004 19:29:07 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1B1E4Q-0003Tg-00 for icar@ietf.org; Wed, 10 Mar 2004 19:29:06 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1B1E3b-0003LT-00 for icar@ietf.org; Wed, 10 Mar 2004 19:28:15 -0500 Received: from smtp.exodus.net ([66.35.230.237] helo=smtp02-w.exodus.net) by ietf-mx with esmtp (Exim 4.12) id 1B1E36-0003Bk-00 for icar@ietf.org; Wed, 10 Mar 2004 19:27:44 -0500 Received: from ms101.mail1.com (ms101.mail1.com [209.1.5.174]) by smtp02-w.exodus.net (8.12.8/8.12.8) with ESMTP id i2ALZbEv017500 for ; Wed, 10 Mar 2004 15:35:37 -0600 Received: from [192.168.2.2] (unverified [207.31.248.169]) by accounting.espmail.com (Rockliffe SMTPRA 5.2.5) with ESMTP id ; Wed, 10 Mar 2004 16:27:13 -0800 Mime-Version: 1.0 X-Sender: margaret@thingmagic.com@ms101.mail1.com Message-Id: In-Reply-To: <20040310225839.GA17999@1-4-5.net> References: <1221060422.20040308164330@brandenburg.com> <035201c40588$c86cc840$0400a8c0@DFNJGL21> <227129254.1078828209@localhost> <161118984.20040310131612@brandenburg.com> <20040310225839.GA17999@1-4-5.net> Date: Wed, 10 Mar 2004 19:26:23 -0500 To: David Meyer From: Margaret Wasserman Subject: Re: Late review management (Re: [Icar] independence of reviews; variability) Cc: Dave Crocker , Harald Tveit Alvestrand , Spencer Dawkins , icar@ietf.org Content-Type: text/plain; charset="us-ascii" ; format="flowed" Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.0 required=5.0 tests=AWL autolearn=no version=2.60 At 2:58 PM -0800 3/10/04, David Meyer wrote: > >> Sometimes, though, we seem to question those assumption, > >> and we end up adding structures, policies or processes that are >>> intended to protect us from the results of our own consensus-driven >>> decisions. I think that is (and has been) a mistake. > > I couldn't really parse the last sentence in this > paragraph, i.e., what do you think is the mistake, > questioning the assumptions or adding structures, > policies, or processes? I meant that I believe it is (and has been) a mistake to add new structures, policies and procedures at the end of the process that are intended to protect us from the less-than-perfect results of the WG consensus process, because I don't believe that is the most effective way to produce good work. It's not that I think we should let small, insular WGs run amok and then blindly publish everything that they produce. However, I think that we would do better to offer community guidance and input during the development of new work (through review, for instance) than to try to block that work at the end. There is really nothing new in what I'm saying... Most modern engineering organizations have reached the point where they know that more bugs can be found and corrected, with less schedule impact, through early review (such as design and code review) than by quality checks at the end of the process (such as traditional post-development QA processes). IMO, we do need some type of evaluation at the end (as a final check to detect and correct failures earlier in the process), but the current situation where a large percentage of the work of the IETF is blocked and modified after the WG believes it is finished, is pathological. IMO, we can't fix this problem at the end (by just publishing the work, for example) we need to fix it at the beginning (in the WGs). An important way to try to fix this problem is to get more community input/review, including review and guidance from IETF leaderhship, earlier in the process. Margaret _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Wed Mar 10 19:36:29 2004 Received: from optimus.ietf.org (optimus.ietf.org [132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id TAA16812 for ; Wed, 10 Mar 2004 19:36:29 -0500 (EST) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B1EB6-0000h9-AH for icar-archive@odin.ietf.org; Wed, 10 Mar 2004 19:36:00 -0500 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i2B0a0fR002665 for icar-archive@odin.ietf.org; Wed, 10 Mar 2004 19:36:00 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B1EB6-0000gu-5L for icar-web-archive@optimus.ietf.org; Wed, 10 Mar 2004 19:36:00 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id TAA16808 for ; Wed, 10 Mar 2004 19:35:59 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1B1EB4-0004b9-00 for icar-web-archive@ietf.org; Wed, 10 Mar 2004 19:35:58 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1B1EA6-0004TD-00 for icar-web-archive@ietf.org; Wed, 10 Mar 2004 19:34:59 -0500 Received: from optimus.ietf.org ([132.151.1.19]) by ietf-mx with esmtp (Exim 4.12) id 1B1E99-0004Km-00 for icar-web-archive@ietf.org; Wed, 10 Mar 2004 19:33:59 -0500 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B1E9A-0000ec-IY; Wed, 10 Mar 2004 19:34:00 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B1E8O-0000Zy-Gn for icar@optimus.ietf.org; Wed, 10 Mar 2004 19:33:12 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id TAA16725 for ; Wed, 10 Mar 2004 19:33:11 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1B1E8M-0004Bz-00 for icar@ietf.org; Wed, 10 Mar 2004 19:33:10 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1B1E7d-00041i-00 for icar@ietf.org; Wed, 10 Mar 2004 19:32:25 -0500 Received: from m106.maoz.com ([205.167.76.9]) by ietf-mx with esmtp (Exim 4.12) id 1B1E6g-0003i6-00 for icar@ietf.org; Wed, 10 Mar 2004 19:31:26 -0500 Received: from m106.maoz.com (localhost.localdomain [127.0.0.1]) by m106.maoz.com (8.12.11/8.12.11) with ESMTP id i2B0UsIA020720; Wed, 10 Mar 2004 16:30:54 -0800 Received: (from dmm@localhost) by m106.maoz.com (8.12.11/8.12.10/Submit) id i2B0UsEl020719; Wed, 10 Mar 2004 16:30:54 -0800 X-Authentication-Warning: m106.maoz.com: dmm set sender to dmm@1-4-5.net using -f Date: Wed, 10 Mar 2004 16:30:54 -0800 From: David Meyer To: Margaret Wasserman Cc: Dave Crocker , Harald Tveit Alvestrand , Spencer Dawkins , icar@ietf.org Subject: Re: Late review management (Re: [Icar] independence of reviews; variability) Message-ID: <20040311003054.GA20694@1-4-5.net> References: <1221060422.20040308164330@brandenburg.com> <035201c40588$c86cc840$0400a8c0@DFNJGL21> <227129254.1078828209@localhost> <161118984.20040310131612@brandenburg.com> <20040310225839.GA17999@1-4-5.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.1i X-public-key: http://www.1-4-5.net/~dmm/public-key.asc X-philosophy: "I just had to let it go" -- John Lennon Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.0 required=5.0 tests=AWL autolearn=no version=2.60 On Wed, Mar 10, 2004 at 07:26:23PM -0500, Margaret Wasserman wrote: >> At 2:58 PM -0800 3/10/04, David Meyer wrote: >> > >> Sometimes, though, we seem to question those assumption, >> > >> and we end up adding structures, policies or processes that are >> >>> intended to protect us from the results of our own consensus-driven >> >>> decisions. I think that is (and has been) a mistake. >> > >> > I couldn't really parse the last sentence in this >> > paragraph, i.e., what do you think is the mistake, >> > questioning the assumptions or adding structures, >> > policies, or processes? >> >> I meant that I believe it is (and has been) a mistake to add new >> structures, policies and procedures at the end of the process that >> are intended to protect us from the less-than-perfect results of the >> WG consensus process, because I don't believe that is the most >> effective way to produce good work. >> >> It's not that I think we should let small, insular WGs run amok and >> then blindly publish everything that they produce. However, I think >> that we would do better to offer community guidance and input during >> the development of new work (through review, for instance) than to >> try to block that work at the end. >> >> There is really nothing new in what I'm saying... Most modern >> engineering organizations have reached the point where they know that >> more bugs can be found and corrected, with less schedule impact, >> through early review (such as design and code review) than by quality >> checks at the end of the process (such as traditional >> post-development QA processes). >> >> IMO, we do need some type of evaluation at the end (as a final check >> to detect and correct failures earlier in the process), but the >> current situation where a large percentage of the work of the IETF is >> blocked and modified after the WG believes it is finished, is >> pathological. IMO, we can't fix this problem at the end (by just >> publishing the work, for example) we need to fix it at the beginning >> (in the WGs). An important way to try to fix this problem is to get >> more community input/review, including review and guidance from IETF >> leaderhship, earlier in the process. I think we're in pretty close agreement. Dave _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Thu Mar 11 10:46:20 2004 Received: from optimus.ietf.org (optimus.ietf.org [132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id KAA07467 for ; Thu, 11 Mar 2004 10:46:20 -0500 (EST) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B1SNd-0004df-KZ for icar-archive@odin.ietf.org; Thu, 11 Mar 2004 10:45:54 -0500 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i2BFjrSa017825 for icar-archive@odin.ietf.org; Thu, 11 Mar 2004 10:45:53 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B1SNc-0004dQ-TV for icar-web-archive@optimus.ietf.org; Thu, 11 Mar 2004 10:45:52 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id KAA07440 for ; Thu, 11 Mar 2004 10:45:49 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1B1SNa-0002mb-00 for icar-web-archive@ietf.org; Thu, 11 Mar 2004 10:45:50 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1B1SMg-0002dL-00 for icar-web-archive@ietf.org; Thu, 11 Mar 2004 10:44:55 -0500 Received: from optimus.ietf.org ([132.151.1.19]) by ietf-mx with esmtp (Exim 4.12) id 1B1SLo-0002Uc-00 for icar-web-archive@ietf.org; Thu, 11 Mar 2004 10:44:00 -0500 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B1SLp-0004ZA-5w; Thu, 11 Mar 2004 10:44:01 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B1SLS-0004WA-FA for icar@optimus.ietf.org; Thu, 11 Mar 2004 10:43:38 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id KAA07320 for ; Thu, 11 Mar 2004 10:43:34 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1B1SLQ-0002Rv-00 for icar@ietf.org; Thu, 11 Mar 2004 10:43:36 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1B1SKS-0002IY-00 for icar@ietf.org; Thu, 11 Mar 2004 10:42:36 -0500 Received: from rtp-iport-1.cisco.com ([64.102.122.148]) by ietf-mx with esmtp (Exim 4.12) id 1B1SJR-000241-00 for icar@ietf.org; Thu, 11 Mar 2004 10:41:33 -0500 Received: from rtp-core-1.cisco.com (64.102.124.12) by rtp-iport-1.cisco.com with ESMTP; 11 Mar 2004 07:46:26 -0800 X-BrightmailFiltered: true Received: from cisco.com (erosen-u10.cisco.com [161.44.70.36]) by rtp-core-1.cisco.com (8.12.10/8.12.6) with ESMTP id i2BFetU7018907; Thu, 11 Mar 2004 10:40:56 -0500 (EST) Message-Id: <200403111540.i2BFetU7018907@rtp-core-1.cisco.com> To: Harald Tveit Alvestrand cc: Dave Crocker , icar@ietf.org Subject: Re: Late review management (Re: [Icar] independence of reviews; variability) In-reply-to: Your message of Wed, 10 Mar 2004 15:53:32 -0800. <332932651.1078934012@localhost> Reply-To: erosen@cisco.com User-Agent: EMH/1.14.1 SEMI/1.14.3 (Ushinoya) FLIM/1.14.3 (=?ISO-8859-4?Q?Unebigory=F2mae?=) APEL/10.3 Emacs/21.3 (sparc-sun-solaris2.8) MULE/5.0 (SAKAKI) MIME-Version: 1.0 (generated by SEMI 1.14.3 - "Ushinoya") Content-Type: text/plain; charset=US-ASCII Date: Thu, 11 Mar 2004 10:40:55 -0500 From: Eric Rosen Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.0 required=5.0 tests=AWL autolearn=no version=2.60 Harald> The right way to deal with ADs that are markedly at odds with IETF Harald> consensus is to correct them, reeducate them or fire them. This view is hopelessly utopian. There really does need to be something in the process which prevents an AD with an agenda from blocking the output of any WGs which don't share his agenda. One advantage of earlier, wider review is that it may make it politically more difficult for an AD to carry out his own political agenda. The trick is to find reviewers outside the WG who are experts in the WG's subject matter and who don't have agendas of their own. If we can't come to an understanding of how to do this, then ICAR itself may be too utopian. _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Thu Mar 11 11:42:20 2004 Received: from optimus.ietf.org (optimus.ietf.org [132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA10345 for ; Thu, 11 Mar 2004 11:42:19 -0500 (EST) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B1TFn-0000aV-BU for icar-archive@odin.ietf.org; Thu, 11 Mar 2004 11:41:52 -0500 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i2BGfpP0002252 for icar-archive@odin.ietf.org; Thu, 11 Mar 2004 11:41:51 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B1TFl-0000aF-Py for icar-web-archive@optimus.ietf.org; Thu, 11 Mar 2004 11:41:49 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA10334 for ; Thu, 11 Mar 2004 11:41:47 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1B1TFk-0003nK-00 for icar-web-archive@ietf.org; Thu, 11 Mar 2004 11:41:48 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1B1TEw-0003eu-00 for icar-web-archive@ietf.org; Thu, 11 Mar 2004 11:40:59 -0500 Received: from optimus.ietf.org ([132.151.1.19]) by ietf-mx with esmtp (Exim 4.12) id 1B1TE1-0003W3-00 for icar-web-archive@ietf.org; Thu, 11 Mar 2004 11:40:01 -0500 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B1TE0-0000Qf-Q7; Thu, 11 Mar 2004 11:40:00 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B1TDu-0000Q8-VS for icar@optimus.ietf.org; Thu, 11 Mar 2004 11:39:55 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA10255 for ; Thu, 11 Mar 2004 11:39:52 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1B1TDt-0003Ug-00 for icar@ietf.org; Thu, 11 Mar 2004 11:39:53 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1B1TCw-0003Lp-00 for icar@ietf.org; Thu, 11 Mar 2004 11:38:55 -0500 Received: from sj-iport-5.cisco.com ([171.68.10.87]) by ietf-mx with esmtp (Exim 4.12) id 1B1TCa-0003DK-00 for icar@ietf.org; Thu, 11 Mar 2004 11:38:32 -0500 Received: from sj-core-3.cisco.com (171.68.223.137) by sj-iport-5.cisco.com with ESMTP; 11 Mar 2004 08:38:32 -0800 Received: from cisco.com (sjc-vpn3-744.cisco.com [10.21.66.232]) by sj-core-3.cisco.com (8.12.10/8.12.6) with SMTP id i2BGbwWK022608; Thu, 11 Mar 2004 08:37:58 -0800 (PST) Received: by cisco.com (sSMTP sendmail emulation); Thu, 11 Mar 2004 11:37:53 -0500 Date: Thu, 11 Mar 2004 11:37:42 -0500 From: Scott W Brim To: Margaret Wasserman Cc: Dave Crocker , Harald Tveit Alvestrand , Spencer Dawkins , icar@ietf.org Subject: Re: Late review management (Re: [Icar] independence of reviews; variability) Message-ID: <20040311163741.GE2460@sbrim-w2k01> References: <1221060422.20040308164330@brandenburg.com> <035201c40588$c86cc840$0400a8c0@DFNJGL21> <227129254.1078828209@localhost> <161118984.20040310131612@brandenburg.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.1i Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.0 required=5.0 tests=none autolearn=no version=2.60 On Wed, Mar 10, 2004 04:42:28PM -0500, Margaret Wasserman allegedly wrote: > While there is no decision making process (hierarchical, democratic, > I just get to decide...) that achieves good results all of the time, > the consensus-driven process has worked well for the IETF over the > years. By failing to trust it, we don't actually move to another > effective decision making process, we just break the one that we have. This whole exercise was triggered because enough people felt that frequency of mistakes in the process had reached a threshold and needed to be dealt with. As the Internet has become more important, the level of mistakes that it tolerable has shrunk to close to zero. Also the level of experience of the participants has decreased. I agree in principle, that we want to keep our basic processes, and I trust the basic process in principle, but we must make some changes to ensure quality. We want to add to it, e.g. a few review processes, preferably in parallel with other work so they don't introduce delay. > I think that this same line of thinking applies to review... We > should put into place the mechanisms, tools, training, etc. to > improve the community's capacity to provide quality review. Perhaps > we should provide mechanisms that help WGs' find and recruit > reviewers. We could even develop some guidelines about what type and > quantity of review makes sense at each level. But, ultimately, I > think that we should trust that our WGs (and our WG chairs, document > editors, etc.) actually _want_ to produce good quality, > well-reviewed work. We can improve their ability to do that by > giving them better tools, but we won't achieve anything by trying to > enforce quality through "better" rules. I don't doubt that WGs want to produce good work. It's not a question of motivation, but of capability (WGs and ADs both, and toss in the IAB members as well). We need to bring more minds to bear on decisions -- that is, review them more -- because the stakes are higher and our ability is not. Conceptually we are not giving the WGs better tools, we are giving the IETF better tools. swb _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Thu Mar 11 13:52:21 2004 Received: from optimus.ietf.org (optimus.ietf.org [132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA16220 for ; Thu, 11 Mar 2004 13:52:21 -0500 (EST) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B1VHc-00067j-PQ for icar-archive@odin.ietf.org; Thu, 11 Mar 2004 13:51:53 -0500 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i2BIpqw4023529 for icar-archive@odin.ietf.org; Thu, 11 Mar 2004 13:51:52 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B1VHc-00067O-Kx for icar-web-archive@optimus.ietf.org; Thu, 11 Mar 2004 13:51:52 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA16185 for ; Thu, 11 Mar 2004 13:51:50 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1B1VHa-00074e-00 for icar-web-archive@ietf.org; Thu, 11 Mar 2004 13:51:50 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1B1VGi-0006wh-00 for icar-web-archive@ietf.org; Thu, 11 Mar 2004 13:50:56 -0500 Received: from optimus.ietf.org ([132.151.1.19]) by ietf-mx with esmtp (Exim 4.12) id 1B1VFo-0006og-00 for icar-web-archive@ietf.org; Thu, 11 Mar 2004 13:50:00 -0500 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B1VFq-0005y3-7A; Thu, 11 Mar 2004 13:50:02 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B1VFY-0005wr-Mv for icar@optimus.ietf.org; Thu, 11 Mar 2004 13:49:44 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA16100 for ; Thu, 11 Mar 2004 13:49:42 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1B1VFW-0006m3-00 for icar@ietf.org; Thu, 11 Mar 2004 13:49:42 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1B1VEY-0006dr-00 for icar@ietf.org; Thu, 11 Mar 2004 13:48:42 -0500 Received: from eikenes.alvestrand.no ([158.38.152.233]) by ietf-mx with esmtp (Exim 4.12) id 1B1VDY-0006Q9-00 for icar@ietf.org; Thu, 11 Mar 2004 13:47:40 -0500 Received: from halvestr-w2k1 (localhost.localdomain [127.0.0.1]) by eikenes.alvestrand.no (Postfix) with ESMTP id B839461BF9; Thu, 11 Mar 2004 19:47:09 +0100 (CET) Date: Thu, 11 Mar 2004 10:47:08 -0800 From: Harald Tveit Alvestrand To: erosen@cisco.com Cc: icar@ietf.org Subject: Re: Late review management (Re: [Icar] independence of reviews; variability) Message-ID: <400945459.1079002028@[192.168.1.51]> In-Reply-To: <200403111540.i2BFetU7018907@rtp-core-1.cisco.com> References: <200403111540.i2BFetU7018907@rtp-core-1.cisco.com> X-Mailer: Mulberry/3.1.0 (Win32) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline Content-Transfer-Encoding: 7bit Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.0 required=5.0 tests=AWL autolearn=no version=2.60 Content-Transfer-Encoding: 7bit Content-Transfer-Encoding: 7bit --On 11. mars 2004 10:40 -0500 Eric Rosen wrote: > > Harald> The right way to deal with ADs that are markedly at odds with > IETF Harald> consensus is to correct them, reeducate them or fire them. > > This view is hopelessly utopian. > > There really does need to be something in the process which prevents an > AD with an agenda from blocking the output of any WGs which don't > share his agenda. I don't think we disagree. But in the case of the current process, I see resolving a blockage as having 3 elements: - Detecting that this is happening on a specific issue - Resolving the specific issue - If it happens more than a few times, do something about the AD I've sometimes had mail to chair@ietf.org that worried about the first, and have sometimes been able to accomplish the second informally; so far, I haven't explicitly done the third. > One advantage of earlier, wider review is that it may make it > politically more difficult for an AD to carry out his own political > agenda. The trick is to find reviewers outside the WG who are > experts in the WG's subject matter and who don't have agendas of > their own. If we can't come to an understanding of how to do this, > then ICAR itself may be too utopian. I agree. While I believe less in the power of political agendas than you do, I believe that getting more reviewers into the process makes it FAR easier to see the difference between an AD (or any process gatekeeper) raising a concern that is widely shared in the community and raising a concern that has no backing. It is one reason why I believe archiving reviews is an *essential* part of the process. Harald _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Thu Mar 11 19:52:43 2004 Received: from optimus.ietf.org (optimus.ietf.org [132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id TAA07363 for ; Thu, 11 Mar 2004 19:52:42 -0500 (EST) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B1auM-0004G6-2h for icar-archive@odin.ietf.org; Thu, 11 Mar 2004 19:52:14 -0500 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i2C0qEDg016366 for icar-archive@odin.ietf.org; Thu, 11 Mar 2004 19:52:14 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B1auL-0004Ft-QL for icar-web-archive@optimus.ietf.org; Thu, 11 Mar 2004 19:52:13 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id TAA07106 for ; Thu, 11 Mar 2004 19:52:12 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1B1auJ-00040o-00 for icar-web-archive@ietf.org; Thu, 11 Mar 2004 19:52:12 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1B1aq7-0002qH-00 for icar-web-archive@ietf.org; Thu, 11 Mar 2004 19:47:52 -0500 Received: from [65.246.255.50] (helo=mx2.foretec.com) by ietf-mx with esmtp (Exim 4.12) id 1B1aof-0002bM-01 for icar-web-archive@ietf.org; Thu, 11 Mar 2004 19:46:21 -0500 Received: from optimus.ietf.org ([132.151.1.19]) by mx2.foretec.com with esmtp (Exim 4.24) id 1B1adZ-0005SG-6P for icar-web-archive@ietf.org; Thu, 11 Mar 2004 19:34:53 -0500 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B1abT-0001va-E6; Thu, 11 Mar 2004 19:32:43 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B1YoT-00079h-KY for icar@optimus.ietf.org; Thu, 11 Mar 2004 17:38:01 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA00799 for ; Thu, 11 Mar 2004 17:37:57 -0500 (EST) From: avri@acm.org Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1B1YoR-0001CO-00 for icar@ietf.org; Thu, 11 Mar 2004 17:37:59 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1B1Ynd-00015X-00 for icar@ietf.org; Thu, 11 Mar 2004 17:37:10 -0500 Received: from tla.crepundia.net ([194.71.127.149] helo=report.tla-group.com) by ietf-mx with esmtp (Exim 4.12) id 1B1YnD-0000xG-00 for icar@ietf.org; Thu, 11 Mar 2004 17:36:43 -0500 Received: from [127.0.0.1] (report.tla-group.com [194.71.127.149]) by report.tla-group.com (8.12.4/8.12.4) with ESMTP id i2BMipep002021 for ; Thu, 11 Mar 2004 23:44:54 +0100 Mime-Version: 1.0 (Apple Message framework v612) In-Reply-To: <200403111540.i2BFetU7018907@rtp-core-1.cisco.com> References: <200403111540.i2BFetU7018907@rtp-core-1.cisco.com> Content-Type: text/plain; charset=US-ASCII; format=flowed Message-Id: <8FB31AF3-73AC-11D8-B275-000393CC2112@acm.org> Content-Transfer-Encoding: 7bit Subject: Re: Late review management (Re: [Icar] independence of reviews; variability) Date: Fri, 12 Mar 2004 07:36:38 +0900 To: icar@ietf.org X-Mailer: Apple Mail (2.612) Content-Transfer-Encoding: 7bit Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.3 required=5.0 tests=NO_REAL_NAME autolearn=no version=2.60 Content-Transfer-Encoding: 7bit Content-Transfer-Encoding: 7bit On 12 mar 2004, at 00.40, Eric Rosen wrote: > One advantage of earlier, wider review is that it may make it > politically > more difficult for an AD to carry out his own political agenda. I wonder. My hypothesis is that early reviews, especially if they are tightly linked with the with the IESG and AD, will be more likely to point out how to come in line with the AD's agenda. I think it is like the trend we see with standardized testing - all the education tends toward passing the test, not with knowledge per se. a. _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Thu Mar 11 21:55:33 2004 Received: from optimus.ietf.org (optimus.ietf.org [132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id VAA13230 for ; Thu, 11 Mar 2004 21:55:33 -0500 (EST) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B1cpF-0002nN-Iv for icar-archive@odin.ietf.org; Thu, 11 Mar 2004 21:55:06 -0500 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i2C2t5xR010738 for icar-archive@odin.ietf.org; Thu, 11 Mar 2004 21:55:05 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B1cpF-0002n7-9F for icar-web-archive@optimus.ietf.org; Thu, 11 Mar 2004 21:55:05 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id VAA13191 for ; Thu, 11 Mar 2004 21:55:02 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1B1cpC-0005kJ-00 for icar-web-archive@ietf.org; Thu, 11 Mar 2004 21:55:02 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1B1coC-0005aT-00 for icar-web-archive@ietf.org; Thu, 11 Mar 2004 21:54:01 -0500 Received: from optimus.ietf.org ([132.151.1.19]) by ietf-mx with esmtp (Exim 4.12) id 1B1cnD-0005Mq-00 for icar-web-archive@ietf.org; Thu, 11 Mar 2004 21:52:59 -0500 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B1cnF-0002hQ-1K; Thu, 11 Mar 2004 21:53:01 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B1cmY-0002gg-Dy for icar@optimus.ietf.org; Thu, 11 Mar 2004 21:52:18 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id VAA12954 for ; Thu, 11 Mar 2004 21:52:15 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1B1cmV-0005FD-00 for icar@ietf.org; Thu, 11 Mar 2004 21:52:15 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1B1clR-00054k-00 for icar@ietf.org; Thu, 11 Mar 2004 21:51:10 -0500 Received: from joy.songbird.com ([208.184.79.7]) by ietf-mx with esmtp (Exim 4.12) id 1B1ckg-0004pC-00 for icar@ietf.org; Thu, 11 Mar 2004 21:50:22 -0500 Received: from bbprime (jay.songbird.com [208.184.79.253]) by joy.songbird.com (8.11.6/8.11.6) with ESMTP id i2C2wid11472; Thu, 11 Mar 2004 18:58:44 -0800 Date: Thu, 11 Mar 2004 18:49:35 -0800 From: Dave Crocker Reply-To: Dave Crocker Organization: Brandenburg InternetWorking X-Priority: 3 (Normal) Message-ID: <1328606943.20040311184935@brandenburg.com> To: avri@acm.org CC: icar@ietf.org Subject: Re: Late review management (Re: [Icar] independence of reviews; variability) In-Reply-To: <8FB31AF3-73AC-11D8-B275-000393CC2112@acm.org> References: <200403111540.i2BFetU7018907@rtp-core-1.cisco.com> <8FB31AF3-73AC-11D8-B275-000393CC2112@acm.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Content-Transfer-Encoding: 7bit Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.7 required=5.0 tests=AWL,PRIORITY_NO_NAME autolearn=no version=2.60 Content-Transfer-Encoding: 7bit Content-Transfer-Encoding: 7bit avri, aao> I wonder. My hypothesis is that early reviews, especially if they are aao> tightly linked with the with the IESG and AD, will be more likely to aao> point out how to come in line with the AD's agenda. I think cognitive dissonance theory says that a process like this causes everyone to move towards each other... d/ -- Dave Crocker Brandenburg InternetWorking Sunnyvale, CA USA _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Fri Mar 12 10:35:03 2004 Received: from optimus.ietf.org (optimus.ietf.org [132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id KAA28275 for ; Fri, 12 Mar 2004 10:35:03 -0500 (EST) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B1ogG-0005MS-Se for icar-archive@odin.ietf.org; Fri, 12 Mar 2004 10:34:37 -0500 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i2CFYaZA020607 for icar-archive@odin.ietf.org; Fri, 12 Mar 2004 10:34:36 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B1ogF-0005M3-0b for icar-web-archive@optimus.ietf.org; Fri, 12 Mar 2004 10:34:35 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id KAA28180 for ; Fri, 12 Mar 2004 10:34:31 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1B1ogC-0005YX-00 for icar-web-archive@ietf.org; Fri, 12 Mar 2004 10:34:32 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1B1ofJ-0005Ow-00 for icar-web-archive@ietf.org; Fri, 12 Mar 2004 10:33:37 -0500 Received: from optimus.ietf.org ([132.151.1.19]) by ietf-mx with esmtp (Exim 4.12) id 1B1oeh-0005H4-00 for icar-web-archive@ietf.org; Fri, 12 Mar 2004 10:32:59 -0500 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B1oei-00058x-EZ; Fri, 12 Mar 2004 10:33:00 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B1oeH-000582-1I for icar@optimus.ietf.org; Fri, 12 Mar 2004 10:32:33 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id KAA28089 for ; Fri, 12 Mar 2004 10:32:29 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1B1oeE-0005F9-00 for icar@ietf.org; Fri, 12 Mar 2004 10:32:30 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1B1odN-00057X-00 for icar@ietf.org; Fri, 12 Mar 2004 10:31:38 -0500 Received: from rtp-iport-1.cisco.com ([64.102.122.148]) by ietf-mx with esmtp (Exim 4.12) id 1B1ocT-0004sN-00 for icar@ietf.org; Fri, 12 Mar 2004 10:30:41 -0500 Received: from rtp-core-1.cisco.com (64.102.124.12) by rtp-iport-1.cisco.com with ESMTP; 12 Mar 2004 07:35:44 -0800 X-BrightmailFiltered: true Received: from cisco.com (erosen-u10.cisco.com [161.44.70.36]) by rtp-core-1.cisco.com (8.12.10/8.12.6) with ESMTP id i2CFU8U7026102; Fri, 12 Mar 2004 10:30:09 -0500 (EST) Message-Id: <200403121530.i2CFU8U7026102@rtp-core-1.cisco.com> To: avri@acm.org cc: icar@ietf.org Subject: Re: Late review management (Re: [Icar] independence of reviews; variability) In-reply-to: Your message of Fri, 12 Mar 2004 07:36:38 +0900. <8FB31AF3-73AC-11D8-B275-000393CC2112@acm.org> Reply-To: erosen@cisco.com User-Agent: EMH/1.14.1 SEMI/1.14.3 (Ushinoya) FLIM/1.14.3 (=?ISO-8859-4?Q?Unebigory=F2mae?=) APEL/10.3 Emacs/21.3 (sparc-sun-solaris2.8) MULE/5.0 (SAKAKI) MIME-Version: 1.0 (generated by SEMI 1.14.3 - "Ushinoya") Content-Type: text/plain; charset=US-ASCII Date: Fri, 12 Mar 2004 10:30:08 -0500 From: Eric Rosen Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.0 required=5.0 tests=AWL autolearn=no version=2.60 Avri> My hypothesis is that early reviews, especially if they are tightly Avri> linked with the with the IESG and AD, will be more likely to point out Avri> how to come in line with the AD's agenda. Excellent point. I had assumed that the "earlier, wider reviews" would be harder for the ADs to control very closely (for reasons of scale, if nothing else), but perhaps that assumption is wrong. If the effect of early reviews is help special interest groups have more impact on the WGs rather than less, we won't have made much progress. This all comes back to the underlying issue of whether "the general interest" is best served by applying a variety of perspectives (which I think means a variety of people with independent perspectives, and by the marketplace), or whether it is best served by having all the decisions made by a small group of know-it-alls. This is one of the issues on which the problem-statement group failed to agree. _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Fri Mar 12 11:05:03 2004 Received: from optimus.ietf.org (optimus.ietf.org [132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA29821 for ; Fri, 12 Mar 2004 11:05:03 -0500 (EST) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B1p9I-0007XZ-Nd for icar-archive@odin.ietf.org; Fri, 12 Mar 2004 11:04:37 -0500 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i2CG4aWH028984 for icar-archive@odin.ietf.org; Fri, 12 Mar 2004 11:04:36 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B1p9I-0007XP-GJ for icar-web-archive@optimus.ietf.org; Fri, 12 Mar 2004 11:04:36 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA29806 for ; Fri, 12 Mar 2004 11:04:32 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1B1p9F-0001ly-00 for icar-web-archive@ietf.org; Fri, 12 Mar 2004 11:04:33 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1B1p8L-0001fM-00 for icar-web-archive@ietf.org; Fri, 12 Mar 2004 11:03:37 -0500 Received: from optimus.ietf.org ([132.151.1.19]) by ietf-mx with esmtp (Exim 4.12) id 1B1p7j-0001Yg-00 for icar-web-archive@ietf.org; Fri, 12 Mar 2004 11:02:59 -0500 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B1p7l-0007Q8-1G; Fri, 12 Mar 2004 11:03:01 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B1p7X-0007OE-Ud for icar@optimus.ietf.org; Fri, 12 Mar 2004 11:02:47 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA29772 for ; Fri, 12 Mar 2004 11:02:43 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1B1p7U-0001YI-00 for icar@ietf.org; Fri, 12 Mar 2004 11:02:44 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1B1p6W-0001Qc-00 for icar@ietf.org; Fri, 12 Mar 2004 11:01:44 -0500 Received: from f112.brocade.com ([66.243.153.112] helo=blasphemy.brocade.com) by ietf-mx with esmtp (Exim 4.12) id 1B1p6C-0001IF-00 for icar@ietf.org; Fri, 12 Mar 2004 11:01:24 -0500 Received: from hq-ex-11.corp.brocade.com (hq-ex-11 [192.168.38.58]) by blasphemy.brocade.com (Postfix) with ESMTP id AE929141F4; Fri, 12 Mar 2004 08:00:52 -0800 (PST) X-MimeOLE: Produced By Microsoft Exchange V6.0.6487.1 content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="US-ASCII" Content-Transfer-Encoding: quoted-printable Subject: RE: Late review management (Re: [Icar] independence of reviews; variability) Date: Fri, 12 Mar 2004 08:00:52 -0800 Message-ID: <191DA4FAE235D24C9BCC8D50F6B17AB906EAF9@hq-ex-11.corp.brocade.com> Thread-Topic: Late review management (Re: [Icar] independence of reviews; variability) Thread-Index: AcQHmbN7ApSFjhHKRRCDEKgDvoD1rgAsL9UA From: "Robert Snively" To: "Harald Tveit Alvestrand" , Cc: Content-Transfer-Encoding: quoted-printable Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.0 required=5.0 tests=AWL autolearn=no version=2.60 Content-Transfer-Encoding: quoted-printable Content-Transfer-Encoding: quoted-printable I am totally in agreement with Harald's statement below. > While I believe less in the power of political=20 > agendas than you do, I believe that getting more=20 > reviewers into the process makes it FAR=20 > easier to see the difference between an AD (or any process=20 > gatekeeper) raising a concern that is widely shared=20 > in the community and raising a=20 > concern that has no backing. >=20 > It is one reason why I believe archiving reviews is an=20 > *essential* part of the process. >=20 > Harald _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Fri Mar 12 11:31:23 2004 Received: from optimus.ietf.org (optimus.ietf.org [132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA01482 for ; Fri, 12 Mar 2004 11:31:23 -0500 (EST) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B1pYl-00011F-1Y for icar-archive@odin.ietf.org; Fri, 12 Mar 2004 11:30:55 -0500 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i2CGUtur003917 for icar-archive@odin.ietf.org; Fri, 12 Mar 2004 11:30:55 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B1pYk-000116-L9 for icar-web-archive@optimus.ietf.org; Fri, 12 Mar 2004 11:30:54 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA01338 for ; Fri, 12 Mar 2004 11:30:51 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1B1pYj-0005lo-00 for icar-web-archive@ietf.org; Fri, 12 Mar 2004 11:30:53 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1B1pWm-0005HH-00 for icar-web-archive@ietf.org; Fri, 12 Mar 2004 11:28:52 -0500 Received: from [65.246.255.50] (helo=mx2.foretec.com) by ietf-mx with esmtp (Exim 4.12) id 1B1pVa-0004yv-00 for icar-web-archive@ietf.org; Fri, 12 Mar 2004 11:27:38 -0500 Received: from optimus.ietf.org ([132.151.1.19]) by mx2.foretec.com with esmtp (Exim 4.24) id 1B1pLM-0002du-03 for icar-web-archive@ietf.org; Fri, 12 Mar 2004 11:17:04 -0500 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B1pLI-0000PW-6p; Fri, 12 Mar 2004 11:17:00 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B1pKu-0000LQ-Hc for icar@optimus.ietf.org; Fri, 12 Mar 2004 11:16:36 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA00763 for ; Fri, 12 Mar 2004 11:16:34 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1B1pKt-0003oa-00 for icar@ietf.org; Fri, 12 Mar 2004 11:16:35 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1B1pK4-0003hm-00 for icar@ietf.org; Fri, 12 Mar 2004 11:15:45 -0500 Received: from f112.brocade.com ([66.243.153.112] helo=blasphemy.brocade.com) by ietf-mx with esmtp (Exim 4.12) id 1B1pJE-0003U8-00 for icar@ietf.org; Fri, 12 Mar 2004 11:14:53 -0500 Received: from hq-ex-11.corp.brocade.com (hq-ex-11 [192.168.38.58]) by blasphemy.brocade.com (Postfix) with ESMTP id 941B8143C2; Fri, 12 Mar 2004 08:14:21 -0800 (PST) X-MimeOLE: Produced By Microsoft Exchange V6.0.6487.1 content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="US-ASCII" Content-Transfer-Encoding: quoted-printable Subject: RE: Late review management (Re: [Icar] independence of reviews; variability) Date: Fri, 12 Mar 2004 08:14:21 -0800 Message-ID: <191DA4FAE235D24C9BCC8D50F6B17AB9093322@hq-ex-11.corp.brocade.com> Thread-Topic: Late review management (Re: [Icar] independence of reviews; variability) Thread-Index: AcQIR1lyURVRNtI4Tn+AV3WDMcIMyQAA9wtA From: "Robert Snively" To: , Cc: Content-Transfer-Encoding: quoted-printable Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.0 required=5.0 tests=AWL autolearn=no version=2.60 Content-Transfer-Encoding: quoted-printable Content-Transfer-Encoding: quoted-printable Eric, For the reasons that I am in agreement with Harald's statement, I am hoping that I misunderstood yours. I believe that early reviews are important precisely because they allow a broader range of knowledge and experience (and yes, even special interests) to impact the working group's direction. Working groups may be made up of like-minded people sharing a delusion. The exposure of a working group to the understandings of those working in related areas helps the working group to a broader view of possible solutions=20 and may impact their basic direction. Similarly, the review of the working group's product educates the reviewers, perhaps helping them to align their goals with the directions of the working group. I am sorry that the problem-statement group failed to agree that the general interest is best served by applying a variety of perspectives. A broader view will either=20 correct the problems in a working group's product or it will provide further assurance that the working group's product is indeed the correct solution. A small group of specialists in any field cannot "know it all". Bob 408-333-8135 >=20 >=20 > Avri> My hypothesis is that early reviews, especially if=20 > they are tightly > Avri> linked with the with the IESG and AD, will be more=20 > likely to point out > Avri> how to come in line with the AD's agenda.=20 >=20 > Excellent point. I had assumed that the "earlier, wider=20 > reviews" would be > harder for the ADs to control very closely (for reasons of=20 > scale, if nothing > else), but perhaps that assumption is wrong. If the effect=20 > of early reviews > is help special interest groups have more impact on the=20 > WGs rather than > less, we won't have made much progress. >=20 > This all comes back to the underlying issue of whether=20 > "the general > interest" is best served by applying a variety of=20 > perspectives (which I > think means a variety of people with independent=20 > perspectives, and by the > marketplace), or whether it is best served by having all the=20 > decisions made > by a small group of know-it-alls. This is one of the=20 > issues on which the > problem-statement group failed to agree. >=20 >=20 >=20 >=20 > _______________________________________________ > Icar mailing list > Icar@ietf.org > https://www1.ietf.org/mailman/listinfo/icar >=20 >=20 _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Fri Mar 12 11:31:24 2004 Received: from optimus.ietf.org (optimus.ietf.org [132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA01505 for ; Fri, 12 Mar 2004 11:31:24 -0500 (EST) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B1pYm-00011k-If for icar-archive@odin.ietf.org; Fri, 12 Mar 2004 11:30:56 -0500 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i2CGUuoa003941 for icar-archive@odin.ietf.org; Fri, 12 Mar 2004 11:30:56 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B1pYm-00011T-Dp for icar-web-archive@optimus.ietf.org; Fri, 12 Mar 2004 11:30:56 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA01351 for ; Fri, 12 Mar 2004 11:30:53 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1B1pYl-0005m7-00 for icar-web-archive@ietf.org; Fri, 12 Mar 2004 11:30:55 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1B1pWp-0005Hq-00 for icar-web-archive@ietf.org; Fri, 12 Mar 2004 11:28:56 -0500 Received: from [65.246.255.50] (helo=mx2.foretec.com) by ietf-mx with esmtp (Exim 4.12) id 1B1pVb-0004yv-01 for icar-web-archive@ietf.org; Fri, 12 Mar 2004 11:27:40 -0500 Received: from optimus.ietf.org ([132.151.1.19]) by mx2.foretec.com with esmtp (Exim 4.24) id 1B1pKO-0002dU-Rt for icar-web-archive@ietf.org; Fri, 12 Mar 2004 11:16:05 -0500 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B1pKK-0000Ei-8l; Fri, 12 Mar 2004 11:16:00 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B1pK3-0000ER-BX for icar@optimus.ietf.org; Fri, 12 Mar 2004 11:15:43 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA00723 for ; Fri, 12 Mar 2004 11:15:40 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1B1pK2-0003hN-00 for icar@ietf.org; Fri, 12 Mar 2004 11:15:42 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1B1pIy-0003aB-00 for icar@ietf.org; Fri, 12 Mar 2004 11:14:37 -0500 Received: from joy.songbird.com ([208.184.79.7]) by ietf-mx with esmtp (Exim 4.12) id 1B1pI5-0003Fw-00 for icar@ietf.org; Fri, 12 Mar 2004 11:13:41 -0500 Received: from bbprime (jay.songbird.com [208.184.79.253]) by joy.songbird.com (8.11.6/8.11.6) with ESMTP id i2CGM7d32305; Fri, 12 Mar 2004 08:22:07 -0800 Date: Fri, 12 Mar 2004 08:12:56 -0800 From: Dave Crocker Reply-To: Dave Crocker Organization: Brandenburg InternetWorking X-Priority: 3 (Normal) Message-ID: <1428866288.20040312081256@brandenburg.com> To: Eric Rosen CC: avri@acm.org, icar@ietf.org Subject: Re: Late review management (Re: [Icar] independence of reviews; variability) In-Reply-To: <200403121530.i2CFU8U7026102@rtp-core-1.cisco.com> References: Your message of Fri, 12 Mar 2004 07:36:38 +0900. <8FB31AF3-73AC-11D8-B275-000393CC2112@acm.org> <200403121530.i2CFU8U7026102@rtp-core-1.cisco.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Content-Transfer-Encoding: 7bit Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.7 required=5.0 tests=AWL,PRIORITY_NO_NAME autolearn=no version=2.60 Content-Transfer-Encoding: 7bit Content-Transfer-Encoding: 7bit Eric, Avri>> My hypothesis is that early reviews, especially if they are tightly Avri>> linked with the with the IESG and AD, will be more likely to point out Avri>> how to come in line with the AD's agenda. ER> Excellent point. I had assumed that the "earlier, wider reviews" would be ER> harder for the ADs to control very closely (for reasons of scale, if nothing ER> else), but perhaps that assumption is wrong. Early reviews wind up requiring _everyone_ to put their expectations and concerns out on the table sooner. Unless I missed something over the last 15 years, there is nothing that gives ADs a unique insight into the creation of perfect protocols. They are typically senior contributors, and that usually does give them excellent insight, but not unique. (For these discussions, I cannot stress strongly enough just how important that distinction is.) So, the idea that the rest of us all must always march to the desires of particular individuals -- no matter their title -- goes exactly against the grain of rough consensus. By getting everyone's expectations and concerns out on the table early, disparities can be reasonably negotiated openly and carefully. ER> If the effect of early reviews ER> is help special interest groups have more impact on the WGs rather than ER> less, we won't have made much progress. One of the areas in which the IETF has demonstrated a consistent lack of skill is political/psychological gaming. We do best when we do what is straightforward and constructive. Special Interests that involve careful, intelligent folk, will almost always be better at the gaming than the the IETF community. We hold sway over them by doing Good Work. So let's not be too distracted by such concerns. Early reviews are pretty obviously a good thing, for all the reasons folks have been saying. ER> This all comes back to the underlying issue of whether "the general ER> interest" is best served by applying a variety of perspectives (which I ER> think means a variety of people with independent perspectives, and by the ER> marketplace), or whether it is best served by having all the decisions made ER> by a small group of know-it-alls. This is one of the issues on which the ER> problem-statement group failed to agree. The essence of rough consensus is the belief that paying attention to a variety of perspectives ensures that there will, later, be both the ability to use the output to satisfy real-world needs, and the general support for pursuing it. The problem with giving an individual or small group excessive control is the likelihood that they will force an outcome that is not, later, embraced by the folks who are needed to make the work successful (ie, useful.) d/ -- Dave Crocker Brandenburg InternetWorking Sunnyvale, CA USA _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Fri Mar 12 11:51:44 2004 Received: from optimus.ietf.org (optimus.ietf.org [132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA02525 for ; Fri, 12 Mar 2004 11:51:43 -0500 (EST) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B1psS-0005Dd-9w for icar-archive@odin.ietf.org; Fri, 12 Mar 2004 11:51:16 -0500 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i2CGpGXH020055 for icar-archive@odin.ietf.org; Fri, 12 Mar 2004 11:51:16 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B1psS-0005DO-09 for icar-web-archive@optimus.ietf.org; Fri, 12 Mar 2004 11:51:16 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA02435 for ; Fri, 12 Mar 2004 11:51:13 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1B1psQ-0000im-00 for icar-web-archive@ietf.org; Fri, 12 Mar 2004 11:51:14 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1B1prW-0000VO-00 for icar-web-archive@ietf.org; Fri, 12 Mar 2004 11:50:19 -0500 Received: from [65.246.255.50] (helo=mx2.foretec.com) by ietf-mx with esmtp (Exim 4.12) id 1B1pqW-0000Hn-01 for icar-web-archive@ietf.org; Fri, 12 Mar 2004 11:49:16 -0500 Received: from optimus.ietf.org ([132.151.1.19]) by mx2.foretec.com with esmtp (Exim 4.24) id 1B1pcn-0003Hy-GU for icar-web-archive@ietf.org; Fri, 12 Mar 2004 11:35:05 -0500 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B1pcj-0001NV-71; Fri, 12 Mar 2004 11:35:01 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B1pcH-0001ME-GF for icar@optimus.ietf.org; Fri, 12 Mar 2004 11:34:33 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA01804 for ; Fri, 12 Mar 2004 11:34:30 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1B1pcG-0006Tp-00 for icar@ietf.org; Fri, 12 Mar 2004 11:34:32 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1B1pbJ-0006Kz-00 for icar@ietf.org; Fri, 12 Mar 2004 11:33:33 -0500 Received: from rtp-iport-2.cisco.com ([64.102.122.149]) by ietf-mx with esmtp (Exim 4.12) id 1B1paP-00063m-00 for icar@ietf.org; Fri, 12 Mar 2004 11:32:37 -0500 Received: from rtp-core-2.cisco.com (64.102.124.13) by rtp-iport-2.cisco.com with ESMTP; 12 Mar 2004 08:29:37 -0800 X-BrightmailFiltered: true Received: from cisco.com (erosen-u10.cisco.com [161.44.70.36]) by rtp-core-2.cisco.com (8.12.10/8.12.6) with ESMTP id i2CGW3Uk008151; Fri, 12 Mar 2004 11:32:03 -0500 (EST) Message-Id: <200403121632.i2CGW3Uk008151@rtp-core-2.cisco.com> To: "Robert Snively" cc: avri@acm.org, icar@ietf.org Subject: Re: Late review management (Re: [Icar] independence of reviews; variability) In-reply-to: Your message of Fri, 12 Mar 2004 08:14:21 -0800. <191DA4FAE235D24C9BCC8D50F6B17AB9093322@hq-ex-11.corp.brocade.com> Reply-To: erosen@cisco.com User-Agent: EMH/1.14.1 SEMI/1.14.3 (Ushinoya) FLIM/1.14.3 (=?ISO-8859-4?Q?Unebigory=F2mae?=) APEL/10.3 Emacs/21.3 (sparc-sun-solaris2.8) MULE/5.0 (SAKAKI) MIME-Version: 1.0 (generated by SEMI 1.14.3 - "Ushinoya") Content-Type: text/plain; charset=US-ASCII Date: Fri, 12 Mar 2004 11:32:03 -0500 From: Eric Rosen Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.0 required=5.0 tests=AWL autolearn=no version=2.60 Robert> Working groups may be made up of like-minded people sharing a Robert> delusion. Similarly, the reviewers could be like-minded people sharing a delusion, especially if they have been carefully chosen by "the leadership" to be such. Another thing to keep in mind is that the "like-minded people sharing" what seems to be "a delusion" might in fact be the true visionaries, and it might be the "wider community" that is really sharing a delusion. This is a less likely result, but not unheard of. Dave> Early reviews are pretty obviously a good thing I wouldn't disagree with that, but any good thing can be turned into a political tool, especially if the process does not guarantee adequate checks and balances. _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Fri Mar 12 11:51:44 2004 Received: from optimus.ietf.org (optimus.ietf.org [132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA02530 for ; Fri, 12 Mar 2004 11:51:44 -0500 (EST) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B1psS-0005Dv-R5 for icar-archive@odin.ietf.org; Fri, 12 Mar 2004 11:51:16 -0500 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i2CGpGWZ020073 for icar-archive@odin.ietf.org; Fri, 12 Mar 2004 11:51:16 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B1psS-0005Dg-Lh for icar-web-archive@optimus.ietf.org; Fri, 12 Mar 2004 11:51:16 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA02438 for ; Fri, 12 Mar 2004 11:51:13 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1B1psR-0000ir-00 for icar-web-archive@ietf.org; Fri, 12 Mar 2004 11:51:15 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1B1prX-0000VX-00 for icar-web-archive@ietf.org; Fri, 12 Mar 2004 11:50:20 -0500 Received: from [65.246.255.50] (helo=mx2.foretec.com) by ietf-mx with esmtp (Exim 4.12) id 1B1pqW-0000ID-00 for icar-web-archive@ietf.org; Fri, 12 Mar 2004 11:49:16 -0500 Received: from optimus.ietf.org ([132.151.1.19]) by mx2.foretec.com with esmtp (Exim 4.24) id 1B1pco-0003I2-Eu for icar-web-archive@ietf.org; Fri, 12 Mar 2004 11:35:06 -0500 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B1pcm-0001QA-78; Fri, 12 Mar 2004 11:35:04 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B1pcX-0001N0-9I for icar@optimus.ietf.org; Fri, 12 Mar 2004 11:34:49 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA01844 for ; Fri, 12 Mar 2004 11:34:46 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1B1pcW-0006W0-00 for icar@ietf.org; Fri, 12 Mar 2004 11:34:48 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1B1pbZ-0006OB-00 for icar@ietf.org; Fri, 12 Mar 2004 11:33:50 -0500 Received: from eikenes.alvestrand.no ([158.38.152.233]) by ietf-mx with esmtp (Exim 4.12) id 1B1pb9-0006E6-00 for icar@ietf.org; Fri, 12 Mar 2004 11:33:23 -0500 Received: from halvestr-w2k1 (localhost.localdomain [127.0.0.1]) by eikenes.alvestrand.no (Postfix) with ESMTP id 347D261C0A; Fri, 12 Mar 2004 17:32:51 +0100 (CET) Date: Fri, 12 Mar 2004 08:21:40 -0800 From: Harald Tveit Alvestrand To: erosen@cisco.com, avri@acm.org Cc: icar@ietf.org Subject: Re: Late review management (Re: [Icar] independence of reviews; variability) Message-ID: <478614561.1079079700@localhost> In-Reply-To: <200403121530.i2CFU8U7026102@rtp-core-1.cisco.com> References: <200403121530.i2CFU8U7026102@rtp-core-1.cisco.com> X-Mailer: Mulberry/3.1.0 (Win32) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline Content-Transfer-Encoding: 7bit Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.0 required=5.0 tests=AWL autolearn=no version=2.60 Content-Transfer-Encoding: 7bit Content-Transfer-Encoding: 7bit --On 12. mars 2004 10:30 -0500 Eric Rosen wrote: > > Avri> My hypothesis is that early reviews, especially if they are > tightly Avri> linked with the with the IESG and AD, will be more likely > to point out Avri> how to come in line with the AD's agenda. > > Excellent point. I had assumed that the "earlier, wider reviews" would > be harder for the ADs to control very closely (for reasons of scale, if > nothing else), but perhaps that assumption is wrong. If the effect of > early reviews is help special interest groups have more impact on the > WGs rather than less, we won't have made much progress. if early review works, it will certainly give more influence to the people who do the reviews......... > This all comes back to the underlying issue of whether "the > general interest" is best served by applying a variety of > perspectives (which I think means a variety of people with independent > perspectives, and by the marketplace), or whether it is best served by > having all the decisions made by a small group of know-it-alls. This > is one of the issues on which the problem-statement group failed to > agree. I don't know if there is a real conflict of theories here, or one of perspectives - in these discussions, I find that people seem to argue that the process they don't like (whether it's the IETF, the IESG or the WG) is a "small group of know-it-alls", while the process they like (whether it's the multiperspective review of the IESG or the review from a wider audience) is "a variety of people with independent perspectives". Wide review seems to be "motherhood and apple pie" - the question we're debating is, I think, how to achieve it. Harald _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Fri Mar 12 20:52:20 2004 Received: from optimus.ietf.org (optimus.ietf.org [132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id UAA05987 for ; Fri, 12 Mar 2004 20:52:20 -0500 (EST) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B1yJc-0002Jg-2m for icar-archive@odin.ietf.org; Fri, 12 Mar 2004 20:51:52 -0500 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i2D1pqnL008898 for icar-archive@odin.ietf.org; Fri, 12 Mar 2004 20:51:52 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B1yJb-0002JR-Sj for icar-web-archive@optimus.ietf.org; Fri, 12 Mar 2004 20:51:51 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id UAA05961 for ; Fri, 12 Mar 2004 20:51:49 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1B1yJZ-0001zD-00 for icar-web-archive@ietf.org; Fri, 12 Mar 2004 20:51:49 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1B1yIe-0001vl-00 for icar-web-archive@ietf.org; Fri, 12 Mar 2004 20:50:53 -0500 Received: from optimus.ietf.org ([132.151.1.19]) by ietf-mx with esmtp (Exim 4.12) id 1B1yHq-0001rb-00 for icar-web-archive@ietf.org; Fri, 12 Mar 2004 20:50:02 -0500 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B1yHr-00026n-4M; Fri, 12 Mar 2004 20:50:03 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B1yHg-00022F-Qx for icar@optimus.ietf.org; Fri, 12 Mar 2004 20:49:52 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id UAA05815 for ; Fri, 12 Mar 2004 20:49:50 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1B1yHe-0001pp-00 for icar@ietf.org; Fri, 12 Mar 2004 20:49:50 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1B1yGl-0001ly-00 for icar@ietf.org; Fri, 12 Mar 2004 20:48:56 -0500 Received: from rwcrmhc13.comcast.net ([204.127.198.39]) by ietf-mx with esmtp (Exim 4.12) id 1B1yFr-0001g5-00 for icar@ietf.org; Fri, 12 Mar 2004 20:47:59 -0500 Received: from dfnjgl21 (c-24-1-97-129.client.comcast.net[24.1.97.129]) by comcast.net (rwcrmhc13) with SMTP id <2004031301472901500skq5ce> (Authid: sdawkins@comcast.net); Sat, 13 Mar 2004 01:47:29 +0000 Message-ID: <004101c4089d$2797e450$0400a8c0@DFNJGL21> Reply-To: "Spencer Dawkins" From: "Spencer Dawkins" To: References: <191DA4FAE235D24C9BCC8D50F6B17AB9093322@hq-ex-11.corp.brocade.com> Subject: Re: Late review management (Re: [Icar] independence of reviews; variability) Date: Fri, 12 Mar 2004 19:47:33 -0600 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2800.1158 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1165 Content-Transfer-Encoding: 7bit Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.0 required=5.0 tests=AWL autolearn=no version=2.60 Content-Transfer-Encoding: 7bit Content-Transfer-Encoding: 7bit I know this isn't the right list to ask this question, and I even know what the right list is, but ... I'm really confused by this statement: "I am sorry that the problem-statement group failed to agree that the general interest is best served by applying a variety of perspectives." Got a pointer for me? Spencer, with apologies in advance to Joel and Mark... _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Mon Mar 15 15:30:33 2004 Received: from optimus.ietf.org (optimus.ietf.org [132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA22344 for ; Mon, 15 Mar 2004 15:30:33 -0500 (EST) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B2yir-0001pV-67 for icar-archive@odin.ietf.org; Mon, 15 Mar 2004 15:30:05 -0500 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i2FKU5Dc007032 for icar-archive@odin.ietf.org; Mon, 15 Mar 2004 15:30:05 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B2yiq-0001pL-U9 for icar-web-archive@optimus.ietf.org; Mon, 15 Mar 2004 15:30:04 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA22294 for ; Mon, 15 Mar 2004 15:30:02 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1B2yip-0002iv-00 for icar-web-archive@ietf.org; Mon, 15 Mar 2004 15:30:03 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1B2yhu-0002eb-00 for icar-web-archive@ietf.org; Mon, 15 Mar 2004 15:29:09 -0500 Received: from optimus.ietf.org ([132.151.1.19]) by ietf-mx with esmtp (Exim 4.12) id 1B2ygz-0002cF-00 for icar-web-archive@ietf.org; Mon, 15 Mar 2004 15:28:09 -0500 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B2ygs-0001cx-EE; Mon, 15 Mar 2004 15:28:02 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B2yg1-0001Zz-FF for icar@optimus.ietf.org; Mon, 15 Mar 2004 15:27:09 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA22154 for ; Mon, 15 Mar 2004 15:27:07 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1B2yfz-0002XI-00 for icar@ietf.org; Mon, 15 Mar 2004 15:27:07 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1B2yf3-0002Rp-00 for icar@ietf.org; Mon, 15 Mar 2004 15:26:11 -0500 Received: from ns.execdsl.net ([208.184.15.238] helo=EXECDSL.COM) by ietf-mx with esmtp (Exim 4.12) id 1B2yeW-0002PM-00 for icar@ietf.org; Mon, 15 Mar 2004 15:25:36 -0500 Received: from [64.254.114.114] (HELO JLaptop.stevecrocker.com) by EXECDSL.COM (CommuniGate Pro SMTP 3.3) with ESMTP id 6733040 for icar@ietf.org; Mon, 15 Mar 2004 15:25:23 -0500 Message-Id: <5.1.0.14.0.20040315145735.022c45a8@localhost> X-Sender: joel@stevecrocker.com@localhost X-Mailer: QUALCOMM Windows Eudora Version 5.1 Date: Mon, 15 Mar 2004 15:25:06 -0500 To: icar@ietf.org From: "Joel M. Halpern" Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed Subject: [Icar] ICAR WG Draft Meeting Minutes Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.0 required=5.0 tests=AWL,HTML_MESSAGE autolearn=no version=2.60 Here is a draft of the minutes. Please comment ASAP if there a need for correction / clarification, as I need to file these with the secreteriat soon. Thank you, Joel M. Halpern ICAR (Improved Cross Area Review) WG Minutes Chairs: Joel Halpern halpern@mesgisto.com Mark Allman mallman@icir.org with Alex Zinin guest starring for Mark who could not make it. Minutes: April Marine (who apologizes for missing or messing up names) Welcome: Joel: welcome. Co-chair Mark could not make it to this one, so Alex Zinin will help to ensure the chairs listen well. What is icar? - provide a place for discussing cross area review - propose useful experiments in early and late stage review - select recommended BCPs on review techniques We need to do recommendations of how to review early so we get on with trying stuff out. WG email is icar@ietf.org. Please send pointers to drafts the group should consider to that email list. If you have a draft you think the group should consider, please use the naming convention: draft--icar--.txt You do not need approval to use that name, and it is an easy way to get relevant docs. Once the group agrees on approaches to recommend, we will use the normal draft-ietf-icar.... naming convention. At this point, we want to be aware of all docs, all ideas, half-baked or not. The group will have an issues tracker. Agenda: First, existing experience. Reports from current efforts: - SIRS, Crocker - MIB Doctors, Wijnen - Security Area Directorate, Bellovin - Routing Area Directorate, Zinin As they speak, consider: what constitutes good review? how can we help cause early review mechanisms to be used? SIRS - Dave Crocker SIRS is based on an idea of senior IETF folks doing external reviews (reviews external to the WG). Requests for review come from WGs or elsewhere. Contact: graybeards.net/sirs ietf-sirs@yahoogroups.com SIRS did some reviews, very informal but with structure considering limited resources. Don't know if early reviews would have reduced the need for later reviews No directions re what should be in a review and no evaluation of the results of a review. For example, icar is explicitly looking for cross area reviews, but SIRS did not figure out reviews by type. Started hearing rumors very early that SIRS had failed. Getting reviews of the nature we need in the type we need is an evolutionary process. The fact that SIRS got early signups was indicator of interest. The criteria for who should/can review is a tough debate. There was a low number of review requests, but that could be due to rumor of failure. Need a sustained effort. Need a central point for tracking review efforts, accessing reviews and seeing collections of them. Such infratstructure takes time and needs a real tracking system and someone to watch it. Some types of reviews that need to happen: - cross area is one - review at design stage vs. later review MIB Doctors - Bert Wijnen psg.com//~bwijnen/iCarMib59.ppt When Bert first became OPS AD, some MIBS could not even be run, so he decided before approving any MIB document, it would be sanity checked. So started organizing some "MIB doctors" to check syntax and etc. www.ops.ietf.org/mib-doctors.html They advise AD what to do re MIBs and can review generic IETF docs from network management point of view. Also, MIB doctors can advise WGs. Guidelines are available in ID draft-ietf-ops-mib-review-guidelines-02.txt. Now, this review takes place too much at the end of the process, which is problematic. They try to do consistent mib reviews so result doesn't vary depending on which reviewer you get. They send comments to authors and WG mailing list and relevant ADs so that the doctors can follow up re comments or fixes. Focus is on MIB syntax and structure/reuse, not on technology specifically, so need to understand what the doctors do and don't do and what they know/don't know to understand what is and isn't reviewed. Originally, this was just a volunteer effort, which didn't quite work. It's better to ask a specific doctor. to review a specific MIB. Ergo, that is a scaling issue because currently the AD is looking for reviewers personally. Security Directorate - Steve Bellovin (no slides) History of sec directorate: - first met for lunch to discuss issues, not so effective - then mailing list/web page that listed docs and asked for volunteers to read docs--big failure, no one volunteered so Jeff (Schiller, former security AD) gave up. Steve re-started the Directorate and has been asking specific people to review a specific doc. Also tracking who responds with reviews and what the RTT is and using this to edit membership security directorate. It's not perfect - people don't always have the expertise and/or they get overloaded. Russ Housley (co-SEC AD) adds that pointing to a particular person is the key vs. just asking for general help. Since we've been doing that, we've got about 80% of the reviews we were asking for, vs. 5%. Steve and Bert have to go to other WGs, questions for them now? and none Bert-- MIB Doctors haven't done much in early cross-area review. They once started looking at --00 docs and that would be useful except that the doctors use several tools to analyze a MIB, so if you want MIB doctor review early, then the doc version has to be mature enough for the tools to work. Routing Directorate - Alex Zinin The routing directorate has about 20 members today. It was started by the current ADs, Bill and Alex. Members are selected by the ADs and the group's official status is as an advisory body for the ADs. rtg-dir@ietf.org (closed list) The Directorate is mainly used to review docs coming out of WGs to the IESG. Sometimes docs go from IESG to the directorate when ADs need advice. General appeals for help rarely work. Each doc is assigned 2 reviewers, depending on topic and expertise of reviewers. A deadline of usually 1 or 2 weeks is set. Usually comments go to WG list or authors, but sometimes they are discussed in rtg-dir list first or sometimes the AD proxys comments to the WG. ADs comments/feedback on things helps the directorate learn what IESG is looking for. The log comments in tracker so is not lose and can be followed up on. The directorate review usually happens during the AD review period. Recently though, they have ask the chairs to cc the routing directorate on final WG last call, so there is less of a delay. Sometimes Alex forwards the document part of the IESG agenda to the rtg-dir so that folks know what is coming up. This is considered a "late" review, so they ask the directorate reviewers to look for show stoppers, plus ID-nits. They separate technical comments from editorial comments. Presentations done. Next, discussion: What constitues good reivew? first, for early reviews - is the number of reviews significant? - is the range of skill sets among reviewers important? Harald: what you get in a review depends on what you are aiming at. For example, directorates are helping AD make a decision. SIRS is very informal, so what review you get depends and you don't really know if you got what you wanted. If you want a good review: figure out what that means. What is the purpose of the review? How many reviews are needed? Are several different people needed? Is breadth or depth of skill set of reviewers important? Dave Thayler: Review should include both problems and also suggestions for improvement. This is especially important in early review process as it is easier to make large changes at that stage. Ole: early reviews are very important, both due to quality and timing. They could speed up work on some things and save time on rat holes. Also broad reviews are important Cathleen Moriarity: Don't get thrown in the wrong direction --reviewer should fully understand the problem in order to make useful suggestions. Some suggestions could send person in wrong direction. Joel: that raises an interesting question. If we raise the bar too high for reviewers, then we might discourage reviews. But it is important for reviewer to understand what problem is being addressed and the context of problem. Dave T: Bert in jabber re the question of whether the number of reivews is signifant says 2 is better than one. Dave agrees that 2 is significantly better than 1, but after that the advantage drops. Dimitri Papadimitriou: do reviews suggest areas that might come up in future as problems in that space? e.g. technical content might generate further issues? Joel: sounds useful if reviewer can note future implications. They might be things that people working on topic don't see because they don't have larger context that reviews might have. Might be "obvious" that such review is needed to some, but not to the folks working on the topic. Loa Anderson Scope of review comment: if reviewer fully has to understand the problem, you probably limit the number of reviewers to folks who are in close cooperation with folks writing the docs. We might want to have expertise coming in from different areas to understand the scope of what you are doing. e.g. if MPLS protocol asks for security review, they need to understand that the feedback comes back is from someone who knows X vs. Y. Harald: I have a review team experiment that is 1 mo. old. They read a doc and see if it is in good English. We need docs that are technically sound, but we also have responsibility to write docs that are comprehensible--can you figure out the problem that they are trying to solve? Useful feedbacd can come from folks who don't totally understand technology for an area. Anderson: good point Dave Crocker via jabber: in near term focus on quality of issues, what topics should be covered, what sorts of comments are helpful, etc. maybe a BCP? Ole: supports Harald in re language point because that is part of experience needing to be quite broad. e.g. getting comments from folks who tried to understand what it is all about is useful too. We'll have an interesting time writing a doc trying to capture depth vs. breadth in experience. Harald: IESG discovered that when folks start reviewing, they don't understand what to look for, but after awhile it's much more obvious. Reviewers are made not born.. Tove, comment on review team "graybeard" and "sirs" as names seem to exclude some folks :) What group recommends will have a neutral name. It is important not to exclude folks. Greg Danly: Re issue of readability: the first RFC is just like a unix man page! :) hard to write something that is technically correct but also approachable by tech professionals in the field. a fairly wide audience. If we have docs that are easy to read, then that should be part of review process. Alex: one addition, important to think about the role of the review process in the IETF consensus process. We don't want WG to ignore comments just because they already came to a consensus. We need balance of WG ideas and input from other areas. Any comment on late reviews? Do comments also apply to late reviews? Dmitri: early and late reviewers could be different people. Later reviews could be more implementor focused. Alex: early review needs as many eyes as possible. Later review is touchier e.g. if directorate feels something was addressed but the IESG doesn't agree, you have a problem. It's important to have consistency with IESG approval process. AD delegation of review needs to be done well. Greg D: I think that there is a conceptual difference between early and late review. Late review could show conceptual problems or changes if no early review was done. If do an early review, then late review can clarify or show cross group consensus. Dura..? nortel Early review should be done in WG for catch syntax, language. Late reviews should involve tech aspects as well as other areas input, Early review w/in wg. Ole: Don't completely agree. The real advantage is to have review outside of the WG. Next agenda: discuss docs that currently talk about early review: SIRS experience allman--problem-wg-revcomm-00 draft-ietf-ietf-ops-mib-review-guidelines draft-carpenter-solutions-sirs-02.txt Joel: How can we identify docs for cross area review? Here "early" means early cross area, not the within WG review. Loa: agreed. If you say we should have folks not involved in WG do early cross area reviews, you really need someone with a particular expertise. He may be in the WG or not, don't exclude people who are in WG. James Polk: Is there a repository, so if i write a draft i can post it and offer for review? and a tracking on progress? Joel: not currently such a repository or even a process. This WG will figure out what needs to happen and what tools are needed. James: I think that a posting process is a good idea, even though there is no way to ensure someone will review a draft. Harald: tools: given our limited resource environment, we'd like to have some experimenting done before committing resources to the final choice. We encourage folks to experiment because experience of 10 people doing 10 reviews using any mechanism to check them is valuable. "genauth" experiment visible on website, mailing list of reviews. I know more about tools needed after having tried a bit than I did before. Jan? what is the relation between early review process and advisors for WG? Joel: good question, and the answer is probably up to WG. need to work with ADs and dirs if we want to tie to their advisors. Russ: getting security advisors for WGs is really hard due to time commitment greater than single review. jabber: dave crocker: we should formulate a guide for reviews and add a Sunday tutorial. BCP would be a separate output from the group. also: sirs web/list suffered from lack of better tools, but had limited resources. Joel: many WG did not ask for SIRS reviews. Why? Did they need better incentives? We need to make sure WG pays attention to feedback. Are there incentives or motivations for encouraging WGs to get early reviews? Could this in some way speed up later iesg review? That could be an incentive. Alex: the more eyes that see the doc, the faster it gets through the IESG. Dave: I strongly suggest the review function be independent of IETF administrative management team. Strickly for technical commentary. Also, getting review requests and a BCP will help motivate WGs. IETF management culture encouraging early reviews. WG should have a doc that is like a technical proposal, more than idea but less tech detail. "the boxes and arrows draft." Harald: if you want to create a body of power and responsibility outside of the IETF management structure, be careful what you ask for. Creating such a body with real power outside of IETF management structure would take longer than other approaches. Dmitri: what about cross body reviews? e.g. ITU. How do we deal with those reviews? can that process be integrated with this groups' recommendations? Alex: We had a comment like that during charter review to "improve coordination with other standards bodies or external standards body review" etc. But we should be careful about that as there are a lot of details that could get in the way of process. I think we should concentrate on our internal review first because we understand those processes much better before do interaction with other SDOs. Joel: good point, that is not within the charter as written, but an important question. Ole: focus on internal issues, but don't forget cross relations. If other SDOs have concerns or someone from there comes, address it. Dave C's response to Harald: what power does a review group have? and what benefit is there in making the review group subject to IETF power? Harald: if review is to make any sense, must address quality, relevance, and timeliness of IETF output. The management of the IETF exists to foster that quality, relevance and timeliness. We can already ignore reviewers? Harald: if you have reviewers that make a difference, they have power. If you have reviews that don't make a difference, they don't have power. We have both kinds. Some docs area already out there: draft-zinin-early-review-00 draft-iesg-alvestrand-twolevel-00 Alex: Area Review Teams doc Motivation for this doc was the same as led to creation of ICAR working group. We want to improve cross area review in order to improve quality of documents and to get that review before docs reach the IESG. Higher document quality should reduce IESG approval time. make reviews consistent with the type of review the IESG does later. This doc assumes IESG still responsible for final approval. ADs are personally responsible to the community for the quality of approved docs. Each Area would have an Area Review Team with a well known mailing list such as -review@ietf.org. The team would include at least ADs and other experts selected by ADs. ADs delegate review to team members. Selection is arbitrary but may include an open call for nominations. Two members assigned to each doc. Other members are asked to review, but not required to. The review is initiated by WG chairs or by AD early in process or automatically started for IETF Last Call. Team member review requires AD approval. The mechanism is an Email request and the AD approves assigns reviewers from ART. Comments go to WG or authors. and WG looks at comments and addresses them as usual. Then as docs are updated, same reviewers review again. During IESG review, the AD can bring up comments that haven't been addressed or override a reviewer's comments. This consistent with personal responsibility ADs have for quality of doc. Starting review during IETF last call means all areas ARTs can look at it if necessary. Documenting review results is important, vital for follow up. WG chairs reponsible for issue tracking and summarizing resolution For individual submissions, either AD or an ART person tracks issues. The model of trust, responsibility and accountability does not change. Delegation of review does not delegate responsibility of content. Motivation and credit for team members important, but not sure how to do. Questions: Loa: if team picked personally by AD, does the change of an AD cause a big change in the review team? New AD will probably look at the members and make changes or add some new people. That has to due with personal responsibility of AD Loa: so review team is appointed by ADs? Alex: yes Dmitri: what if docs have several WG last calls? Alex: some WGs use "last call" to really initiate early review. so WG chairs will use judgement as to when to request cross-area review. Ole: I'm worried about how reviewers are selected. If it is only to off load AD work, then that may be good, but we also miss other views, not just an extended AD. It's important to see diversity of opinions on review teams. Especially if their review is equal to the decision or just input. Alex: yes, there is a balance here. Most Ads probably are interested in diverse opinions. Not sure how to require that, but seems we have the same goal. One idea was to have the ART members selected by nomcom, but that has a bunch of drawbacks. Alex is working from idea that current structure stays the same. Greg: there's a conception that the ART teams might be small personal army for the ADs. However, when someone is asked to review a doc, they are using their own technical judgement, which may or may not align with the AD's. There is no guarantee of outcome. It is based on individual integrity. So even hand selecting the ART does not guaranatee any particular result. But getting folks who are qualified and know the area is the most important thing. Alex: yes, it's a bi-directional process. The comments of reviewers influence AD. Then there is also the process of educating each other. Joel: this is one proposal we can consider for late state formal review Harald: also has a draft. The relevant section in draft very short. Mainly it deals with noting how the IESG works when reviewing docs and that it doesn't have to be those people doing that review. It suggests we put together teams where the collective judgement of that team covers all the aspects that need to be covered for late review. For when a WG thinks that they have covered everything. In the IESG, people tend to learn from each other and adjust to each other's reviews, knowing what the other folks will usually catch. Put together review teams that are cross area from the beginning, vs. having one review team per area. Vertical (Alex) vs. horizontal (Harald) approaches. Harald suggested teams be able to approve documents, but that is probably out of scope for this WG. But it might be a worthwhile direction to experiment in. Dave T: How is it determined which team approves a doc? Harald: Throw dice, except that if a group of docs refers to each other, the same team should cover. Dave: but not all review teams will not be equal in terms of quality or also re direction. if they have the power to approve, one team could say yes and another no to same doc. We need a central management point of ultimate responsibility. The WG can't shop around, might be able to do so with Harald's group. Harald: yeah, valid concern. if we want to have judgement exercised by more than one gourp of people, we'll have to expect and deal with differences. We have to have a place in the system that can tell a review team it is out of line, so need a central authority. Greg Danly: we need to look at cross area section. Seems the ART thing has 2 reviewers per area, but this is not as much cross area. missed name: not sure how can come up with a team and know whether or not they have authority to pass docs. By the way, I don't think the team should have authority to pass documents. Review teams should be helpers. Harald: two possible outcomes for teams (3 if you count failure). 1) get docs reviewed and IESG disagrees. Review wasn't thorough enough, in which case the review teams should be advisory. or 2) we run the experiment and see the review team finds all significant problems with the doc. The IESG sees review, the team is happy, the WG deals with all comments, IESG finds nothing more. Then if that is the normal case, why not allow team to approve docs?? Need to run the experiment first. Finally, AD has a question of the chairs: what are the next steps? Joel: - put web site up, - get tracker set up, - get all relevant docs issued with the name convention so they can be reviewed, - then have discussion of those docs against the topics of early and late review as appropriate with the goal of trying to - clarify what experiments can be done soon and - recommend those experiments. - Then start BCP on what review contents and longer review procedures should be. Harald hopes we have the energy to try it to the finish. _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Mon Mar 15 15:58:31 2004 Received: from optimus.ietf.org (optimus.ietf.org [132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA24691 for ; Mon, 15 Mar 2004 15:58:31 -0500 (EST) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B2z9v-0003li-77 for icar-archive@odin.ietf.org; Mon, 15 Mar 2004 15:58:03 -0500 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i2FKw3fq014486 for icar-archive@odin.ietf.org; Mon, 15 Mar 2004 15:58:03 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B2z9v-0003lZ-0E for icar-web-archive@optimus.ietf.org; Mon, 15 Mar 2004 15:58:03 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA24606 for ; Mon, 15 Mar 2004 15:58:00 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1B2z9t-00051v-00 for icar-web-archive@ietf.org; Mon, 15 Mar 2004 15:58:01 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1B2z8v-0004tr-00 for icar-web-archive@ietf.org; Mon, 15 Mar 2004 15:57:01 -0500 Received: from optimus.ietf.org ([132.151.1.19]) by ietf-mx with esmtp (Exim 4.12) id 1B2z7x-0004mx-00 for icar-web-archive@ietf.org; Mon, 15 Mar 2004 15:56:01 -0500 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B2z7y-0003LY-Ee; Mon, 15 Mar 2004 15:56:02 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B2z7L-0003Io-3E for icar@optimus.ietf.org; Mon, 15 Mar 2004 15:55:23 -0500 Received: from CNRI.Reston.VA.US (localhost [127.0.0.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA24125; Mon, 15 Mar 2004 15:55:20 -0500 (EST) Message-Id: <200403152055.PAA24125@ietf.org> Mime-Version: 1.0 Content-Type: Multipart/Mixed; Boundary="NextPart" To: IETF-Announce: ; From: Internet-Drafts@ietf.org Cc: icar@ietf.org Reply-to: Internet-Drafts@ietf.org Date: Mon, 15 Mar 2004 15:55:20 -0500 Subject: [Icar] I-D ACTION:draft-zinin-icar-arts-00.txt Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.5 required=5.0 tests=AWL,MIME_BOUND_NEXTPART, NO_REAL_NAME autolearn=no version=2.60 --NextPart A New Internet-Draft is available from the on-line Internet-Drafts directories. Title : Area Review Teams for Early Cross-functional Reviews Author(s) : A. Zinin Filename : draft-zinin-icar-arts-00.txt Pages : 11 Date : 2004-3-15 This document contains a proposal for cross-functional IETF review process that can be initiated at early stages of a document life cycle. The approach is based on existing experience with area directorates and other expert groups within the IETF. A URL for this Internet-Draft is: http://www.ietf.org/internet-drafts/draft-zinin-icar-arts-00.txt To remove yourself from the IETF Announcement list, send a message to ietf-announce-request with the word unsubscribe in the body of the message. Internet-Drafts are also available by anonymous FTP. Login with the username "anonymous" and a password of your e-mail address. After logging in, type "cd internet-drafts" and then "get draft-zinin-icar-arts-00.txt". A list of Internet-Drafts directories can be found in http://www.ietf.org/shadow.html or ftp://ftp.ietf.org/ietf/1shadow-sites.txt Internet-Drafts can also be obtained by e-mail. Send a message to: mailserv@ietf.org. In the body type: "FILE /internet-drafts/draft-zinin-icar-arts-00.txt". NOTE: The mail server at ietf.org can return the document in MIME-encoded form by using the "mpack" utility. To use this feature, insert the command "ENCODING mime" before the "FILE" command. To decode the response(s), you will need "munpack" or a MIME-compliant mail reader. Different MIME-compliant mail readers exhibit different behavior, especially when dealing with "multipart" MIME messages (i.e. documents which have been split up into multiple messages), so check your local documentation on how to manipulate these messages. Below is the data which will enable a MIME compliant mail reader implementation to automatically retrieve the ASCII version of the Internet-Draft. --NextPart Content-Type: Multipart/Alternative; Boundary="OtherAccess" --OtherAccess Content-Type: Message/External-body; access-type="mail-server"; server="mailserv@ietf.org" Content-Type: text/plain Content-ID: <2004-3-15142630.I-D@ietf.org> ENCODING mime FILE /internet-drafts/draft-zinin-icar-arts-00.txt --OtherAccess Content-Type: Message/External-body; name="draft-zinin-icar-arts-00.txt"; site="ftp.ietf.org"; access-type="anon-ftp"; directory="internet-drafts" Content-Type: text/plain Content-ID: <2004-3-15142630.I-D@ietf.org> --OtherAccess-- --NextPart-- _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Mon Mar 15 16:42:36 2004 Received: from optimus.ietf.org (optimus.ietf.org [132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA29777 for ; Mon, 15 Mar 2004 16:42:36 -0500 (EST) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B2zqb-0006j7-0L for icar-archive@odin.ietf.org; Mon, 15 Mar 2004 16:42:09 -0500 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i2FLg8vQ025851 for icar-archive@odin.ietf.org; Mon, 15 Mar 2004 16:42:08 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B2zqa-0006is-Mj for icar-web-archive@optimus.ietf.org; Mon, 15 Mar 2004 16:42:08 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA29750 for ; Mon, 15 Mar 2004 16:42:05 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1B2zqY-0002TU-00 for icar-web-archive@ietf.org; Mon, 15 Mar 2004 16:42:06 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1B2zpZ-0002MO-00 for icar-web-archive@ietf.org; Mon, 15 Mar 2004 16:41:06 -0500 Received: from optimus.ietf.org ([132.151.1.19]) by ietf-mx with esmtp (Exim 4.12) id 1B2zoa-0002C1-00 for icar-web-archive@ietf.org; Mon, 15 Mar 2004 16:40:04 -0500 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B2zoa-0006ZB-EI; Mon, 15 Mar 2004 16:40:04 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B2znm-0006Ve-LI for icar@optimus.ietf.org; Mon, 15 Mar 2004 16:39:14 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA29419 for ; Mon, 15 Mar 2004 16:39:11 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1B2znk-00022r-00 for icar@ietf.org; Mon, 15 Mar 2004 16:39:12 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1B2zly-0001jn-00 for icar@ietf.org; Mon, 15 Mar 2004 16:37:23 -0500 Received: from netcore.fi ([193.94.160.1]) by ietf-mx with esmtp (Exim 4.12) id 1B2zkt-0001SZ-00 for icar@ietf.org; Mon, 15 Mar 2004 16:36:15 -0500 Received: from localhost (pekkas@localhost) by netcore.fi (8.11.6/8.11.6) with ESMTP id i2FLZfI07730 for ; Mon, 15 Mar 2004 23:35:41 +0200 Date: Mon, 15 Mar 2004 23:35:41 +0200 (EET) From: Pekka Savola To: icar@ietf.org In-Reply-To: <200403152055.PAA24125@ietf.org> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Subject: [Icar] Re: I-D ACTION:draft-zinin-icar-arts-00.txt Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.0 required=5.0 tests=AWL autolearn=no version=2.60 On Mon, 15 Mar 2004 Internet-Drafts@ietf.org wrote: > A New Internet-Draft is available from the on-line Internet-Drafts directories. > > > Title : Area Review Teams for Early Cross-functional Reviews > Author(s) : A. Zinin > Filename : draft-zinin-icar-arts-00.txt > Pages : 11 > Date : 2004-3-15 > > This document contains a proposal for cross-functional IETF review > process that can be initiated at early stages of a document life > cycle. The approach is based on existing experience with area > directorates and other expert groups within the IETF. It seems this is just re-publication of the earlier draft under the icar name.. as was apparently requested in the minutes. I recall sending in some feedback to the previous version, but I guess the document will be revved later on. -- Pekka Savola "You each name yourselves king, yet the Netcore Oy kingdom bleeds." Systems. Networks. Security. -- George R.R. Martin: A Clash of Kings _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Thu Mar 18 14:23:22 2004 Received: from optimus.ietf.org (optimus.ietf.org [132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA01902 for ; Thu, 18 Mar 2004 14:23:21 -0500 (EST) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B436U-000494-K6 for icar-archive@odin.ietf.org; Thu, 18 Mar 2004 14:22:54 -0500 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i2IJMsIq015933 for icar-archive@odin.ietf.org; Thu, 18 Mar 2004 14:22:54 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B436U-00048u-FJ for icar-web-archive@optimus.ietf.org; Thu, 18 Mar 2004 14:22:54 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA01874 for ; Thu, 18 Mar 2004 14:22:51 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1B436S-0004gw-00 for icar-web-archive@ietf.org; Thu, 18 Mar 2004 14:22:52 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1B435V-0004bL-00 for icar-web-archive@ietf.org; Thu, 18 Mar 2004 14:21:54 -0500 Received: from optimus.ietf.org ([132.151.1.19]) by ietf-mx with esmtp (Exim 4.12) id 1B434e-0004Wc-00 for icar-web-archive@ietf.org; Thu, 18 Mar 2004 14:21:00 -0500 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B434g-000441-8V; Thu, 18 Mar 2004 14:21:02 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B434U-0003uX-AW for icar@optimus.ietf.org; Thu, 18 Mar 2004 14:20:50 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA01754 for ; Thu, 18 Mar 2004 14:20:47 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1B434R-0004VE-00 for icar@ietf.org; Thu, 18 Mar 2004 14:20:47 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1B433b-0004Qx-00 for icar@ietf.org; Thu, 18 Mar 2004 14:19:56 -0500 Received: from psg.com ([147.28.0.62] ident=mailnull) by ietf-mx with esmtp (Exim 4.12) id 1B432u-0004M1-00 for icar@ietf.org; Thu, 18 Mar 2004 14:19:12 -0500 Received: from [147.28.0.62] (helo=127.0.0.1) by psg.com with esmtp (Exim 4.30; FreeBSD) id 1B432u-000EfQ-4r; Thu, 18 Mar 2004 19:19:12 +0000 Date: Thu, 18 Mar 2004 11:17:53 -0800 From: Alex Zinin X-Mailer: The Bat! (v1.62i) Personal Reply-To: Alex Zinin X-Priority: 3 (Normal) Message-ID: <10669074413.20040318111753@psg.com> To: Pekka Savola CC: icar@ietf.org Subject: Re: [Icar] Re: I-D ACTION:draft-zinin-icar-arts-00.txt In-Reply-To: References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Content-Transfer-Encoding: 7bit Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.3 required=5.0 tests=AWL,RCVD_NUMERIC_HELO autolearn=no version=2.60 Content-Transfer-Encoding: 7bit Content-Transfer-Encoding: 7bit Pekka, Yes, and yes. Thanks for the reminder! ;) -- Alex http://www.psg.com/~zinin/ Monday, March 15, 2004, 1:35:41 PM, Pekka Savola wrote: > On Mon, 15 Mar 2004 Internet-Drafts@ietf.org wrote: >> A New Internet-Draft is available from the on-line Internet-Drafts directories. >> >> >> Title : Area Review Teams for Early Cross-functional Reviews >> Author(s) : A. Zinin >> Filename : draft-zinin-icar-arts-00.txt >> Pages : 11 >> Date : 2004-3-15 >> >> This document contains a proposal for cross-functional IETF review >> process that can be initiated at early stages of a document life >> cycle. The approach is based on existing experience with area >> directorates and other expert groups within the IETF. > It seems this is just re-publication of the earlier draft under the > icar name.. as was apparently requested in the minutes. I recall > sending in some feedback to the previous version, but I guess the > document will be revved later on. _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Fri Mar 19 14:55:09 2004 Received: from optimus.ietf.org (optimus.ietf.org [132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA18530 for ; Fri, 19 Mar 2004 14:55:09 -0500 (EST) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B4Q4g-0002Jc-Tv for icar-archive@odin.ietf.org; Fri, 19 Mar 2004 14:54:41 -0500 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i2JJsYI6008900 for icar-archive@odin.ietf.org; Fri, 19 Mar 2004 14:54:34 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B4Q4g-0002JT-A3 for icar-web-archive@optimus.ietf.org; Fri, 19 Mar 2004 14:54:34 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA18507 for ; Fri, 19 Mar 2004 14:54:30 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1B4Q4c-0002oB-00 for icar-web-archive@ietf.org; Fri, 19 Mar 2004 14:54:30 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1B4Q3k-0002ie-00 for icar-web-archive@ietf.org; Fri, 19 Mar 2004 14:53:39 -0500 Received: from optimus.ietf.org ([132.151.1.19]) by ietf-mx with esmtp (Exim 4.12) id 1B4Q39-0002cm-00 for icar-web-archive@ietf.org; Fri, 19 Mar 2004 14:52:59 -0500 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B4Q3B-0002EH-3J; Fri, 19 Mar 2004 14:53:01 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B4Q2a-0002Dh-8e for icar@optimus.ietf.org; Fri, 19 Mar 2004 14:52:24 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA18427 for ; Fri, 19 Mar 2004 14:52:21 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1B4Q2X-0002Yi-00 for icar@ietf.org; Fri, 19 Mar 2004 14:52:21 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1B4Q1X-0002RB-00 for icar@ietf.org; Fri, 19 Mar 2004 14:51:21 -0500 Received: from ns.execdsl.net ([208.184.15.238] helo=EXECDSL.COM) by ietf-mx with esmtp (Exim 4.12) id 1B4Q0Y-0002MJ-00; Fri, 19 Mar 2004 14:50:18 -0500 Received: from [64.254.114.114] (HELO JLaptop.stevecrocker.com) by EXECDSL.COM (CommuniGate Pro SMTP 3.3) with ESMTP id 6745209; Fri, 19 Mar 2004 14:50:18 -0500 Message-Id: <5.1.0.14.0.20040319144516.023d73c8@localhost> X-Sender: joel@stevecrocker.com@localhost X-Mailer: QUALCOMM Windows Eudora Version 5.1 Date: Fri, 19 Mar 2004 14:50:09 -0500 To: proceedings@ietf.org From: "Joel M. Halpern" Cc: icar@ietf.org Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed Subject: [Icar] ICAR WG Meeting Minutes from IETF 59 Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.0 required=5.0 tests=AWL,HTML_MESSAGE autolearn=no version=2.60 ICAR (Improved Cross Area Review) WG Minutes Chairs: Joel Halpern jhalpern@megisto.com Mark Allman mallman@icir.org with Alex Zinin guest starring for Mark who could not make it. Minutes: April Marine (who apologizes for missing or messing up names) IETF 59, Seoul, South Korea. March 3, 2004 9:00-11:30 am Welcome: Joel: welcome. Co-chair Mark could not make it to this one, so Alex Zinin will help to ensure the chairs listen well. What is icar? - provide a place for discussing cross area review - propose useful experiments in early and late stage review - select recommended BCPs on review techniques We need to do recommendations of how to review early so we get on with trying stuff out. WG email is icar@ietf.org. Please send pointers to drafts the group should consider to that email list. If you have a draft you think the group should consider, please use the naming convention: draft--icar--.txt You do not need approval to use that name, and it is an easy way to get relevant docs. Once the group agrees on approaches to recommend, we will use the normal draft-ietf-icar.... naming convention. At this point, we want to be aware of all docs, all ideas, half-baked or not. The group will have an issues tracker. Agenda: First, existing experience. Reports from current efforts: - SIRS, Crocker - MIB Doctors, Wijnen - Security Area Directorate, Bellovin - Routing Area Directorate, Zinin As they speak, consider: what constitutes good review? how can we help cause early review mechanisms to be used? SIRS - Dave Crocker SIRS is based on an idea of senior IETF folks doing external reviews (reviews external to the WG). Requests for review come from WGs or elsewhere. Contact: graybeards.net/sirs ietf-sirs@yahoogroups.com SIRS did some reviews, very informal but with structure considering limited resources. Don't know if early reviews would have reduced the need for later reviews No directions re what should be in a review and no evaluation of the results of a review. For example, icar is explicitly looking for cross area reviews, but SIRS did not figure out reviews by type. Started hearing rumors very early that SIRS had failed. Getting reviews of the nature we need in the type we need is an evolutionary process. The fact that SIRS got early signups was indicator of interest. The criteria for who should/can review is a tough debate. There was a low number of review requests, but that could be due to rumor of failure. Need a sustained effort. Need a central point for tracking review efforts, accessing reviews and seeing collections of them. Such infratstructure takes time and needs a real tracking system and someone to watch it. Some types of reviews that need to happen: - cross area is one - review at design stage vs. later review MIB Doctors - Bert Wijnen psg.com//~bwijnen/iCarMib59.ppt When Bert first became OPS AD, some MIBS could not even be run, so he decided before approving any MIB document, it would be sanity checked. So started organizing some "MIB doctors" to check syntax and etc. www.ops.ietf.org/mib-doctors.html They advise AD what to do re MIBs and can review generic IETF docs from network management point of view. Also, MIB doctors can advise WGs. Guidelines are available in ID draft-ietf-ops-mib-review-guidelines-02.txt. Now, this review takes place too much at the end of the process, which is problematic. They try to do consistent mib reviews so result doesn't vary depending on which reviewer you get. They send comments to authors and WG mailing list and relevant ADs so that the doctors can follow up re comments or fixes. Focus is on MIB syntax and structure/reuse, not on technology specifically, so need to understand what the doctors do and don't do and what they know/don't know to understand what is and isn't reviewed. Originally, this was just a volunteer effort, which didn't quite work. It's better to ask a specific doctor. to review a specific MIB. Ergo, that is a scaling issue because currently the AD is looking for reviewers personally. Security Directorate - Steve Bellovin (no slides) History of sec directorate: - first met for lunch to discuss issues, not so effective - then mailing list/web page that listed docs and asked for volunteers to read docs--big failure, no one volunteered so Jeff (Schiller, former security AD) gave up. Steve re-started the Directorate and has been asking specific people to review a specific doc. Also tracking who responds with reviews and what the RTT is and using this to edit membership security directorate. It's not perfect - people don't always have the expertise and/or they get overloaded. Russ Housley (co-SEC AD) adds that pointing to a particular person is the key vs. just asking for general help. Since we've been doing that, we've got about 80% of the reviews we were asking for, vs. 5%. Steve and Bert have to go to other WGs, questions for them now? and none Bert-- MIB Doctors haven't done much in early cross-area review. They once started looking at --00 docs and that would be useful except that the doctors use several tools to analyze a MIB, so if you want MIB doctor review early, then the doc version has to be mature enough for the tools to work. Routing Directorate - Alex Zinin The routing directorate has about 20 members today. It was started by the current ADs, Bill and Alex. Members are selected by the ADs and the group's official status is as an advisory body for the ADs. rtg-dir@ietf.org (closed list) The Directorate is mainly used to review docs coming out of WGs to the IESG. Sometimes docs go from IESG to the directorate when ADs need advice. General appeals for help rarely work. Each doc is assigned 2 reviewers, depending on topic and expertise of reviewers. A deadline of usually 1 or 2 weeks is set. Usually comments go to WG list or authors, but sometimes they are discussed in rtg-dir list first or sometimes the AD proxys comments to the WG. ADs comments/feedback on things helps the directorate learn what IESG is looking for. The log comments in tracker so is not lose and can be followed up on. The directorate review usually happens during the AD review period. Recently though, they have ask the chairs to cc the routing directorate on final WG last call, so there is less of a delay. Sometimes Alex forwards the document part of the IESG agenda to the rtg-dir so that folks know what is coming up. This is considered a "late" review, so they ask the directorate reviewers to look for show stoppers, plus ID-nits. They separate technical comments from editorial comments. Presentations done. Next, discussion: What constitues good reivew? first, for early reviews - is the number of reviews significant? - is the range of skill sets among reviewers important? Harald: what you get in a review depends on what you are aiming at. For example, directorates are helping AD make a decision. SIRS is very informal, so what review you get depends and you don't really know if you got what you wanted. If you want a good review: figure out what that means. What is the purpose of the review? How many reviews are needed? Are several different people needed? Is breadth or depth of skill set of reviewers important? Dave Thayler: Review should include both problems and also suggestions for improvement. This is especially important in early review process as it is easier to make large changes at that stage. Ole: early reviews are very important, both due to quality and timing. They could speed up work on some things and save time on rat holes. Also broad reviews are important Cathleen Moriarity: Don't get thrown in the wrong direction --reviewer should fully understand the problem in order to make useful suggestions. Some suggestions could send person in wrong direction. Joel: that raises an interesting question. If we raise the bar too high for reviewers, then we might discourage reviews. But it is important for reviewer to understand what problem is being addressed and the context of problem. Dave T: Bert in jabber re the question of whether the number of reivews is signifant says 2 is better than one. Dave agrees that 2 is significantly better than 1, but after that the advantage drops. Dimitri Papadimitriou: do reviews suggest areas that might come up in future as problems in that space? e.g. technical content might generate further issues? Joel: sounds useful if reviewer can note future implications. They might be things that people working on topic don't see because they don't have larger context that reviews might have. Might be "obvious" that such review is needed to some, but not to the folks working on the topic. Loa Anderson Scope of review comment: if reviewer fully has to understand the problem, you probably limit the number of reviewers to folks who are in close cooperation with folks writing the docs. We might want to have expertise coming in from different areas to understand the scope of what you are doing. e.g. if MPLS protocol asks for security review, they need to understand that the feedback comes back is from someone who knows X vs. Y. Harald: I have a review team experiment that is 1 mo. old. They read a doc and see if it is in good English. We need docs that are technically sound, but we also have responsibility to write docs that are comprehensible--can you figure out the problem that they are trying to solve? Useful feedbacd can come from folks who don't totally understand technology for an area. Anderson: good point Dave Crocker via jabber: in near term focus on quality of issues, what topics should be covered, what sorts of comments are helpful, etc. maybe a BCP? Ole: supports Harald in re language point because that is part of experience needing to be quite broad. e.g. getting comments from folks who tried to understand what it is all about is useful too. We'll have an interesting time writing a doc trying to capture depth vs. breadth in experience. Harald: IESG discovered that when folks start reviewing, they don't understand what to look for, but after awhile it's much more obvious. Reviewers are made not born.. Tove, comment on review team "graybeard" and "sirs" as names seem to exclude some folks :) What group recommends will have a neutral name. It is important not to exclude folks. Greg Danly: Re issue of readability: the first RFC is just like a unix man page! :) hard to write something that is technically correct but also approachable by tech professionals in the field. a fairly wide audience. If we have docs that are easy to read, then that should be part of review process. Alex: one addition, important to think about the role of the review process in the IETF consensus process. We don't want WG to ignore comments just because they already came to a consensus. We need balance of WG ideas and input from other areas. Any comment on late reviews? Do comments also apply to late reviews? Dmitri: early and late reviewers could be different people. Later reviews could be more implementor focused. Alex: early review needs as many eyes as possible. Later review is touchier e.g. if directorate feels something was addressed but the IESG doesn't agree, you have a problem. It's important to have consistency with IESG approval process. AD delegation of review needs to be done well. Greg D: I think that there is a conceptual difference between early and late review. Late review could show conceptual problems or changes if no early review was done. If do an early review, then late review can clarify or show cross group consensus. Dura..? nortel Early review should be done in WG for catch syntax, language. Late reviews should involve tech aspects as well as other areas input, Early review w/in wg. Ole: Don't completely agree. The real advantage is to have review outside of the WG. Next agenda: discuss docs that currently talk about early review: SIRS experience allman--problem-wg-revcomm-00 draft-ietf-ietf-ops-mib-review-guidelines draft-carpenter-solutions-sirs-02.txt Joel: How can we identify docs for cross area review? Here "early" means early cross area, not the within WG review. Loa: agreed. If you say we should have folks not involved in WG do early cross area reviews, you really need someone with a particular expertise. He may be in the WG or not, don't exclude people who are in WG. James Polk: Is there a repository, so if i write a draft i can post it and offer for review? and a tracking on progress? Joel: not currently such a repository or even a process. This WG will figure out what needs to happen and what tools are needed. James: I think that a posting process is a good idea, even though there is no way to ensure someone will review a draft. Harald: tools: given our limited resource environment, we'd like to have some experimenting done before committing resources to the final choice. We encourage folks to experiment because experience of 10 people doing 10 reviews using any mechanism to check them is valuable. "genauth" experiment visible on website, mailing list of reviews. I know more about tools needed after having tried a bit than I did before. Jan? what is the relation between early review process and advisors for WG? Joel: good question, and the answer is probably up to WG. need to work with ADs and dirs if we want to tie to their advisors. Russ: getting security advisors for WGs is really hard due to time commitment greater than single review. jabber: dave crocker: we should formulate a guide for reviews and add a Sunday tutorial. BCP would be a separate output from the group. also: sirs web/list suffered from lack of better tools, but had limited resources. Joel: many WG did not ask for SIRS reviews. Why? Did they need better incentives? We need to make sure WG pays attention to feedback. Are there incentives or motivations for encouraging WGs to get early reviews? Could this in some way speed up later iesg review? That could be an incentive. Alex: the more eyes that see the doc, the faster it gets through the IESG. Dave: I strongly suggest the review function be independent of IETF administrative management team. Strickly for technical commentary. Also, getting review requests and a BCP will help motivate WGs. IETF management culture encouraging early reviews. WG should have a doc that is like a technical proposal, more than idea but less tech detail. "the boxes and arrows draft." Harald: if you want to create a body of power and responsibility outside of the IETF management structure, be careful what you ask for. Creating such a body with real power outside of IETF management structure would take longer than other approaches. Dmitri: what about cross body reviews? e.g. ITU. How do we deal with those reviews? can that process be integrated with this groups' recommendations? Alex: We had a comment like that during charter review to "improve coordination with other standards bodies or external standards body review" etc. But we should be careful about that as there are a lot of details that could get in the way of process. I think we should concentrate on our internal review first because we understand those processes much better before do interaction with other SDOs. Joel: good point, that is not within the charter as written, but an important question. Ole: focus on internal issues, but don't forget cross relations. If other SDOs have concerns or someone from there comes, address it. Dave C's response to Harald: what power does a review group have? and what benefit is there in making the review group subject to IETF power? Harald: if review is to make any sense, must address quality, relevance, and timeliness of IETF output. The management of the IETF exists to foster that quality, relevance and timeliness. We can already ignore reviewers? Harald: if you have reviewers that make a difference, they have power. If you have reviews that don't make a difference, they don't have power. We have both kinds. Some docs area already out there: draft-zinin-early-review-00 draft-iesg-alvestrand-twolevel-00 Alex: Area Review Teams doc Motivation for this doc was the same as led to creation of ICAR working group. We want to improve cross area review in order to improve quality of documents and to get that review before docs reach the IESG. Higher document quality should reduce IESG approval time. make reviews consistent with the type of review the IESG does later. This doc assumes IESG still responsible for final approval. ADs are personally responsible to the community for the quality of approved docs. Each Area would have an Area Review Team with a well known mailing list such as -review@ietf.org. The team would include at least ADs and other experts selected by ADs. ADs delegate review to team members. Selection is arbitrary but may include an open call for nominations. Two members assigned to each doc. Other members are asked to review, but not required to. The review is initiated by WG chairs or by AD early in process or automatically started for IETF Last Call. Team member review requires AD approval. The mechanism is an Email request and the AD approves assigns reviewers from ART. Comments go to WG or authors. and WG looks at comments and addresses them as usual. Then as docs are updated, same reviewers review again. During IESG review, the AD can bring up comments that haven't been addressed or override a reviewer's comments. This consistent with personal responsibility ADs have for quality of doc. Starting review during IETF last call means all areas ARTs can look at it if necessary. Documenting review results is important, vital for follow up. WG chairs reponsible for issue tracking and summarizing resolution For individual submissions, either AD or an ART person tracks issues. The model of trust, responsibility and accountability does not change. Delegation of review does not delegate responsibility of content. Motivation and credit for team members important, but not sure how to do. Questions: Loa: if team picked personally by AD, does the change of an AD cause a big change in the review team? New AD will probably look at the members and make changes or add some new people. That has to due with personal responsibility of AD Loa: so review team is appointed by ADs? Alex: yes Dmitri: what if docs have several WG last calls? Alex: some WGs use "last call" to really initiate early review. so WG chairs will use judgement as to when to request cross-area review. Ole: I'm worried about how reviewers are selected. If it is only to off load AD work, then that may be good, but we also miss other views, not just an extended AD. It's important to see diversity of opinions on review teams. Especially if their review is equal to the decision or just input. Alex: yes, there is a balance here. Most Ads probably are interested in diverse opinions. Not sure how to require that, but seems we have the same goal. One idea was to have the ART members selected by nomcom, but that has a bunch of drawbacks. Alex is working from idea that current structure stays the same. Greg: there's a conception that the ART teams might be small personal army for the ADs. However, when someone is asked to review a doc, they are using their own technical judgement, which may or may not align with the AD's. There is no guarantee of outcome. It is based on individual integrity. So even hand selecting the ART does not guaranatee any particular result. But getting folks who are qualified and know the area is the most important thing. Alex: yes, it's a bi-directional process. The comments of reviewers influence AD. Then there is also the process of educating each other. Joel: this is one proposal we can consider for late state formal review Harald: also has a draft. The relevant section in draft very short. Mainly it deals with noting how the IESG works when reviewing docs and that it doesn't have to be those people doing that review. It suggests we put together teams where the collective judgement of that team covers all the aspects that need to be covered for late review. For when a WG thinks that they have covered everything. In the IESG, people tend to learn from each other and adjust to each other's reviews, knowing what the other folks will usually catch. Put together review teams that are cross area from the beginning, vs. having one review team per area. Vertical (Alex) vs. horizontal (Harald) approaches. Harald suggested teams be able to approve documents, but that is probably out of scope for this WG. But it might be a worthwhile direction to experiment in. Dave T: How is it determined which team approves a doc? Harald: Throw dice, except that if a group of docs refers to each other, the same team should cover. Dave: but not all review teams will not be equal in terms of quality or also re direction. if they have the power to approve, one team could say yes and another no to same doc. We need a central management point of ultimate responsibility. The WG can't shop around, might be able to do so with Harald's group. Harald: yeah, valid concern. if we want to have judgement exercised by more than one gourp of people, we'll have to expect and deal with differences. We have to have a place in the system that can tell a review team it is out of line, so need a central authority. Greg Danly: we need to look at cross area section. Seems the ART thing has 2 reviewers per area, but this is not as much cross area. missed name: not sure how can come up with a team and know whether or not they have authority to pass docs. By the way, I don't think the team should have authority to pass documents. Review teams should be helpers. Harald: two possible outcomes for teams (3 if you count failure). 1) get docs reviewed and IESG disagrees. Review wasn't thorough enough, in which case the review teams should be advisory. or 2) we run the experiment and see the review team finds all significant problems with the doc. The IESG sees review, the team is happy, the WG deals with all comments, IESG finds nothing more. Then if that is the normal case, why not allow team to approve docs?? Need to run the experiment first. Finally, AD has a question of the chairs: what are the next steps? Joel: - put web site up, - get tracker set up, - get all relevant docs issued with the name convention so they can be reviewed, - then have discussion of those docs against the topics of early and late review as appropriate with the goal of trying to - clarify what experiments can be done soon and - recommend those experiments. - Then start BCP on what review contents and longer review procedures should be. Harald hopes we have the energy to try it to the finish. _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Fri Mar 26 05:08:24 2004 Received: from optimus.ietf.org (optimus.ietf.org [132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id FAA10675 for ; Fri, 26 Mar 2004 05:08:24 -0500 (EST) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B6oFo-0000yf-Gm for icar-archive@odin.ietf.org; Fri, 26 Mar 2004 05:07:56 -0500 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i2QA7ug4003748 for icar-archive@odin.ietf.org; Fri, 26 Mar 2004 05:07:56 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B6oFn-0000yN-Un for icar-web-archive@optimus.ietf.org; Fri, 26 Mar 2004 05:07:56 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id FAA10650 for ; Fri, 26 Mar 2004 05:07:52 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1B6oFk-00068q-00 for icar-web-archive@ietf.org; Fri, 26 Mar 2004 05:07:52 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1B6oEo-00063K-00 for icar-web-archive@ietf.org; Fri, 26 Mar 2004 05:06:55 -0500 Received: from optimus.ietf.org ([132.151.1.19]) by ietf-mx with esmtp (Exim 4.12) id 1B6oDv-0005yZ-00 for icar-web-archive@ietf.org; Fri, 26 Mar 2004 05:05:59 -0500 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B6oDx-0000gU-NO; Fri, 26 Mar 2004 05:06:01 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B6oDq-0000fl-VY for icar@optimus.ietf.org; Fri, 26 Mar 2004 05:05:55 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id FAA10566 for ; Fri, 26 Mar 2004 05:05:51 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1B6oDn-0005xR-00 for icar@ietf.org; Fri, 26 Mar 2004 05:05:51 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1B6oCp-0005sj-00 for icar@ietf.org; Fri, 26 Mar 2004 05:04:52 -0500 Received: from imhotep.hursley.ibm.com ([195.212.14.170] helo=mail-gw2.hursley.ibm.com) by ietf-mx with esmtp (Exim 4.12) id 1B6oCA-0005lQ-00 for icar@ietf.org; Fri, 26 Mar 2004 05:04:11 -0500 Received: from localhost.localdomain ([127.0.0.1] helo=mail-gw2.hursley.ibm.com) by mail-gw2.hursley.ibm.com with esmtp (Exim 4.12) id 1B6oBh-00062L-00; Fri, 26 Mar 2004 10:03:41 +0000 Received: from [9.20.136.27] (helo=sp15en17.hursley.ibm.com) by mail-gw2.hursley.ibm.com with esmtp (Exim 4.12) id 1B6oBh-00061y-00; Fri, 26 Mar 2004 10:03:41 +0000 Received: from zurich.ibm.com (sig-9-145-246-252.de.ibm.com [9.145.246.252]) by sp15en17.hursley.ibm.com (AIX5.1/8.11.6p2/8.11.0) with ESMTP id i2QA3eF63254; Fri, 26 Mar 2004 10:03:41 GMT Message-ID: <4063F7B9.5424BB6E@zurich.ibm.com> Date: Fri, 26 Mar 2004 10:28:25 +0100 From: Brian E Carpenter Organization: IBM X-Mailer: Mozilla 4.79 [en] (Windows NT 5.0; U) X-Accept-Language: en,fr,de MIME-Version: 1.0 To: icar@ietf.org, zinin@psg.com References: <200403152055.PAA24125@ietf.org> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Content-Transfer-Encoding: 7bit Subject: [Icar] Re: I-D ACTION:draft-zinin-icar-arts-00.txt Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.0 required=5.0 tests=AWL autolearn=no version=2.60 Content-Transfer-Encoding: 7bit Content-Transfer-Encoding: 7bit I realise this is an oldish version but I do have a few comments... > 2.1 Overview > > Briefly, the cross-functional review process may be described as fol- > lows. > > Each area has an area review team (ART) which ADs delegate the > interim document review function to. When necessary (early in the > process, or during the WG Last call, or both), the WG chairs request > the review for a document by sending an e-mail to all required ARTs > (at a minimum the ART of the area the WG belongs to). I don't see why it has to be the WG chairs, in the case of early review. As soon as a document is in serious discussion, I think the authors should be able to request a review. We should be strongly encouraging early review. If it is a review required as part of a last call process, it's fine that the WG chairs request it - but that is already too late for the first cross-area review. The other advantage of allowing authors to request review is that it provides a path for early review of independent submissions. That is a hole in the draft. > 2.3 Area Review Teams ... > Selection of the ART members is done personally by the ADs. Possible > variations, however, may include open call for nominations, followed > up by ADs interviewing the candidates and personally approving them. This does not follow from assumption 2 in section 2.2. On the contrary, if the objective is to maintain trust in the ADs --or more accurately, to extend that trust to the ART-- there must be some degree of community process in creating the team. I think an open call should be obligatory, and an open call for confidential comments. After that, it's OK for the AD to decide. > When a document needs to be reviewed by ART, the AD assigns two ART > members as "token holders". All ART members are encouraged to review > the document, however, the token holders are held responsible for > providing comments within a 2-week time frame and following up on > them with the document authors and/or the hosting WG. 2 weeks is really very tough if we are talking about deep review of serious specs. I wouldn't like to be asked to review NFSv4 in 2 weeks. > 2.5 Role of Cross-Area Review within the Standards Process ... > Note that ADs are given a tool they can use to off-load docu- > ment review to the extent they believe is necessary, but they > are not required to do so. It is then left to the ADs to make > sure they are using this tool appropriately and sufficiently. I can't quarrel with these words, but somehow they don't get the point that *early* cross area review, before the document gets anywhere near an AD, should significantly increase document quality and reduce the need for detailed review at the end of the process. > > 2.6 Initiation of Review Process > > The cross-functional review process can be initiated either by an AD > or by a WG chair after consultation with the ADs. As noted above, I believe that authors should be *encouraged* to initiate early review themselves. Don't put the load at the end of the process - put it as early as possible. > The review process may be initiated at an early stage of a document > (e.g. when the WG is starting to consider an approach) to ensure > architectural validity and correctness of the general direction This should be "should" IMHO. > The same process may be used by the ADs and the RFC-Editor to request > cross-functional review for individual submissions they are shepherd- > ing or checking for conflicts. This should be a separate section with its own headline, and should be a "should" IMHO. I would be very concerned to see a duplication of review teams for independent submissions. Brian _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Fri Mar 26 07:43:27 2004 Received: from optimus.ietf.org (optimus.ietf.org [132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id HAA17048 for ; Fri, 26 Mar 2004 07:43:27 -0500 (EST) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B6qfq-0001Fi-9b for icar-archive@odin.ietf.org; Fri, 26 Mar 2004 07:42:58 -0500 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i2QCgwJe004808 for icar-archive@odin.ietf.org; Fri, 26 Mar 2004 07:42:58 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B6qfq-0001FT-0C for icar-web-archive@optimus.ietf.org; Fri, 26 Mar 2004 07:42:58 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id HAA17029 for ; Fri, 26 Mar 2004 07:42:56 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1B6qfp-0005Pg-00 for icar-web-archive@ietf.org; Fri, 26 Mar 2004 07:42:57 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1B6qer-0005KM-00 for icar-web-archive@ietf.org; Fri, 26 Mar 2004 07:41:58 -0500 Received: from optimus.ietf.org ([132.151.1.19]) by ietf-mx with esmtp (Exim 4.12) id 1B6qdv-0005F7-00 for icar-web-archive@ietf.org; Fri, 26 Mar 2004 07:40:59 -0500 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B6qdv-0001BE-J1; Fri, 26 Mar 2004 07:40:59 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B6qcy-00019b-7L for icar@optimus.ietf.org; Fri, 26 Mar 2004 07:40:00 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id HAA16963 for ; Fri, 26 Mar 2004 07:39:59 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1B6qcx-0005AZ-00 for icar@ietf.org; Fri, 26 Mar 2004 07:39:59 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1B6qc5-00056c-00 for icar@ietf.org; Fri, 26 Mar 2004 07:39:05 -0500 Received: from hoemail1.lucent.com ([192.11.226.161] helo=hoemail1.firewall.lucent.com) by ietf-mx with esmtp (Exim 4.12) id 1B6qbo-00051z-00 for icar@ietf.org; Fri, 26 Mar 2004 07:38:48 -0500 Received: from nl0006exch001h.wins.lucent.com (h135-85-76-62.lucent.com [135.85.76.62]) by hoemail1.firewall.lucent.com (Switch-2.2.8/Switch-2.2.8) with ESMTP id i2QCbv216025 for ; Fri, 26 Mar 2004 06:38:02 -0600 (CST) Received: by nl0006exch001h.nl.lucent.com with Internet Mail Service (5.5.2657.72) id <16997THG>; Fri, 26 Mar 2004 13:37:56 +0100 Message-ID: <7D5D48D2CAA3D84C813F5B154F43B15503ED0805@nl0006exch001u.nl.lucent.com> From: "Wijnen, Bert (Bert)" To: Brian E Carpenter , icar@ietf.org Subject: RE: [Icar] Re: I-D ACTION:draft-zinin-icar-arts-00.txt Date: Fri, 26 Mar 2004 13:37:49 +0100 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2657.72) Content-Type: text/plain Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.0 required=5.0 tests=AWL autolearn=no version=2.60 Brian writes/comments: > > I realise this is an oldish version but I do have a few comments... > > > 2.1 Overview > > > > Briefly, the cross-functional review process may be described as fol- > > lows. > > > > Each area has an area review team (ART) which ADs delegate the > > interim document review function to. When necessary (early in the > > process, or during the WG Last call, or both), the WG chairs request > > the review for a document by sending an e-mail to all required ARTs > > (at a minimum the ART of the area the WG belongs to). > > I don't see why it has to be the WG chairs, in the case of early review. > As soon as a document is in serious discussion, I think the authors should > be able to request a review. We should be strongly encouraging early review. > By letting the WG chairs ask for it, it would (I hope) be clear that it is a document under consideration by a WG. > If it is a review required as part of a last call process, it's fine that > the WG chairs request it - but that is already too late for the first > cross-area review. > > The other advantage of allowing authors to request review is that it > provides a path for early review of independent submissions. That is > a hole in the draft. > I would appreciate cross-area review of all documents. But if we could get to the point FIRST to have all WG produced documents to get more and better (and earlier) cross-area review, then we would have gained a lot. Once we do better in that space I would start to worry more about independent submissions. Bert _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Fri Mar 26 07:53:29 2004 Received: from optimus.ietf.org (optimus.ietf.org [132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id HAA17516 for ; Fri, 26 Mar 2004 07:53:29 -0500 (EST) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B6qpY-0001po-HT for icar-archive@odin.ietf.org; Fri, 26 Mar 2004 07:53:00 -0500 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i2QCr07A007046 for icar-archive@odin.ietf.org; Fri, 26 Mar 2004 07:53:00 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B6qpY-0001pZ-Ds for icar-web-archive@optimus.ietf.org; Fri, 26 Mar 2004 07:53:00 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id HAA17507 for ; Fri, 26 Mar 2004 07:52:59 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1B6qpX-0006Oq-00 for icar-web-archive@ietf.org; Fri, 26 Mar 2004 07:52:59 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1B6qoa-0006KE-00 for icar-web-archive@ietf.org; Fri, 26 Mar 2004 07:52:01 -0500 Received: from optimus.ietf.org ([132.151.1.19]) by ietf-mx with esmtp (Exim 4.12) id 1B6qnc-0006Ff-00 for icar-web-archive@ietf.org; Fri, 26 Mar 2004 07:51:00 -0500 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B6qnc-0001ms-T0; Fri, 26 Mar 2004 07:51:00 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B6qmg-0001l1-Dm for icar@optimus.ietf.org; Fri, 26 Mar 2004 07:50:02 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id HAA17427 for ; Fri, 26 Mar 2004 07:50:01 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1B6qmf-0006Aq-00 for icar@ietf.org; Fri, 26 Mar 2004 07:50:01 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1B6qlq-00066c-00 for icar@ietf.org; Fri, 26 Mar 2004 07:49:10 -0500 Received: from rwcrmhc13.comcast.net ([204.127.198.39]) by ietf-mx with esmtp (Exim 4.12) id 1B6qlL-0005yf-00 for icar@ietf.org; Fri, 26 Mar 2004 07:48:39 -0500 Received: from dfnjgl21 (c-24-1-99-5.client.comcast.net[24.1.99.5]) by comcast.net (rwcrmhc13) with SMTP id <20040326124803015007rt3me> (Authid: sdawkins@comcast.net); Fri, 26 Mar 2004 12:48:03 +0000 Message-ID: <039701c41330$9ba24f00$0300a8c0@DFNJGL21> Reply-To: "Spencer Dawkins" From: "Spencer Dawkins" To: References: <200403152055.PAA24125@ietf.org> <4063F7B9.5424BB6E@zurich.ibm.com> Subject: Re: [Icar] Re: I-D ACTION:draft-zinin-icar-arts-00.txt Date: Fri, 26 Mar 2004 06:48:15 -0600 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2800.1158 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1165 Content-Transfer-Encoding: 7bit Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.0 required=5.0 tests=AWL autolearn=no version=2.60 Content-Transfer-Encoding: 7bit Content-Transfer-Encoding: 7bit From: "Brian E Carpenter" To: ; Sent: Friday, March 26, 2004 3:28 AM Subject: [Icar] Re: I-D ACTION:draft-zinin-icar-arts-00.txt > I realise this is an oldish version but I do have a few comments... > FWIW, I read almost all Brian's comments as "somebody besides ADs and WG chairs needs to be worried about getting review that is wide enough and early enough, and document authors/editors are the obvious stuckees". I agree. It's pretty tempting to slap out a proposal that assumes that the management structure needs to do something, but we'd be better off if everyone contributed. I know it's hard to step back, but I'd ask people to try to think about ICAR as a community responsibility, and not just another thing for ADs and WG chairs to worry about. Spencer _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Fri Mar 26 09:59:30 2004 Received: from optimus.ietf.org (optimus.ietf.org [132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id JAA22871 for ; Fri, 26 Mar 2004 09:59:30 -0500 (EST) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B6snW-0002nY-TJ for icar-archive@odin.ietf.org; Fri, 26 Mar 2004 09:59:03 -0500 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i2QEx2ho010756 for icar-archive@odin.ietf.org; Fri, 26 Mar 2004 09:59:02 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B6snW-0002nP-J9 for icar-web-archive@optimus.ietf.org; Fri, 26 Mar 2004 09:59:02 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id JAA22853 for ; Fri, 26 Mar 2004 09:58:59 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1B6snU-0002zR-00 for icar-web-archive@ietf.org; Fri, 26 Mar 2004 09:59:00 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1B6smV-0002uG-00 for icar-web-archive@ietf.org; Fri, 26 Mar 2004 09:58:00 -0500 Received: from optimus.ietf.org ([132.151.1.19]) by ietf-mx with esmtp (Exim 4.12) id 1B6slY-0002rP-00 for icar-web-archive@ietf.org; Fri, 26 Mar 2004 09:57:00 -0500 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B6slZ-0002i6-G1; Fri, 26 Mar 2004 09:57:01 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B6skt-0002ez-CG for icar@optimus.ietf.org; Fri, 26 Mar 2004 09:56:19 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id JAA22732 for ; Fri, 26 Mar 2004 09:56:16 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1B6skr-0002q8-00 for icar@ietf.org; Fri, 26 Mar 2004 09:56:17 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1B6sjr-0002n9-00 for icar@ietf.org; Fri, 26 Mar 2004 09:55:16 -0500 Received: from eikenes.alvestrand.no ([158.38.152.233]) by ietf-mx with esmtp (Exim 4.12) id 1B6sj7-0002hF-00 for icar@ietf.org; Fri, 26 Mar 2004 09:54:29 -0500 Received: from localhost (localhost.localdomain [127.0.0.1]) by eikenes.alvestrand.no (Postfix) with ESMTP id 4A67A61AD5; Fri, 26 Mar 2004 15:53:59 +0100 (CET) Received: from eikenes.alvestrand.no ([127.0.0.1]) by localhost (eikenes.alvestrand.no [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 27844-01; Fri, 26 Mar 2004 15:53:57 +0100 (CET) Received: from halvestr-w2k1 (localhost.localdomain [127.0.0.1]) by eikenes.alvestrand.no (Postfix) with ESMTP id 3717861B8C; Fri, 26 Mar 2004 15:53:53 +0100 (CET) Date: Fri, 26 Mar 2004 06:52:59 -0800 From: Harald Tveit Alvestrand To: icar@ietf.org, solutions@alvestrand.no, poised@lists.tislabs.com Message-ID: <344298074.1080283979@localhost> X-Mailer: Mulberry/3.1.0 (Win32) MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="==========74194B1F7A699C7D9CE8==========" X-Virus-Scanned: by amavisd-new at alvestrand.no Subject: [Icar] IESG review of RFC Editor documents Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.1 required=5.0 tests=AWL,MIME_SUSPECT_NAME autolearn=no version=2.60 --==========74194B1F7A699C7D9CE8========== Content-Type: text/plain; charset=us-ascii; format=flowed Content-Disposition: inline Content-Transfer-Encoding: 7bit [NOTE - please have the discussion on ietf@ietf.org!] The IESG has proposed a change in its present review procedures for IESG review of documents submitted directly to the RFC Editor for publication. The IESG will be discussing this in detail, and with the RFC Editor, next week - the input document for that discussion is published as an I-D below Your input is welcome! Copy of the announcement below. (note - between solutions, icar, poised and the IETF list, I chose the IETF list - I've asked for discussion of this on the IETF list.... judgment call. Sorry for the duplicates!). Harald Harald Alvestrand ---------- Forwarded Message ---------- Date: 25. mars 2004 15:38 -0500 From: Internet-Drafts@ietf.org To: IETF-Announce Cc: iesg@ietf.org Subject: I-D ACTION:draft-iesg-rfced-documents-00.txt A New Internet-Draft is available from the on-line Internet-Drafts directories. This draft is a work item of the Internet Engineering Steering Group Working Group of the IETF. Title : The IESG and RFC Editor documents: Procedures Author(s) : H. Alvestrand Filename : draft-iesg-rfced-documents-00.txt Pages : 6 Date : 2004-3-25 This document gives the IESG's procedures for handling documents submitted for RFC publication via the RFC Editor, subsequent to the changes proposed by the IESG at the Seoul IETF, March 2004. NOTE IN DRAFT: These guidelines are proposed, not adopted. Comments are welcome - please send them to iesg@ietf.org. A URL for this Internet-Draft is: http://www.ietf.org/internet-drafts/draft-iesg-rfced-documents-00.txt To remove yourself from the IETF Announcement list, send a message to ietf-announce-request with the word unsubscribe in the body of the message. Internet-Drafts are also available by anonymous FTP. Login with the username "anonymous" and a password of your e-mail address. After logging in, type "cd internet-drafts" and then "get draft-iesg-rfced-documents-00.txt". A list of Internet-Drafts directories can be found in http://www.ietf.org/shadow.html or ftp://ftp.ietf.org/ietf/1shadow-sites.txt Internet-Drafts can also be obtained by e-mail. Send a message to: mailserv@ietf.org. In the body type: "FILE /internet-drafts/draft-iesg-rfced-documents-00.txt". NOTE: The mail server at ietf.org can return the document in MIME-encoded form by using the "mpack" utility. To use this feature, insert the command "ENCODING mime" before the "FILE" command. To decode the response(s), you will need "munpack" or a MIME-compliant mail reader. Different MIME-compliant mail readers exhibit different behavior, especially when dealing with "multipart" MIME messages (i.e. documents which have been split up into multiple messages), so check your local documentation on how to manipulate these messages. Below is the data which will enable a MIME compliant mail reader implementation to automatically retrieve the ASCII version of the Internet-Draft. ---------- End Forwarded Message ---------- --==========74194B1F7A699C7D9CE8========== Content-Type: message/rfc822; name="I-D ACTION:draft-iesg-rfced-documents-00.txt" Return-Path: Received: from eikenes.alvestrand.no ([unix socket]) by eikenes.alvestrand.no (Cyrus v2.1.11-Mandrake-RPM-2.1.11-1mdk) with LMTP; Thu, 25 Mar 2004 21:39:02 +0100 X-Sieve: CMU Sieve 2.2 Return-Path: Received: from localhost (localhost.localdomain [127.0.0.1]) by eikenes.alvestrand.no (Postfix) with ESMTP id 0695F61B9C for ; Thu, 25 Mar 2004 21:39:02 +0100 (CET) Received: from eikenes.alvestrand.no ([127.0.0.1]) by localhost (eikenes.alvestrand.no [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 30744-08 for ; Thu, 25 Mar 2004 21:39:01 +0100 (CET) Received: from optimus.ietf.org (optimus22.ietf.org [132.151.6.22]) by eikenes.alvestrand.no (Postfix) with ESMTP id 03ED161B93 for ; Thu, 25 Mar 2004 21:39:01 +0100 (CET) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B6bcy-0005cf-DM; Thu, 25 Mar 2004 15:39:00 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B6bcb-0005bt-7c for iesg@optimus.ietf.org; Thu, 25 Mar 2004 15:38:37 -0500 Received: from CNRI.Reston.VA.US (localhost [127.0.0.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA19475; Thu, 25 Mar 2004 15:38:34 -0500 (EST) Message-Id: <200403252038.PAA19475@ietf.org> Mime-Version: 1.0 Content-Type: Multipart/Mixed; Boundary="NextPart" To: IETF-Announce: ; Cc: iesg@ietf.org From: Internet-Drafts@ietf.org Reply-To: Internet-Drafts@ietf.org Subject: I-D ACTION:draft-iesg-rfced-documents-00.txt Date: Thu, 25 Mar 2004 15:38:34 -0500 Sender: iesg-admin@ietf.org Errors-To: iesg-admin@ietf.org X-BeenThere: iesg@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: List-Post: List-Help: List-Subscribe: , X-Virus-Scanned: by amavisd-new at alvestrand.no --NextPart A New Internet-Draft is available from the on-line Internet-Drafts directories. This draft is a work item of the Internet Engineering Steering Group Working Group of the IETF. Title : The IESG and RFC Editor documents: Procedures Author(s) : H. Alvestrand Filename : draft-iesg-rfced-documents-00.txt Pages : 6 Date : 2004-3-25 This document gives the IESG's procedures for handling documents submitted for RFC publication via the RFC Editor, subsequent to the changes proposed by the IESG at the Seoul IETF, March 2004. NOTE IN DRAFT: These guidelines are proposed, not adopted. Comments are welcome - please send them to iesg@ietf.org. A URL for this Internet-Draft is: http://www.ietf.org/internet-drafts/draft-iesg-rfced-documents-00.txt To remove yourself from the IETF Announcement list, send a message to ietf-announce-request with the word unsubscribe in the body of the message. Internet-Drafts are also available by anonymous FTP. Login with the username "anonymous" and a password of your e-mail address. After logging in, type "cd internet-drafts" and then "get draft-iesg-rfced-documents-00.txt". A list of Internet-Drafts directories can be found in http://www.ietf.org/shadow.html or ftp://ftp.ietf.org/ietf/1shadow-sites.txt Internet-Drafts can also be obtained by e-mail. Send a message to: mailserv@ietf.org. In the body type: "FILE /internet-drafts/draft-iesg-rfced-documents-00.txt". NOTE: The mail server at ietf.org can return the document in MIME-encoded form by using the "mpack" utility. To use this feature, insert the command "ENCODING mime" before the "FILE" command. To decode the response(s), you will need "munpack" or a MIME-compliant mail reader. Different MIME-compliant mail readers exhibit different behavior, especially when dealing with "multipart" MIME messages (i.e. documents which have been split up into multiple messages), so check your local documentation on how to manipulate these messages. Below is the data which will enable a MIME compliant mail reader implementation to automatically retrieve the ASCII version of the Internet-Draft. --NextPart Content-Type: Multipart/Alternative; Boundary="OtherAccess" --OtherAccess Content-Type: Message/External-body; access-type="mail-server"; server="mailserv@ietf.org" Content-Type: text/plain Content-ID: <2004-3-25154628.I-D@ietf.org> ENCODING mime FILE /internet-drafts/draft-iesg-rfced-documents-00.txt --OtherAccess Content-Type: Message/External-body; name="draft-iesg-rfced-documents-00.txt"; site="ftp.ietf.org"; access-type="anon-ftp"; directory="internet-drafts" Content-Type: text/plain Content-ID: <2004-3-25154628.I-D@ietf.org> --OtherAccess-- --NextPart-- --==========74194B1F7A699C7D9CE8==========-- _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Sun Mar 28 07:36:07 2004 Received: from optimus.ietf.org (optimus.ietf.org [132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id HAA15503 for ; Sun, 28 Mar 2004 07:36:07 -0500 (EST) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B7ZVq-0005GX-HS for icar-archive@odin.ietf.org; Sun, 28 Mar 2004 07:35:38 -0500 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i2SCZcnl020240 for icar-archive@odin.ietf.org; Sun, 28 Mar 2004 07:35:38 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B7ZVq-0005GN-A2 for icar-web-archive@optimus.ietf.org; Sun, 28 Mar 2004 07:35:38 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id HAA15493 for ; Sun, 28 Mar 2004 07:35:36 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1B7ZVp-0002Ee-00 for icar-web-archive@ietf.org; Sun, 28 Mar 2004 07:35:37 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1B7ZUt-00027b-00 for icar-web-archive@ietf.org; Sun, 28 Mar 2004 07:34:40 -0500 Received: from optimus.ietf.org ([132.151.1.19]) by ietf-mx with esmtp (Exim 4.12) id 1B7ZUH-00020l-00 for icar-web-archive@ietf.org; Sun, 28 Mar 2004 07:34:01 -0500 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B7ZUH-00056u-Ks; Sun, 28 Mar 2004 07:34:01 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B7ZTq-00055P-GU for icar@optimus.ietf.org; Sun, 28 Mar 2004 07:33:34 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id HAA15449 for ; Sun, 28 Mar 2004 07:33:32 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1B7ZTp-0001zD-00 for icar@ietf.org; Sun, 28 Mar 2004 07:33:33 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1B7ZSt-0001sS-00 for icar@ietf.org; Sun, 28 Mar 2004 07:32:35 -0500 Received: from imhotep.hursley.ibm.com ([195.212.14.170] helo=mail-gw2.hursley.ibm.com) by ietf-mx with esmtp (Exim 4.12) id 1B7ZRs-0001fJ-00 for icar@ietf.org; Sun, 28 Mar 2004 07:31:32 -0500 Received: from localhost.localdomain ([127.0.0.1] helo=mail-gw2.hursley.ibm.com) by mail-gw2.hursley.ibm.com with esmtp (Exim 4.12) id 1B7ZRN-0002qA-00; Sun, 28 Mar 2004 13:31:01 +0100 Received: from [9.20.136.27] (helo=sp15en17.hursley.ibm.com) by mail-gw2.hursley.ibm.com with esmtp (Exim 4.12) id 1B7ZRN-0002q5-00; Sun, 28 Mar 2004 13:31:01 +0100 Received: from zurich.ibm.com (sig-9-145-226-149.de.ibm.com [9.145.226.149]) by sp15en17.hursley.ibm.com (AIX5.1/8.11.6p2/8.11.0) with ESMTP id i2SCV0F91226; Sun, 28 Mar 2004 13:31:00 +0100 Message-ID: <406695D1.7A94B506@zurich.ibm.com> Date: Sun, 28 Mar 2004 11:07:29 +0200 From: Brian E Carpenter Organization: IBM X-Mailer: Mozilla 4.79 [en] (Windows NT 5.0; U) X-Accept-Language: en,fr,de MIME-Version: 1.0 To: "Wijnen, Bert (Bert)" CC: icar@ietf.org Subject: Re: [Icar] Re: I-D ACTION:draft-zinin-icar-arts-00.txt References: <7D5D48D2CAA3D84C813F5B154F43B15503ED0805@nl0006exch001u.nl.lucent.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Content-Transfer-Encoding: 7bit Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.4 required=5.0 tests=AWL,DATE_IN_PAST_03_06 autolearn=no version=2.60 Content-Transfer-Encoding: 7bit Content-Transfer-Encoding: 7bit Bert, "Wijnen, Bert (Bert)" wrote: > > Brian writes/comments: > > > > I realise this is an oldish version but I do have a few comments... > > > > > 2.1 Overview > > > > > > Briefly, the cross-functional review process may be described as fol- > > > lows. > > > > > > Each area has an area review team (ART) which ADs delegate the > > > interim document review function to. When necessary (early in the > > > process, or during the WG Last call, or both), the WG chairs request > > > the review for a document by sending an e-mail to all required ARTs > > > (at a minimum the ART of the area the WG belongs to). > > > > I don't see why it has to be the WG chairs, in the case of early review. > > As soon as a document is in serious discussion, I think the authors should > > be able to request a review. We should be strongly encouraging early review. > > > By letting the WG chairs ask for it, it would (I hope) be clear that it is > a document under consideration by a WG. > > > If it is a review required as part of a last call process, it's fine that > > the WG chairs request it - but that is already too late for the first > > cross-area review. > > > > The other advantage of allowing authors to request review is that it > > provides a path for early review of independent submissions. That is > > a hole in the draft. > > > I would appreciate cross-area review of all documents. > But if we could get to the point FIRST to have all WG produced documents > to get more and better (and earlier) cross-area review, then we would > have gained a lot. Once we do better in that space I would start to > worry more about independent submissions. Tactically, I agree. But I would like to get the review of independent submissions in the strategy, as a Phase 2 or Phase 3 feature. That only needs minor changes to the draft. Brian _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Sun Mar 28 07:36:07 2004 Received: from optimus.ietf.org (optimus.ietf.org [132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id HAA15516 for ; Sun, 28 Mar 2004 07:36:07 -0500 (EST) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B7ZVr-0005Gu-5f for icar-archive@odin.ietf.org; Sun, 28 Mar 2004 07:35:39 -0500 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i2SCZdTi020258 for icar-archive@odin.ietf.org; Sun, 28 Mar 2004 07:35:39 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B7ZVr-0005Gf-07 for icar-web-archive@optimus.ietf.org; Sun, 28 Mar 2004 07:35:39 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id HAA15497 for ; Sun, 28 Mar 2004 07:35:37 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1B7ZVq-0002Ej-00 for icar-web-archive@ietf.org; Sun, 28 Mar 2004 07:35:38 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1B7ZUu-00027j-00 for icar-web-archive@ietf.org; Sun, 28 Mar 2004 07:34:40 -0500 Received: from optimus.ietf.org ([132.151.1.19]) by ietf-mx with esmtp (Exim 4.12) id 1B7ZUH-00020m-00 for icar-web-archive@ietf.org; Sun, 28 Mar 2004 07:34:01 -0500 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B7ZUH-00057D-QM; Sun, 28 Mar 2004 07:34:01 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B7ZTr-00055U-4x for icar@optimus.ietf.org; Sun, 28 Mar 2004 07:33:35 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id HAA15452 for ; Sun, 28 Mar 2004 07:33:33 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1B7ZTq-0001zQ-00 for icar@ietf.org; Sun, 28 Mar 2004 07:33:34 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1B7ZSt-0001sa-00 for icar@ietf.org; Sun, 28 Mar 2004 07:32:36 -0500 Received: from imhotep.hursley.ibm.com ([195.212.14.170] helo=mail-gw2.hursley.ibm.com) by ietf-mx with esmtp (Exim 4.12) id 1B7ZRv-0001fL-00 for icar@ietf.org; Sun, 28 Mar 2004 07:31:35 -0500 Received: from localhost.localdomain ([127.0.0.1] helo=mail-gw2.hursley.ibm.com) by mail-gw2.hursley.ibm.com with esmtp (Exim 4.12) id 1B7ZRQ-0002qR-00 for icar@ietf.org; Sun, 28 Mar 2004 13:31:04 +0100 Received: from [9.20.136.27] (helo=sp15en17.hursley.ibm.com) by mail-gw2.hursley.ibm.com with esmtp (Exim 4.12) id 1B7ZRQ-0002qM-00 for icar@ietf.org; Sun, 28 Mar 2004 13:31:04 +0100 Received: from zurich.ibm.com (sig-9-145-226-149.de.ibm.com [9.145.226.149]) by sp15en17.hursley.ibm.com (AIX5.1/8.11.6p2/8.11.0) with ESMTP id i2SCV4F132104 for ; Sun, 28 Mar 2004 13:31:04 +0100 Message-ID: <40669B7E.4D9156DA@zurich.ibm.com> Date: Sun, 28 Mar 2004 11:31:42 +0200 From: Brian E Carpenter Organization: IBM X-Mailer: Mozilla 4.79 [en] (Windows NT 5.0; U) X-Accept-Language: en,fr,de MIME-Version: 1.0 To: icar WG References: <200403252038.PAA19396@ietf.org> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Content-Transfer-Encoding: 7bit Subject: [Icar] Re: I-D ACTION:draft-dawkins-newtrk-wgs-00.txt Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.0 required=5.0 tests=AWL autolearn=no version=2.60 Content-Transfer-Encoding: 7bit Content-Transfer-Encoding: 7bit I think this version is a bit too lax. I think two explicit requirements should be placed on the draft in section 3.1 A WGS a) MUST have received cross-area review outside the WG, judged to be adequate by the WG Chair(s) b) MUST have received rough consensus based on a Working Group Last Call Brian _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Sun Mar 28 12:02:18 2004 Received: from optimus.ietf.org (optimus.ietf.org [132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA24156 for ; Sun, 28 Mar 2004 12:02:18 -0500 (EST) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B7dfS-0003uu-6d for icar-archive@odin.ietf.org; Sun, 28 Mar 2004 12:01:51 -0500 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i2SH1oBE015054 for icar-archive@odin.ietf.org; Sun, 28 Mar 2004 12:01:50 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B7dfR-0003uj-V0 for icar-web-archive@optimus.ietf.org; Sun, 28 Mar 2004 12:01:50 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA24124 for ; Sun, 28 Mar 2004 12:01:46 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1B7dfQ-0004Ey-00 for icar-web-archive@ietf.org; Sun, 28 Mar 2004 12:01:48 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1B7deY-00046g-00 for icar-web-archive@ietf.org; Sun, 28 Mar 2004 12:00:55 -0500 Received: from optimus.ietf.org ([132.151.1.19]) by ietf-mx with esmtp (Exim 4.12) id 1B7ddi-0003yU-00 for icar-web-archive@ietf.org; Sun, 28 Mar 2004 12:00:02 -0500 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B7ddh-0003ST-Qo; Sun, 28 Mar 2004 12:00:01 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B7ddc-0003Rv-Ev for icar@optimus.ietf.org; Sun, 28 Mar 2004 11:59:57 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA24020 for ; Sun, 28 Mar 2004 11:59:53 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1B7ddb-0003xZ-00 for icar@ietf.org; Sun, 28 Mar 2004 11:59:55 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1B7dck-0003pB-00 for icar@ietf.org; Sun, 28 Mar 2004 11:59:03 -0500 Received: from joy.songbird.com ([208.184.79.7]) by ietf-mx with esmtp (Exim 4.12) id 1B7dc3-0003bY-00 for icar@ietf.org; Sun, 28 Mar 2004 11:58:19 -0500 Received: from dialup-67.30.105.103.Dial1.SanJose1.Level3.net (jay.songbird.com [208.184.79.253]) by joy.songbird.com (8.11.6/8.11.6) with ESMTP id i2SGved23347; Sun, 28 Mar 2004 08:57:41 -0800 Date: Sun, 28 Mar 2004 08:57:39 -0800 From: Dave Crocker Reply-To: Dave Crocker Organization: Brandenburg InternetWorking X-Priority: 3 (Normal) Message-ID: <96350946.20040328085739@brandenburg.com> To: Brian E Carpenter CC: icar WG Subject: Re: [Icar] Re: I-D ACTION:draft-dawkins-newtrk-wgs-00.txt In-Reply-To: <40669B7E.4D9156DA@zurich.ibm.com> References: <200403252038.PAA19396@ietf.org> <40669B7E.4D9156DA@zurich.ibm.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Content-Transfer-Encoding: 7bit Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.7 required=5.0 tests=AWL,PRIORITY_NO_NAME autolearn=no version=2.60 Content-Transfer-Encoding: 7bit Content-Transfer-Encoding: 7bit Brian, BEC> I think this version is a bit too lax. I think two explicit BEC> requirements should be placed on the draft in section 3.1 That is because you want to make the label heavy-weight, witha high barriers. This, of course, is exactly the philosophy that got us to the current problem with PS. (It worked so well with PS, let's repeat it for WGS?) The goal for WGS is to have a label that is in the transient, developmental spirit of working group I-Ds, rather than the long-term, reputation-of-the-ietf orientation of standards track. And, by the way, requiring cross-area review before assigning the label is a good way to prevent using the label for coordinating the process of _obtaining_ reviews (cross area or otherwise.) d/ -- Dave Crocker Brandenburg InternetWorking Sunnyvale, CA USA _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Mon Mar 29 08:31:12 2004 Received: from optimus.ietf.org (optimus.ietf.org [132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id IAA01447 for ; Mon, 29 Mar 2004 08:31:12 -0500 (EST) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B7wqh-000722-Vq for icar-archive@odin.ietf.org; Mon, 29 Mar 2004 08:30:44 -0500 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i2TDUhUF027026 for icar-archive@odin.ietf.org; Mon, 29 Mar 2004 08:30:43 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B7wqh-00071m-Pz for icar-web-archive@optimus.ietf.org; Mon, 29 Mar 2004 08:30:43 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id IAA01444 for ; Mon, 29 Mar 2004 08:30:41 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1B7wqg-0001g4-00 for icar-web-archive@ietf.org; Mon, 29 Mar 2004 08:30:42 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1B7wpr-0001XX-00 for icar-web-archive@ietf.org; Mon, 29 Mar 2004 08:29:52 -0500 Received: from optimus.ietf.org ([132.151.1.19]) by ietf-mx with esmtp (Exim 4.12) id 1B7wp3-0001Mv-00 for icar-web-archive@ietf.org; Mon, 29 Mar 2004 08:29:01 -0500 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B7wp3-0006w8-CY; Mon, 29 Mar 2004 08:29:01 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B7woa-0006ur-DP for icar@optimus.ietf.org; Mon, 29 Mar 2004 08:28:32 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id IAA01322 for ; Mon, 29 Mar 2004 08:28:29 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1B7woZ-0001KH-00 for icar@ietf.org; Mon, 29 Mar 2004 08:28:31 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1B7wna-0001Av-00 for icar@ietf.org; Mon, 29 Mar 2004 08:27:31 -0500 Received: from imhotep.hursley.ibm.com ([195.212.14.170] helo=mail-gw2.hursley.ibm.com) by ietf-mx with esmtp (Exim 4.12) id 1B7wmi-0000vp-00 for icar@ietf.org; Mon, 29 Mar 2004 08:26:36 -0500 Received: from localhost.localdomain ([127.0.0.1] helo=mail-gw2.hursley.ibm.com) by mail-gw2.hursley.ibm.com with esmtp (Exim 4.12) id 1B7wm9-0003Bj-00; Mon, 29 Mar 2004 14:26:01 +0100 Received: from [9.20.136.27] (helo=sp15en17.hursley.ibm.com) by mail-gw2.hursley.ibm.com with esmtp (Exim 4.12) id 1B7wm9-0003Be-00; Mon, 29 Mar 2004 14:26:01 +0100 Received: from zurich.ibm.com (sig-9-145-227-237.de.ibm.com [9.145.227.237]) by sp15en17.hursley.ibm.com (AIX5.1/8.11.6p2/8.11.0) with ESMTP id i2TDQ1F115192; Mon, 29 Mar 2004 14:26:01 +0100 Message-ID: <406823FF.D6452F26@zurich.ibm.com> Date: Mon, 29 Mar 2004 15:26:23 +0200 From: Brian E Carpenter Organization: IBM X-Mailer: Mozilla 4.79 [en] (Windows NT 5.0; U) X-Accept-Language: en,fr,de MIME-Version: 1.0 To: Dave Crocker CC: icar WG Subject: Re: [Icar] Re: I-D ACTION:draft-dawkins-newtrk-wgs-00.txt References: <200403252038.PAA19396@ietf.org> <40669B7E.4D9156DA@zurich.ibm.com> <96350946.20040328085739@brandenburg.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Content-Transfer-Encoding: 7bit Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.0 required=5.0 tests=AWL autolearn=no version=2.60 Content-Transfer-Encoding: 7bit Content-Transfer-Encoding: 7bit Dave, Dave Crocker wrote: > > Brian, > > BEC> I think this version is a bit too lax. I think two explicit > BEC> requirements should be placed on the draft in section 3.1 > > That is because you want to make the label heavy-weight, witha high > barriers. This, of course, is exactly the philosophy that got us to > the current problem with PS. (It worked so well with PS, let's > repeat it for WGS?) No, I don't want to make it heavyweight, I want just enough checks and balances to prevent egregious results. > > The goal for WGS is to have a label that is in the transient, > developmental spirit of working group I-Ds, rather than the > long-term, reputation-of-the-ietf orientation of standards track. Indeed. That's why I agree that the consensus to publish a WGS should be judged by the WG chairs. > > And, by the way, requiring cross-area review before assigning the > label is a good way to prevent using the label for coordinating the > process of _obtaining_ reviews (cross area or otherwise.) And that's how it should be IMHO. But again, my suggestion is that we give the WG Chairs the responsibility --- no process required outside the WG. Brian _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Mon Mar 29 09:28:57 2004 Received: from optimus.ietf.org (optimus.ietf.org [132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id JAA04067 for ; Mon, 29 Mar 2004 09:28:57 -0500 (EST) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B7xkb-0004ep-TZ for icar-archive@odin.ietf.org; Mon, 29 Mar 2004 09:28:30 -0500 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i2TESTHP017897 for icar-archive@odin.ietf.org; Mon, 29 Mar 2004 09:28:29 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B7xkb-0004ea-Ln for icar-web-archive@optimus.ietf.org; Mon, 29 Mar 2004 09:28:29 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id JAA04063 for ; Mon, 29 Mar 2004 09:28:26 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1B7xkZ-0002N0-00 for icar-web-archive@ietf.org; Mon, 29 Mar 2004 09:28:27 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1B7xjj-0002GM-00 for icar-web-archive@ietf.org; Mon, 29 Mar 2004 09:27:36 -0500 Received: from optimus.ietf.org ([132.151.1.19]) by ietf-mx with esmtp (Exim 4.12) id 1B7xjA-00028v-00 for icar-web-archive@ietf.org; Mon, 29 Mar 2004 09:27:00 -0500 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B7xjB-0004V3-1L; Mon, 29 Mar 2004 09:27:01 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B7xik-0004U6-H9 for icar@optimus.ietf.org; Mon, 29 Mar 2004 09:26:34 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id JAA04002 for ; Mon, 29 Mar 2004 09:26:31 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1B7xii-00027P-00 for icar@ietf.org; Mon, 29 Mar 2004 09:26:32 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1B7xho-0001zw-00 for icar@ietf.org; Mon, 29 Mar 2004 09:25:37 -0500 Received: from adsl-68-76-113-50.dsl.bcvloh.ameritech.net ([68.76.113.50] helo=guns.icir.org) by ietf-mx with esmtp (Exim 4.12) id 1B7xh4-0001o6-00 for icar@ietf.org; Mon, 29 Mar 2004 09:24:51 -0500 Received: from guns.icir.org (localhost [127.0.0.1]) by guns.icir.org (Postfix) with ESMTP id 2776977AB5A; Mon, 29 Mar 2004 09:24:20 -0500 (EST) To: "Spencer Dawkins" From: Mark Allman Reply-To: mallman@icir.org Cc: icar@ietf.org Subject: Re: [Icar] Re: I-D ACTION:draft-zinin-icar-arts-00.txt In-Reply-To: <039701c41330$9ba24f00$0300a8c0@DFNJGL21> Organization: ICSI Center for Internet Research (ICIR) Song-of-the-Day: Money MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha1; protocol="application/pgp-signature" Date: Mon, 29 Mar 2004 09:24:20 -0500 Message-Id: <20040329142420.2776977AB5A@guns.icir.org> Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.0 required=5.0 tests=AWL autolearn=no version=2.60 --=-=-= > FWIW, I read almost all Brian's comments as "somebody besides ADs and > WG chairs needs to be worried about getting review that is wide enough > and early enough, and document authors/editors are the obvious > stuckees". I agree. > > It's pretty tempting to slap out a proposal that assumes that the > management structure needs to do something, but we'd be better off if > everyone contributed. I know it's hard to step back, but I'd ask > people to try to think about ICAR as a community responsibility, and > not just another thing for ADs and WG chairs to worry about. I have not made up my mind about this yet. But, if I could play the devil's advocate for a moment, I'd wonder .... The review team is only so big. And, the community produces *a lot* of documents. So, in some sense if the request has to come from a WG chair then that is a first gate to making sure the document being reviewed is something the IETF potentially cares about. We want to get very early input and so the bar for the WG chair to request review would be low (i.e., "this looks like something the WG might take up", not "the WG just took this doc up"). But, might that be better than allowing just anyone to throw anything to the review team? Or, is the review team DDoS attack just me looking for problems to solve? allman -- Mark Allman -- ICIR -- http://www.icir.org/mallman/ --=-=-= Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.3 (FreeBSD) iD8DBQFAaDGUWyrrWs4yIs4RAgWaAKCQ4aygBZr3Eae0QQWCHhYIUiVQrwCeKX1V wYbW/R//Z3QYNNyI1zOo6Gk= =Oglv -----END PGP SIGNATURE----- --=-=-=-- _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Mon Mar 29 10:19:52 2004 Received: from optimus.ietf.org (optimus.ietf.org [132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id KAA07319 for ; Mon, 29 Mar 2004 10:19:52 -0500 (EST) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B7yXu-0006Q2-5R for icar-archive@odin.ietf.org; Mon, 29 Mar 2004 10:19:26 -0500 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i2TFJQWE024673 for icar-archive@odin.ietf.org; Mon, 29 Mar 2004 10:19:26 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B7yXt-0006Pr-VL for icar-web-archive@optimus.ietf.org; Mon, 29 Mar 2004 10:19:26 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id KAA07240 for ; Mon, 29 Mar 2004 10:19:22 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1B7yXr-0001mf-00 for icar-web-archive@ietf.org; Mon, 29 Mar 2004 10:19:23 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1B7yWt-0001e2-00 for icar-web-archive@ietf.org; Mon, 29 Mar 2004 10:18:23 -0500 Received: from optimus.ietf.org ([132.151.1.19]) by ietf-mx with esmtp (Exim 4.12) id 1B7yVv-0001VH-00 for icar-web-archive@ietf.org; Mon, 29 Mar 2004 10:17:23 -0500 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B7yV1-0005V2-2n; Mon, 29 Mar 2004 10:16:27 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B7yUS-0005Qs-Tw for icar@optimus.ietf.org; Mon, 29 Mar 2004 10:15:53 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id KAA06875 for ; Mon, 29 Mar 2004 10:15:48 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1B7yUP-0001FB-00 for icar@ietf.org; Mon, 29 Mar 2004 10:15:49 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1B7yTb-00015d-00 for icar@ietf.org; Mon, 29 Mar 2004 10:15:00 -0500 Received: from imhotep.hursley.ibm.com ([195.212.14.170] helo=mail-gw2.hursley.ibm.com) by ietf-mx with esmtp (Exim 4.12) id 1B7ySZ-0000nk-00 for icar@ietf.org; Mon, 29 Mar 2004 10:13:56 -0500 Received: from localhost.localdomain ([127.0.0.1] helo=mail-gw2.hursley.ibm.com) by mail-gw2.hursley.ibm.com with esmtp (Exim 4.12) id 1B7yS1-0000hq-00; Mon, 29 Mar 2004 16:13:21 +0100 Received: from [9.20.136.27] (helo=sp15en17.hursley.ibm.com) by mail-gw2.hursley.ibm.com with esmtp (Exim 4.12) id 1B7yS1-0000hl-00; Mon, 29 Mar 2004 16:13:21 +0100 Received: from zurich.ibm.com (sig-9-145-227-237.de.ibm.com [9.145.227.237]) by sp15en17.hursley.ibm.com (AIX5.1/8.11.6p2/8.11.0) with ESMTP id i2TFDLF130390; Mon, 29 Mar 2004 16:13:21 +0100 Message-ID: <40683D27.21D7BB2A@zurich.ibm.com> Date: Mon, 29 Mar 2004 17:13:43 +0200 From: Brian E Carpenter Organization: IBM X-Mailer: Mozilla 4.79 [en] (Windows NT 5.0; U) X-Accept-Language: en,fr,de MIME-Version: 1.0 To: mallman@icir.org CC: Spencer Dawkins , icar@ietf.org Subject: Re: [Icar] Re: I-D ACTION:draft-zinin-icar-arts-00.txt References: <20040329142420.2776977AB5A@guns.icir.org> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Content-Transfer-Encoding: 7bit Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.0 required=5.0 tests=AWL autolearn=no version=2.60 Content-Transfer-Encoding: 7bit Content-Transfer-Encoding: 7bit Mark, Mark Allman wrote: > > > FWIW, I read almost all Brian's comments as "somebody besides ADs and > > WG chairs needs to be worried about getting review that is wide enough > > and early enough, and document authors/editors are the obvious > > stuckees". I agree. > > > > It's pretty tempting to slap out a proposal that assumes that the > > management structure needs to do something, but we'd be better off if > > everyone contributed. I know it's hard to step back, but I'd ask > > people to try to think about ICAR as a community responsibility, and > > not just another thing for ADs and WG chairs to worry about. > > I have not made up my mind about this yet. But, if I could play the > devil's advocate for a moment, I'd wonder .... The review team is only > so big. And, the community produces *a lot* of documents. So, in some > sense if the request has to come from a WG chair then that is a first > gate to making sure the document being reviewed is something the IETF > potentially cares about. We want to get very early input and so the bar > for the WG chair to request review would be low (i.e., "this looks like > something the WG might take up", not "the WG just took this doc up"). > But, might that be better than allowing just anyone to throw anything to > the review team? > > Or, is the review team DDoS attack just me looking for problems to > solve? The i-d-announce list is the DDoS attack today. Targetted requests for early review might actually reduce that effect. But there is a risk, which is why I would ideally want to see some Web tooling for requesting reviews, able to respond "No, Brian already has 17 drafts in his review queue, try Mark instead." Brian _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Mon Mar 29 14:43:14 2004 Received: from optimus.ietf.org (optimus.ietf.org [132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA20304 for ; Mon, 29 Mar 2004 14:43:14 -0500 (EST) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B82el-0003Kh-Kj for icar-archive@odin.ietf.org; Mon, 29 Mar 2004 14:42:47 -0500 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i2TJglAo012805 for icar-archive@odin.ietf.org; Mon, 29 Mar 2004 14:42:47 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B82el-0003KS-Fk for icar-web-archive@optimus.ietf.org; Mon, 29 Mar 2004 14:42:47 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA20253 for ; Mon, 29 Mar 2004 14:42:43 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1B82ei-0001LO-00 for icar-web-archive@ietf.org; Mon, 29 Mar 2004 14:42:44 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1B82dv-0001CI-00 for icar-web-archive@ietf.org; Mon, 29 Mar 2004 14:41:55 -0500 Received: from optimus.ietf.org ([132.151.1.19]) by ietf-mx with esmtp (Exim 4.12) id 1B82d1-00012a-00 for icar-web-archive@ietf.org; Mon, 29 Mar 2004 14:40:59 -0500 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B82d2-0003Bp-Bc; Mon, 29 Mar 2004 14:41:00 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B82ce-00039M-1R for icar@optimus.ietf.org; Mon, 29 Mar 2004 14:40:36 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA20108 for ; Mon, 29 Mar 2004 14:40:32 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1B82cb-0000yY-00 for icar@ietf.org; Mon, 29 Mar 2004 14:40:33 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1B82bf-0000p9-00 for icar@ietf.org; Mon, 29 Mar 2004 14:39:36 -0500 Received: from rwcrmhc13.comcast.net ([204.127.198.39]) by ietf-mx with esmtp (Exim 4.12) id 1B82ai-0000aa-00 for icar@ietf.org; Mon, 29 Mar 2004 14:38:36 -0500 Received: from dfnjgl21 (c-24-1-99-5.client.comcast.net[24.1.99.5]) by comcast.net (rwcrmhc13) with SMTP id <20040329193755015007p2jae> (Authid: sdawkins@comcast.net); Mon, 29 Mar 2004 19:37:55 +0000 Message-ID: <00c601c415c5$5449cd80$0300a8c0@DFNJGL21> Reply-To: "Spencer Dawkins" From: "Spencer Dawkins" To: References: <20040329142420.2776977AB5A@guns.icir.org> <40683D27.21D7BB2A@zurich.ibm.com> Subject: Re: [Icar] Re: I-D ACTION:draft-zinin-icar-arts-00.txt Date: Mon, 29 Mar 2004 13:37:53 -0600 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2800.1158 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1165 Content-Transfer-Encoding: 7bit Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.0 required=5.0 tests=AWL autolearn=no version=2.60 Content-Transfer-Encoding: 7bit Content-Transfer-Encoding: 7bit From: "Brian E Carpenter" To: Cc: "Spencer Dawkins" ; Sent: Monday, March 29, 2004 9:13 AM Subject: Re: [Icar] Re: I-D ACTION:draft-zinin-icar-arts-00.txt > Mark, > > Mark Allman wrote: > > Or, is the review team DDoS attack just me looking for problems to > > solve? > > The i-d-announce list is the DDoS attack today. Targetted requests for I couldn't agree more, and note that i-d announcing with a cc: to a reasonable mailing list is common practice today. > early review might actually reduce that effect. But there is a risk, which > is why I would ideally want to see some Web tooling for requesting reviews, > able to respond "No, Brian already has 17 drafts in his review queue, try > Mark instead." Brian, why do you need Web tooling? You can tell people this today, whether it's true or not :-) _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Mon Mar 29 15:56:11 2004 Received: from optimus.ietf.org (optimus.ietf.org [132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA26142 for ; Mon, 29 Mar 2004 15:56:11 -0500 (EST) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B83nL-0003lU-E8 for icar-archive@odin.ietf.org; Mon, 29 Mar 2004 15:55:43 -0500 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i2TKthsW014470 for icar-archive@odin.ietf.org; Mon, 29 Mar 2004 15:55:43 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B83nL-0003lJ-6W for icar-web-archive@optimus.ietf.org; Mon, 29 Mar 2004 15:55:43 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA26120 for ; Mon, 29 Mar 2004 15:55:40 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1B83nJ-0004fW-00 for icar-web-archive@ietf.org; Mon, 29 Mar 2004 15:55:41 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1B83mV-0004YR-00 for icar-web-archive@ietf.org; Mon, 29 Mar 2004 15:54:52 -0500 Received: from optimus.ietf.org ([132.151.1.19]) by ietf-mx with esmtp (Exim 4.12) id 1B83lg-0004QG-00 for icar-web-archive@ietf.org; Mon, 29 Mar 2004 15:54:00 -0500 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B83lg-0003fg-EG; Mon, 29 Mar 2004 15:54:00 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B83kf-0003dN-3e for icar@optimus.ietf.org; Mon, 29 Mar 2004 15:53:00 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA25961 for ; Mon, 29 Mar 2004 15:52:51 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1B83ka-0004FG-00 for icar@ietf.org; Mon, 29 Mar 2004 15:52:52 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1B83jn-00045s-00 for icar@ietf.org; Mon, 29 Mar 2004 15:52:03 -0500 Received: from adsl-68-76-113-50.dsl.bcvloh.ameritech.net ([68.76.113.50] helo=guns.icir.org) by ietf-mx with esmtp (Exim 4.12) id 1B83is-0003r2-00 for icar@ietf.org; Mon, 29 Mar 2004 15:51:06 -0500 Received: from guns.icir.org (localhost [127.0.0.1]) by guns.icir.org (Postfix) with ESMTP id 8201F77AB5A; Mon, 29 Mar 2004 15:50:35 -0500 (EST) To: Brian E Carpenter From: Mark Allman Reply-To: mallman@icir.org Cc: Spencer Dawkins , icar@ietf.org Subject: Re: [Icar] Re: I-D ACTION:draft-zinin-icar-arts-00.txt In-Reply-To: <40683D27.21D7BB2A@zurich.ibm.com> Organization: ICSI Center for Internet Research (ICIR) Song-of-the-Day: Money MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha1; protocol="application/pgp-signature" Date: Mon, 29 Mar 2004 15:50:35 -0500 Message-Id: <20040329205035.8201F77AB5A@guns.icir.org> Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.0 required=5.0 tests=AWL autolearn=no version=2.60 --=-=-= > The i-d-announce list is the DDoS attack today. Bingo. This was my point. No matter what web tooling you have if the number of reviews you need is very large then you're still in deep. So, if you winnow that by having WG chairs do the requests for things that are going to be considered (by the WG or for inclusion in a WG charter) then maybe that is OK. I am willing to believe that I am chasing daemons that are not there. I am also betting that the additional work on the part of the chairs is not much. And, so, I'm willing to believe that it might be useful to think about. (And, I'll also say that I am mostly concerned with IETF documents, not individual submissions. If we were to include individual submissions then the RFC Editor could easily take on the role of "WG chair" and request review.) > "No, Brian already has 17 drafts in his review queue, try Mark > instead." Actually, probably better to wait on Brian ... :-) Thanks! allman -- Mark Allman -- ICIR -- http://www.icir.org/mallman/ --=-=-= Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.3 (FreeBSD) iD8DBQFAaIwbWyrrWs4yIs4RAs2gAJ955vP3Grhc3XQeY0QaGmQ6EMPzxQCfU1QG 6Egs9NbN9TEZbAQ/p7lfs9k= =SJlk -----END PGP SIGNATURE----- --=-=-=-- _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Tue Mar 30 18:12:54 2004 Received: from optimus.ietf.org (optimus.ietf.org [132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id SAA26343 for ; Tue, 30 Mar 2004 18:12:54 -0500 (EST) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B8RjU-0006Xa-Vn for icar-archive@odin.ietf.org; Tue, 30 Mar 2004 17:29:20 -0500 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i295UOck017123 for icar-archive@odin.ietf.org; Tue, 9 Mar 2004 00:30:24 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B0Zkh-0003cD-7L for icar-web-archive@optimus.ietf.org; Tue, 09 Mar 2004 00:26:03 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id AAA22519 for ; Tue, 9 Mar 2004 00:25:33 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1B0ZkE-0003Q4-00 for icar-web-archive@ietf.org; Tue, 09 Mar 2004 00:25:34 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1B0Zj4-0003D1-00 for icar-web-archive@ietf.org; Tue, 09 Mar 2004 00:24:23 -0500 Received: from optimus.ietf.org ([132.151.1.19]) by ietf-mx with esmtp (Exim 4.12) id 1B0Zik-00032D-00 for icar-web-archive@ietf.org; Tue, 09 Mar 2004 00:24:02 -0500 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B0Zil-0002Z1-TR; Tue, 09 Mar 2004 00:24:03 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B0Ytb-0007kJ-29 for icar@optimus.ietf.org; Mon, 08 Mar 2004 23:31:11 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id XAA20543 for ; Mon, 8 Mar 2004 23:31:07 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1B0YtZ-0002rQ-00 for icar@ietf.org; Mon, 08 Mar 2004 23:31:09 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1B0Ysf-0002ig-00 for icar@ietf.org; Mon, 08 Mar 2004 23:30:14 -0500 Received: from adsl-68-76-113-50.dsl.bcvloh.ameritech.net ([68.76.113.50] helo=guns.icir.org) by ietf-mx with esmtp (Exim 4.12) id 1B0Yrx-0002QZ-00 for icar@ietf.org; Mon, 08 Mar 2004 23:29:29 -0500 Received: from lawyers.icir.org (adsl-68-76-113-50.dsl.bcvloh.ameritech.net [68.76.113.50]) by guns.icir.org (Postfix) with ESMTP id 0ECC577A6D4; Mon, 8 Mar 2004 23:28:58 -0500 (EST) Received: from lawyers.icir.org (localhost [127.0.0.1]) by lawyers.icir.org (Postfix) with ESMTP id 2E00310F4AE; Mon, 8 Mar 2004 23:28:49 -0500 (EST) To: Dave Crocker From: Mark Allman Reply-To: mallman@icir.org Cc: "Spencer Dawkins" , icar@ietf.org Subject: Re: [Icar] independence of reviews; variability In-Reply-To: <17710675617.20040308201158@brandenburg.com> Organization: ICSI Center for Internet Research (ICIR) Song-of-the-Day: Moondance MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha1; protocol="application/pgp-signature" Date: Mon, 08 Mar 2004 23:28:49 -0500 Message-Id: <20040309042849.2E00310F4AE@lawyers.icir.org> Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.0 required=5.0 tests=AWL autolearn=no version=2.60 --=-=-= > my mantra these days is "review early, review often". (However, doing > the former will reduce the need for the latter.) I completely agree. > The idea for reviews needs to be that they are for the purpose of > improving working group technical quality, rather than for swaying an > area director. Got it. One would really hope that swaying an area director would be roughly the same as swaying the community. In other words, that the ADs are reflecting community consensus when they lodge big complaints. E.g., a transport AD saying "nope, this doesn't have congestion control, try again" is not the opinion of that AD. That is a broad opinion of the IETF, I believe (for most protocols). Therefore, a reviewer saying "there is no CC and that is not going to fly" is relly just a reflection of community consensus that will, if unchanged, will manifest itself in the IESG review. Of course, I am sure that maybe some ADs have hot-button issues that do not enjoy IETF consensus. Is it a goal of some review mechanism to beat such opinions out of the process? It'd be nice to try to minimize these things, I think. But, I am not sure a review process is the way to do it. (Or, am I using different words to say the same thing you're trying to say?) Thanks! allman -- Mark Allman -- ICIR -- http://www.icir.org/mallman/ --=-=-= Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.3 (Darwin) iD8DBQFATUgAWyrrWs4yIs4RAlXbAKCed0ZISe8CXQBrBHbCzL2gVurixQCePl0N C4DNCPYpPoNbhGjjnlmqV6M= =ZXbd -----END PGP SIGNATURE----- --=-=-=-- _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar