From exim@www1.ietf.org Wed Jan 7 15:40:48 2004 Received: from optimus.ietf.org ([132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA29811 for ; Wed, 7 Jan 2004 15:40:48 -0500 (EST) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AeKTU-00080X-AA for icar-archive@odin.ietf.org; Wed, 07 Jan 2004 15:40:20 -0500 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i07KeKnU030781 for icar-archive@odin.ietf.org; Wed, 7 Jan 2004 15:40:20 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AeKTU-00080O-1c for icar-web-archive@optimus.ietf.org; Wed, 07 Jan 2004 15:40:20 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA29684 for ; Wed, 7 Jan 2004 15:40:18 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1AeKTS-0002DP-00 for icar-web-archive@ietf.org; Wed, 07 Jan 2004 15:40:18 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1AeKRu-000237-00 for icar-web-archive@ietf.org; Wed, 07 Jan 2004 15:38:43 -0500 Received: from [65.246.255.50] (helo=mx2.foretec.com) by ietf-mx with esmtp (Exim 4.12) id 1AeKMR-0001oq-00 for icar-web-archive@ietf.org; Wed, 07 Jan 2004 15:33:03 -0500 Received: from optimus22.ietf.org ([132.151.6.22] helo=optimus.ietf.org) by mx2.foretec.com with esmtp (Exim 4.24) id 1AeKM2-0006Vb-A8 for icar-web-archive@ietf.org; Wed, 07 Jan 2004 15:32:38 -0500 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AeKLQ-0007Ml-6J; Wed, 07 Jan 2004 15:32:00 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AeKKg-0007MC-IX for icar@optimus.ietf.org; Wed, 07 Jan 2004 15:31:14 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA29362 for ; Wed, 7 Jan 2004 15:31:07 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1AeKKV-0001jz-00 for icar@ietf.org; Wed, 07 Jan 2004 15:31:03 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1AeKFh-0001XF-00 for icar@ietf.org; Wed, 07 Jan 2004 15:26:05 -0500 Received: from psg.com ([147.28.0.62] ident=mailnull) by ietf-mx with esmtp (Exim 4.12) id 1AeK9r-0001GE-00 for icar@ietf.org; Wed, 07 Jan 2004 15:20:03 -0500 Received: from [147.28.0.62] (helo=127.0.0.1) by psg.com with esmtp (Exim 4.24; FreeBSD) id 1AeK9R-000BL4-S7 for icar@ietf.org; Wed, 07 Jan 2004 20:19:38 +0000 Date: Wed, 7 Jan 2004 12:19:16 -0800 From: Alex Zinin X-Mailer: The Bat! (v1.62i) Personal Reply-To: Alex Zinin X-Priority: 3 (Normal) Message-ID: <1364301485.20040107121916@psg.com> To: icar@ietf.org Subject: Re: [Icar] Moving on with ICAR: WG formation and charter In-Reply-To: <9627AE40-3168-11D8-B7AE-000A95E35274@cisco.com> References: <9627AE40-3168-11D8-B7AE-000A95E35274@cisco.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Content-Transfer-Encoding: 7bit Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.4 required=5.0 tests=AWL,RCVD_NUMERIC_HELO autolearn=no version=2.60 Content-Transfer-Encoding: 7bit Content-Transfer-Encoding: 7bit I've reviewed the discussion on the list and we've discussed it within the IESG. Though there has been some disagreement on this list on whether a focused WG like ICAR is a good idea and whether we should wait till the big picture changes are more apparent, we still would like to go ahead with WG formation. I will send the proposed charter for ICAR in my next message. Please do read and express your opinion on the list or send a message to me unicast, even if it is something like "yes, looks good". Regarding a "bigger picture" group that several people suggested. Harald is creating a mailing list for a coordination group, which will include at least WG chairs of relevant WGs and involved IESG members. More info from Harald later. Alex _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Wed Jan 7 15:42:38 2004 Received: from optimus.ietf.org ([132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA29962 for ; Wed, 7 Jan 2004 15:42:38 -0500 (EST) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AeKVG-00087N-56 for icar-archive@odin.ietf.org; Wed, 07 Jan 2004 15:42:10 -0500 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i07KgAFu031199 for icar-archive@odin.ietf.org; Wed, 7 Jan 2004 15:42:10 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AeKVG-000878-0y for icar-web-archive@optimus.ietf.org; Wed, 07 Jan 2004 15:42:10 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA29909 for ; Wed, 7 Jan 2004 15:42:08 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1AeKVE-0002Kv-00 for icar-web-archive@ietf.org; Wed, 07 Jan 2004 15:42:08 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1AeKTM-0002CS-00 for icar-web-archive@ietf.org; Wed, 07 Jan 2004 15:40:13 -0500 Received: from [65.246.255.50] (helo=mx2.foretec.com) by ietf-mx with esmtp (Exim 4.12) id 1AeKRt-00021t-00 for icar-web-archive@ietf.org; Wed, 07 Jan 2004 15:38:41 -0500 Received: from optimus22.ietf.org ([132.151.6.22] helo=optimus.ietf.org) by mx2.foretec.com with esmtp (Exim 4.24) id 1AeKRj-0006yM-1Y for icar-web-archive@ietf.org; Wed, 07 Jan 2004 15:38:31 -0500 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AeKRE-0007t3-OU; Wed, 07 Jan 2004 15:38:00 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AeKQQ-0007ey-G9 for icar@optimus.ietf.org; Wed, 07 Jan 2004 15:37:10 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA29535 for ; Wed, 7 Jan 2004 15:37:08 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1AeKQK-0001xq-00 for icar@ietf.org; Wed, 07 Jan 2004 15:37:04 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1AeKJk-0001jP-00 for icar@ietf.org; Wed, 07 Jan 2004 15:30:16 -0500 Received: from psg.com ([147.28.0.62] ident=mailnull) by ietf-mx with esmtp (Exim 4.12) id 1AeKFD-0001Ug-00 for icar@ietf.org; Wed, 07 Jan 2004 15:25:35 -0500 Received: from [147.28.0.62] (helo=127.0.0.1) by psg.com with esmtp (Exim 4.24; FreeBSD) id 1AeKF2-000BrN-SY for icar@ietf.org; Wed, 07 Jan 2004 20:25:24 +0000 Date: Wed, 7 Jan 2004 12:25:03 -0800 From: Alex Zinin X-Mailer: The Bat! (v1.62i) Personal Reply-To: Alex Zinin X-Priority: 3 (Normal) Message-ID: <1044648133.20040107122503@psg.com> To: icar@ietf.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Content-Transfer-Encoding: 7bit Subject: [Icar] ICAR draft charter Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.4 required=5.0 tests=AWL,RCVD_NUMERIC_HELO autolearn=no version=2.60 Content-Transfer-Encoding: 7bit Content-Transfer-Encoding: 7bit Folks- Draft charter below. Please read. I'd like to hear from as many people as possible. Thanks. -- Alex WG name: improved cross-area review (icar) Chairs: General Area Director(s): Harald Alvestrand General Area Advisor: Harald Alvestrand Mailing list: icar@ietf.org Subscription: icar-request@ietf.org Archives : https://www1.ietf.org/mail-archive/working-groups/icar/current/maillist.html WG Description: Work out mechanisms for improved cross-functional review within the IETF. This includes a better community review, as well as more structured (formal and role-based) pre-IESG review that may be used to improve scalability of the IESG review function. It is an explicit goal of the WG to come up with mechanisms encouraging earlier review of the documents. An early review is best for catching architectural problems while they're still relatively easy to solve. In particular, many cross-area interactions can be spotted and dealt with, thus avoiding many "late surprises". A final review can catch remaining cross-area interactions, as well as deal with overall quality issues. The WG will cooperate with others in starting and evaluating experiments with both early reviews and structured reviews. The evaluation of such experiments may be published as Informational RFCs if the group so desires. The WG will also coordinate with other WGs on proposed changes to the IETF Working Group operations and Standards process if those are considered necessary. WG milestones: FEB 2004: Submit -00 draft on improved community review FEB 2004: Submit -00 draft on improved structured review SEP 2004: Submit draft on improved community review to the IESG for publication as BCP SEP 2004: Submit draft on improved structured community review to the IESG for publication as BCP SEP 2005: Evaluate WG progress and potential; close or recharter _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Wed Jan 7 16:10:41 2004 Received: from optimus.ietf.org ([132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA02293 for ; Wed, 7 Jan 2004 16:10:41 -0500 (EST) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AeKwP-000246-5p for icar-archive@odin.ietf.org; Wed, 07 Jan 2004 16:10:13 -0500 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i07LADCo007932 for icar-archive@odin.ietf.org; Wed, 7 Jan 2004 16:10:13 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AeKwP-00023r-1Z for icar-web-archive@optimus.ietf.org; Wed, 07 Jan 2004 16:10:13 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA02253 for ; Wed, 7 Jan 2004 16:10:10 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1AeKwN-00048W-00 for icar-web-archive@ietf.org; Wed, 07 Jan 2004 16:10:11 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1AeKug-000443-00 for icar-web-archive@ietf.org; Wed, 07 Jan 2004 16:08:27 -0500 Received: from [132.151.1.19] (helo=optimus.ietf.org) by ietf-mx with esmtp (Exim 4.12) id 1AeKtP-0003zh-00 for icar-web-archive@ietf.org; Wed, 07 Jan 2004 16:07:07 -0500 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AeKtP-0001gC-Ow; Wed, 07 Jan 2004 16:07:07 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AeKss-0001bb-Aw for icar@optimus.ietf.org; Wed, 07 Jan 2004 16:06:34 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA02107 for ; Wed, 7 Jan 2004 16:06:31 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1AeKsq-0003wP-00 for icar@ietf.org; Wed, 07 Jan 2004 16:06:32 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1AeKrL-0003nv-00 for icar@ietf.org; Wed, 07 Jan 2004 16:05:00 -0500 Received: from measurement-factory.com ([206.168.0.5]) by ietf-mx with esmtp (Exim 4.12) id 1AeKpr-0003eG-00 for icar@ietf.org; Wed, 07 Jan 2004 16:03:27 -0500 Received: from measurement-factory.com (localhost [127.0.0.1]) by measurement-factory.com (8.12.9/8.12.9) with ESMTP id i07L3Qk3065730; Wed, 7 Jan 2004 14:03:26 -0700 (MST) (envelope-from rousskov@measurement-factory.com) Received: (from rousskov@localhost) by measurement-factory.com (8.12.9/8.12.9/Submit) id i07L3QUH065729; Wed, 7 Jan 2004 14:03:26 -0700 (MST) (envelope-from rousskov) Date: Wed, 7 Jan 2004 14:03:26 -0700 (MST) From: Alex Rousskov To: Alex Zinin cc: icar@ietf.org Subject: Re: [Icar] ICAR draft charter In-Reply-To: <1044648133.20040107122503@psg.com> Message-ID: References: <1044648133.20040107122503@psg.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.0 required=5.0 tests=AWL autolearn=no version=2.60 Alex, The revised charter does not address many comments about the first version of the charter. What do you want us to do: - assume that all unaddressed comments were heard and left unaddressed intentionally by the people who know better; - assume that all unaddressed comments were missed by mistake and should be repeated again by people who have nothing better to do than to dig through old postings and repeat old comments; - something else? Please advice. Also, unrelated to the above, do the proposed milestones meet Dave Crocker's "usefulness" criteria? Should they? Thanks, Alex. On Wed, 7 Jan 2004, Alex Zinin wrote: > Folks- > > Draft charter below. Please read. I'd like to hear from as many people > as possible. > > Thanks. > > -- > Alex > > > WG name: improved cross-area review (icar) > > Chairs: > > General Area Director(s): > Harald Alvestrand > > General Area Advisor: > Harald Alvestrand > > Mailing list: icar@ietf.org > Subscription: icar-request@ietf.org > Archives : > https://www1.ietf.org/mail-archive/working-groups/icar/current/maillist.html > > WG Description: > > Work out mechanisms for improved cross-functional review within > the IETF. This includes a better community review, as well as > more structured (formal and role-based) pre-IESG review that may > be used to improve scalability of the IESG review function. It is > an explicit goal of the WG to come up with mechanisms encouraging > earlier review of the documents. An early review is best for > catching architectural problems while they're still relatively > easy to solve. In particular, many cross-area interactions can be > spotted and dealt with, thus avoiding many "late surprises". A > final review can catch remaining cross-area interactions, as well > as deal with overall quality issues. > > The WG will cooperate with others in starting and evaluating > experiments with both early reviews and structured reviews. The > evaluation of such experiments may be published as Informational > RFCs if the group so desires. > > The WG will also coordinate with other WGs on proposed changes to > the IETF Working Group operations and Standards process if those > are considered necessary. > > WG milestones: > > FEB 2004: Submit -00 draft on improved community review > FEB 2004: Submit -00 draft on improved structured review > SEP 2004: Submit draft on improved community review to > the IESG for publication as BCP > SEP 2004: Submit draft on improved structured community review to > the IESG for publication as BCP > SEP 2005: Evaluate WG progress and potential; close or recharter > > > _______________________________________________ > Icar mailing list > Icar@ietf.org > https://www1.ietf.org/mailman/listinfo/icar > _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Wed Jan 7 17:01:09 2004 Received: from optimus.ietf.org ([132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA07482 for ; Wed, 7 Jan 2004 17:01:09 -0500 (EST) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AeLjF-0004FP-OU for icar-archive@odin.ietf.org; Wed, 07 Jan 2004 17:00:41 -0500 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i07M0fud016316 for icar-archive@odin.ietf.org; Wed, 7 Jan 2004 17:00:41 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AeLjE-0004Ev-Kd for icar-web-archive@optimus.ietf.org; Wed, 07 Jan 2004 17:00:40 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA07278 for ; Wed, 7 Jan 2004 17:00:37 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1AeLjC-0002JD-00 for icar-web-archive@ietf.org; Wed, 07 Jan 2004 17:00:38 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1AeLev-0001Tw-00 for icar-web-archive@ietf.org; Wed, 07 Jan 2004 16:56:14 -0500 Received: from [65.246.255.50] (helo=mx2.foretec.com) by ietf-mx with esmtp (Exim 4.12) id 1AeLcx-00015e-00 for icar-web-archive@ietf.org; Wed, 07 Jan 2004 16:54:11 -0500 Received: from optimus22.ietf.org ([132.151.6.22] helo=optimus.ietf.org) by mx2.foretec.com with esmtp (Exim 4.24) id 1AeLXz-00040a-8w for icar-web-archive@ietf.org; Wed, 07 Jan 2004 16:49:06 -0500 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AeLXw-0003v1-A0; Wed, 07 Jan 2004 16:49:00 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AeLX5-0003tZ-IJ for icar@optimus.ietf.org; Wed, 07 Jan 2004 16:48:07 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA05417 for ; Wed, 7 Jan 2004 16:48:04 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1AeLX3-00009Y-00 for icar@ietf.org; Wed, 07 Jan 2004 16:48:05 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1AeLUF-0007N7-00 for icar@ietf.org; Wed, 07 Jan 2004 16:45:11 -0500 Received: from sj-iport-4.cisco.com ([171.68.10.86]) by ietf-mx with esmtp (Exim 4.12) id 1AeLQp-0006UZ-00 for icar@ietf.org; Wed, 07 Jan 2004 16:41:39 -0500 Received: from mira-sjc5-c.cisco.com (IDENT:mirapoint@mira-sjc5-c.cisco.com [171.71.163.17]) by sj-core-4.cisco.com (8.12.6/8.12.6) with ESMTP id i07Lf5lx028399; Wed, 7 Jan 2004 13:41:06 -0800 (PST) Received: from cisco.com (sjc-vpn2-618.cisco.com [10.21.114.106]) by mira-sjc5-c.cisco.com (Mirapoint Messaging Server MOS 3.3.6-GR) with ESMTP id APR17581; Wed, 7 Jan 2004 13:41:04 -0800 (PST) Date: Wed, 7 Jan 2004 16:41:02 -0500 Subject: Re: [Icar] ICAR draft charter Content-Type: text/plain; charset=US-ASCII; format=flowed Mime-Version: 1.0 (Apple Message framework v553) Cc: icar@ietf.org To: Alex Zinin From: Melinda Shore In-Reply-To: <1044648133.20040107122503@psg.com> Message-Id: <30863767-415A-11D8-AFFF-000A95E35274@cisco.com> Content-Transfer-Encoding: 7bit X-Mailer: Apple Mail (2.553) Content-Transfer-Encoding: 7bit Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.0 required=5.0 tests=none autolearn=no version=2.60 Content-Transfer-Encoding: 7bit Content-Transfer-Encoding: 7bit I prefer the new, terser version. There's still some material in it that could be clearer, though. You use a lot of different terms to describe various kinds of reviews and it's not always obvious what you mean by them. I think it would be useful to consolidate where possible and then define each of the terms and what their relationship is (I assume that some are classes of cross-area review, but I don't know that). Similarly, you say that the wg will cooperate with "others," but I don't know who "others" means. Also, have we decided to use a wg process to make broader changes to IETF structure and processes? The last paragraph may need to change as the superstructure is defined more clearly by the IESG. So, I like the new version but think it could benefit from more terminological clarity. Melinda _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Wed Jan 7 17:01:33 2004 Received: from optimus.ietf.org ([132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA07676 for ; Wed, 7 Jan 2004 17:01:33 -0500 (EST) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AeLje-0004Jo-2E for icar-archive@odin.ietf.org; Wed, 07 Jan 2004 17:01:06 -0500 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i07M164P016592 for icar-archive@odin.ietf.org; Wed, 7 Jan 2004 17:01:06 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AeLjc-0004IX-Dg for icar-web-archive@optimus.ietf.org; Wed, 07 Jan 2004 17:01:04 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA07443 for ; Wed, 7 Jan 2004 17:01:00 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1AeLjZ-0002QE-00 for icar-web-archive@ietf.org; Wed, 07 Jan 2004 17:01:01 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1AeLfg-0001bv-00 for icar-web-archive@ietf.org; Wed, 07 Jan 2004 16:57:01 -0500 Received: from [65.246.255.50] (helo=mx2.foretec.com) by ietf-mx with esmtp (Exim 4.12) id 1AeLdR-0001B3-00 for icar-web-archive@ietf.org; Wed, 07 Jan 2004 16:54:42 -0500 Received: from optimus22.ietf.org ([132.151.6.22] helo=optimus.ietf.org) by mx2.foretec.com with esmtp (Exim 4.24) id 1AeLV9-0003i6-ID for icar-web-archive@ietf.org; Wed, 07 Jan 2004 16:46:16 -0500 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AeLV2-0003mK-Re; Wed, 07 Jan 2004 16:46:00 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AeLU7-0003k6-UO for icar@optimus.ietf.org; Wed, 07 Jan 2004 16:45:03 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA04705 for ; Wed, 7 Jan 2004 16:44:59 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1AeLU4-0007LM-00 for icar@ietf.org; Wed, 07 Jan 2004 16:45:00 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1AeLQc-0006Yi-00 for icar@ietf.org; Wed, 07 Jan 2004 16:41:27 -0500 Received: from psg.com ([147.28.0.62] ident=mailnull) by ietf-mx with esmtp (Exim 4.12) id 1AeLIj-0005Ym-00 for icar@ietf.org; Wed, 07 Jan 2004 16:33:18 -0500 Received: from [147.28.0.62] (helo=127.0.0.1) by psg.com with esmtp (Exim 4.24; FreeBSD) id 1AeLIJ-000Ika-Ow; Wed, 07 Jan 2004 21:32:51 +0000 Date: Wed, 7 Jan 2004 13:32:22 -0800 From: Alex Zinin X-Mailer: The Bat! (v1.62i) Personal Reply-To: Alex Zinin X-Priority: 3 (Normal) Message-ID: <838686851.20040107133222@psg.com> To: Alex Rousskov CC: icar@ietf.org Subject: Re: [Icar] ICAR draft charter In-Reply-To: References: <1044648133.20040107122503@psg.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Content-Transfer-Encoding: 7bit Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.4 required=5.0 tests=AWL,RCVD_NUMERIC_HELO autolearn=no version=2.60 Content-Transfer-Encoding: 7bit Content-Transfer-Encoding: 7bit Alex, > Alex, > The revised charter does not address many comments about the > first version of the charter. What do you want us to do: > - assume that all unaddressed comments were heard > and left unaddressed intentionally by the people > who know better; > - assume that all unaddressed comments were missed > by mistake and should be repeated again by people > who have nothing better to do than to dig through > old postings and repeat old comments; > - something else? I did read your comments and some of them were addressed, some weren't. See below: > a) The scope of the WG is unclear or wrong. "Cross-area" scope is too > narrow (assuming formal IETF areas are implied). "Cross-functional" > scope is undefined in the proposed charter. It is not clear > whether "cross-functional" predicate eliminates something useful. "cross-functional" seems to be quite self-explanatory to me and adding a vocabulary to the charter would be too much, I think. If you have a wording suggestion, please send the text. > b) The playing space for the WG is undefined: it is not clear whether > the WG is allowed to propose changes to core IETF processes, rules, > and roles and, if yes, which processes/rules/roles are in change > scope. The charter says: The WG will also coordinate with other WGs on proposed changes to the IETF Working Group operations and Standards process if those are considered necessary. > c) The charter assumes that "IESG review function" remains mostly > unchanged. There is currently no IETF consensus whether that > function needs to change, IMO. Even some of the quoted drafts > seem to imply significant changes to IESG review function. I don't see how I could address this comment. Since there's no consensus on this issue, we can't say the WG will change the IESG review function. However, in the process of discussion, the WG may very well reach consensus that such change is a good idea and it may recommend a certain process change. > d) The implied difference between "peer" and "structural" is unclear > to me. Define and contrast both terms better since your milestones > depend on the difference. Furthermore, an a priory assumption of > separation between "peer" and "structural" review may hinder WG > creativity. I changed the text to say: This includes a better community review, as well as ^^^^^^^^^^^^^^^^ more structured (formal and role-based) pre-IESG review that may ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ be used to improve scalability of the IESG review function. If you have a suggestion that would improve the text, please send your wording. > Please advice. > Also, unrelated to the above, do the proposed milestones meet Dave > Crocker's "usefulness" criteria? Should they? If you have a problem with the milestones, please explain what it is and how you believe it should be fixed. Thank you. Alex _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Wed Jan 7 17:35:55 2004 Received: from optimus.ietf.org ([132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA10113 for ; Wed, 7 Jan 2004 17:35:54 -0500 (EST) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AeMGu-0005uQ-16 for icar-archive@odin.ietf.org; Wed, 07 Jan 2004 17:35:28 -0500 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i07MZSIr022713 for icar-archive@odin.ietf.org; Wed, 7 Jan 2004 17:35:28 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AeMGt-0005uC-6P for icar-web-archive@optimus.ietf.org; Wed, 07 Jan 2004 17:35:27 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA09984 for ; Wed, 7 Jan 2004 17:35:23 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1AeMGq-00055f-00 for icar-web-archive@ietf.org; Wed, 07 Jan 2004 17:35:24 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1AeMBt-0004qm-00 for icar-web-archive@ietf.org; Wed, 07 Jan 2004 17:30:18 -0500 Received: from [132.151.1.19] (helo=optimus.ietf.org) by ietf-mx with esmtp (Exim 4.12) id 1AeM82-0004aW-00 for icar-web-archive@ietf.org; Wed, 07 Jan 2004 17:26:18 -0500 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AeM7m-0005eD-0R; Wed, 07 Jan 2004 17:26:02 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AeM7B-0005dY-KM for icar@optimus.ietf.org; Wed, 07 Jan 2004 17:25:25 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA09641 for ; Wed, 7 Jan 2004 17:25:22 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1AeM74-0004ZO-00 for icar@ietf.org; Wed, 07 Jan 2004 17:25:18 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1AeM26-0004Oh-00 for icar@ietf.org; Wed, 07 Jan 2004 17:20:10 -0500 Received: from measurement-factory.com ([206.168.0.5]) by ietf-mx with esmtp (Exim 4.12) id 1AeLyK-0004Dp-00 for icar@ietf.org; Wed, 07 Jan 2004 17:16:17 -0500 Received: from measurement-factory.com (localhost [127.0.0.1]) by measurement-factory.com (8.12.9/8.12.9) with ESMTP id i07MFpk3068582; Wed, 7 Jan 2004 15:15:51 -0700 (MST) (envelope-from rousskov@measurement-factory.com) Received: (from rousskov@localhost) by measurement-factory.com (8.12.9/8.12.9/Submit) id i07MFpm1068581; Wed, 7 Jan 2004 15:15:51 -0700 (MST) (envelope-from rousskov) Date: Wed, 7 Jan 2004 15:15:51 -0700 (MST) From: Alex Rousskov To: Alex Zinin cc: icar@ietf.org Subject: Re: [Icar] ICAR draft charter In-Reply-To: <838686851.20040107133222@psg.com> Message-ID: References: <1044648133.20040107122503@psg.com> <838686851.20040107133222@psg.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.0 required=5.0 tests=AWL autolearn=no version=2.60 On Wed, 7 Jan 2004, Alex Zinin wrote: > > a) The scope of the WG is unclear or wrong. "Cross-area" scope is too > > narrow (assuming formal IETF areas are implied). "Cross-functional" > > scope is undefined in the proposed charter. It is not clear > > whether "cross-functional" predicate eliminates something useful. > > "cross-functional" seems to be quite self-explanatory to me and > adding a vocabulary to the charter would be too much, I think. If > you have a wording suggestion, please send the text. Cross-functional is not self-explanatory to me. I cannot send the text since I do not know what you mean by cross-functional. I do not see why adding a definition of the primary scoping term to the charter is "too much", especially if the definition is clear/obvious to you. > > b) The playing space for the WG is undefined: it is not clear whether > > the WG is allowed to propose changes to core IETF processes, rules, > > and roles and, if yes, which processes/rules/roles are in change > > scope. > > The charter says: > > The WG will also coordinate with other WGs on proposed changes to > the IETF Working Group operations and Standards process if those > are considered necessary. The above does not clarify the scope. It is not clear whether the WG is allowed to propose changes that alter core IETF processes, especially if those processes affect things other than just cross-functional review. We must be explicit about what changes, of any are "allowed" or are in the work scope. Coordination with other WGs is fine, but does not define the scope. For example, is it in scope of this WG to discuss enforcement of review rules, such as that all comments must be recorded and addressed or that coverage criteria must be satisfied? Is it in scope of this WG to discuss IETF document management system and propose changes that affect areas other than review (e.g., registration of IETF participants to access the system)? Is it in scope of this WG to discuss whether WG chairs can and should say "no" to already reviewed drafts if their own review contradicts others opinions? Etc. These are just random examples to illustrate the scoping problem. Addressing each example in isolation does not address the problem. > > c) The charter assumes that "IESG review function" remains mostly > > unchanged. There is currently no IETF consensus whether that > > function needs to change, IMO. Even some of the quoted drafts > > seem to imply significant changes to IESG review function. > > I don't see how I could address this comment. Since there's no > consensus on this issue, we can't say the WG will change the IESG > review function. However, in the process of discussion, the WG may > very well reach consensus that such change is a good idea and it may > recommend a certain process change. This needs to be documented explicitly. For example, "The WG is to evaluate IESG review function and propose IESG role and process changes as and if necessary". Or, alternatively, "The WG must operate under the assumption that the current IESG review function and role are not going to change much. Any changes to IESG functions and roles are, hence, out of the WG scope." > > d) The implied difference between "peer" and "structural" is unclear > > to me. Define and contrast both terms better since your milestones > > depend on the difference. Furthermore, an a priory assumption of > > separation between "peer" and "structural" review may hinder WG > > creativity. > > I changed the text to say: > > This includes a better community review, as well as > ^^^^^^^^^^^^^^^^ > more structured (formal and role-based) pre-IESG review that may > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > be used to improve scalability of the IESG review function. I still do not know exactly what the difference is and why the two are a priori separated from each other. > If you have a suggestion that would improve the text, please > send your wording. Since I am unsure of the intent, it is difficult for me to suggest specific improvements. Said that, you can consider replacing Submit -00 draft on improved community review Submit -00 draft on improved structured review with Submit -00 draft(s) on improved IETF review > > Also, unrelated to the above, do the proposed milestones meet Dave > > Crocker's "usefulness" criteria? Should they? > > If you have a problem with the milestones, please explain what it is > and how you believe it should be fixed. The above questions are not an implication of a problem, but an invitation to other IETFers to discuss whether we are eating our own dog food with this charter. I hope that, for example, Dave can review the deadlines and indicate whether they satisfy his usefulness criteria. Alex. _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Wed Jan 7 17:36:01 2004 Received: from optimus.ietf.org ([132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA10138 for ; Wed, 7 Jan 2004 17:36:00 -0500 (EST) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AeMH0-0005uy-1C for icar-archive@odin.ietf.org; Wed, 07 Jan 2004 17:35:34 -0500 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i07MZY2C022742 for icar-archive@odin.ietf.org; Wed, 7 Jan 2004 17:35:34 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AeMGz-0005uj-Tm for icar-web-archive@optimus.ietf.org; Wed, 07 Jan 2004 17:35:33 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA10036 for ; Wed, 7 Jan 2004 17:35:30 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1AeMGx-00057g-00 for icar-web-archive@ietf.org; Wed, 07 Jan 2004 17:35:31 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1AeMCQ-0004tL-00 for icar-web-archive@ietf.org; Wed, 07 Jan 2004 17:30:51 -0500 Received: from [132.151.1.19] (helo=optimus.ietf.org) by ietf-mx with esmtp (Exim 4.12) id 1AeM8w-0004dW-00 for icar-web-archive@ietf.org; Wed, 07 Jan 2004 17:27:14 -0500 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AeM8j-0005h1-Ck; Wed, 07 Jan 2004 17:27:01 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AeM7r-0005ez-GB for icar@optimus.ietf.org; Wed, 07 Jan 2004 17:26:07 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA09664 for ; Wed, 7 Jan 2004 17:25:54 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1AeM7a-0004a6-00 for icar@ietf.org; Wed, 07 Jan 2004 17:25:50 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1AeM2O-0004P9-00 for icar@ietf.org; Wed, 07 Jan 2004 17:20:28 -0500 Received: from ns.execdsl.net ([208.184.15.238] helo=EXECDSL.COM) by ietf-mx with esmtp (Exim 4.12) id 1AeLyg-0004Dt-00 for icar@ietf.org; Wed, 07 Jan 2004 17:16:38 -0500 Received: from [64.254.114.114] (HELO JLaptop.stevecrocker.com) by EXECDSL.COM (CommuniGate Pro SMTP 3.3) with ESMTP id 6052062 for icar@ietf.org; Wed, 07 Jan 2004 17:16:03 -0500 Message-Id: <5.1.0.14.0.20040107171310.01a40458@localhost> X-Sender: joel@stevecrocker.com@localhost X-Mailer: QUALCOMM Windows Eudora Version 5.1 Date: Wed, 07 Jan 2004 17:16:01 -0500 To: icar@ietf.org From: "Joel M. Halpern" Subject: Re: [Icar] ICAR draft charter In-Reply-To: <1044648133.20040107122503@psg.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.0 required=5.0 tests=none autolearn=no version=2.60 Looking at the milestones, I actually hope that we will have several drafts on review for consideration by the working group in February. I would not expect to see working group consensus on a single starting point. I also would not be surprised if some drafts cover both community and structured review. Hence, I would suggest that the first two milestones be combined into: FEB 2004: at least two drafts on improved review covering community and structured reviews. Yours, Joel At 12:25 PM 1/7/2004 -0800, Alex Zinin wrote: >Folks- > >Draft charter below. Please read. I'd like to hear from as many people >as possible. > >Thanks. > >-- >Alex > > WG name: improved cross-area review (icar) > > WG milestones: > > FEB 2004: Submit -00 draft on improved community review > FEB 2004: Submit -00 draft on improved structured review > SEP 2004: Submit draft on improved community review to > the IESG for publication as BCP > SEP 2004: Submit draft on improved structured community review to > the IESG for publication as BCP > SEP 2005: Evaluate WG progress and potential; close or recharter > > >_______________________________________________ >Icar mailing list >Icar@ietf.org >https://www1.ietf.org/mailman/listinfo/icar _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Wed Jan 7 17:39:10 2004 Received: from optimus.ietf.org ([132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA10377 for ; Wed, 7 Jan 2004 17:39:09 -0500 (EST) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AeMK2-0006XV-Pm for icar-archive@odin.ietf.org; Wed, 07 Jan 2004 17:38:43 -0500 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i07Mcg5C025131 for icar-archive@odin.ietf.org; Wed, 7 Jan 2004 17:38:42 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AeMK2-0006XG-JM for icar-web-archive@optimus.ietf.org; Wed, 07 Jan 2004 17:38:42 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA10347 for ; Wed, 7 Jan 2004 17:38:39 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1AeMK0-0005VH-00 for icar-web-archive@ietf.org; Wed, 07 Jan 2004 17:38:40 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1AeMIF-0005N9-00 for icar-web-archive@ietf.org; Wed, 07 Jan 2004 17:36:52 -0500 Received: from [132.151.1.19] (helo=optimus.ietf.org) by ietf-mx with esmtp (Exim 4.12) id 1AeMHQ-0005Ds-00 for icar-web-archive@ietf.org; Wed, 07 Jan 2004 17:36:00 -0500 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AeMHS-0005x8-KK; Wed, 07 Jan 2004 17:36:02 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AeMGo-0005sF-RZ for icar@optimus.ietf.org; Wed, 07 Jan 2004 17:35:22 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA09955 for ; Wed, 7 Jan 2004 17:35:19 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1AeMGm-00054B-00 for icar@ietf.org; Wed, 07 Jan 2004 17:35:20 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1AeMBa-0004nW-00 for icar@ietf.org; Wed, 07 Jan 2004 17:29:59 -0500 Received: from f070.brocade.com ([66.243.153.70] helo=blasphemy.brocade.com) by ietf-mx with esmtp (Exim 4.12) id 1AeM7A-0004YV-00 for icar@ietf.org; Wed, 07 Jan 2004 17:25:24 -0500 Received: from hq-ex-3.corp.brocade.com (hq-ex-3 [192.168.38.35]) by blasphemy.brocade.com (Postfix) with ESMTP id 9B2A51418D; Wed, 7 Jan 2004 14:24:16 -0800 (PST) X-MimeOLE: Produced By Microsoft Exchange V6.0.6249.0 content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Subject: RE: [Icar] ICAR draft charter Date: Wed, 7 Jan 2004 14:24:16 -0800 Message-ID: Thread-Topic: [Icar] ICAR draft charter Thread-Index: AcPVXjCXeH4rRKCPSsSKgR83nl29SwAB4RVg From: "Robert Snively" To: "Alex Zinin" , Content-Transfer-Encoding: quoted-printable Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.0 required=5.0 tests=AWL autolearn=no version=2.60 Content-Transfer-Encoding: quoted-printable Content-Transfer-Encoding: quoted-printable Alex, I like this approach. =20 I think you might be able to clarify the first paragraph a little bit. As I understand it, you are actually focusing on two separate (and probably separable) issues with separate drafts. I had a bit of a problem understanding what kinds of things might be included in each of the drafts. I believe it would help if you were a bit more explicit about the expected scope of each. As an example, I could make up several different scopes for a document considering "community review". I would suggest you make the scope of the WG more explicit by modifying the first paragraph to read something like the following: (note that I am only guessing about your intent with the words "community review" and "structured review" and I may be completely wrong, so your wording for each might be completely different). WG Description: Develop mechanisms for improved cross-functional review within the IETF. It is an explicit goal of the WG to come up with mechanisms encouraging earlier and better review of documents. [Should this apply to all drafts, or just standards track technical drafts?] An early review is best for catching architectural problems while they are relatively easy to solve. It is an explicit goal of the WG to assure that the review process brings in=20 cross-area expertise to spot possible unforeseen interactions and improve overall document quality. Two documents are expected to be created: 1) Community Review Guidelines: Mechanisms and techniques to solicit early=20 cross-functional review, which may include: a) Guidelines for selecting time of reviews. b) Guidelines for selecting and soliciting review from cross-functional experts. c) Other material relevant to this scope. 2) Structured Review Guidelines: Mechanisms and techniques to solicit and record meaningful review of documents, which may include: a) Guidelines for notifying potential reviewers of review requirement b) Guidelines for assuring reviewer participation c) Guidelines for accumulating review issues d) Guidelines for responding to review issues e) Guidelines for measuring and achieving consensus f) Other material relevant to this scope. > WG Description: >=20 > Work out mechanisms for improved cross-functional review within > the IETF. This includes a better community review, as well as > more structured (formal and role-based) pre-IESG review that may > be used to improve scalability of the IESG review function. It is > an explicit goal of the WG to come up with mechanisms encouraging > earlier review of the documents. An early review is best for > catching architectural problems while they're still relatively > easy to solve. In particular, many cross-area interactions can be > spotted and dealt with, thus avoiding many "late surprises". A > final review can catch remaining cross-area interactions, as well > as deal with overall quality issues. _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Thu Jan 8 12:09:02 2004 Received: from optimus.ietf.org ([132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA02886 for ; Thu, 8 Jan 2004 12:09:02 -0500 (EST) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1Aede6-0003HU-IC for icar-archive@odin.ietf.org; Thu, 08 Jan 2004 12:08:34 -0500 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i08H8YTe012611 for icar-archive@odin.ietf.org; Thu, 8 Jan 2004 12:08:34 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1Aede4-0003HH-UB for icar-web-archive@optimus.ietf.org; Thu, 08 Jan 2004 12:08:32 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA02852 for ; Thu, 8 Jan 2004 12:08:29 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1Aede3-0000o4-00 for icar-web-archive@ietf.org; Thu, 08 Jan 2004 12:08:31 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1AedcG-0000hF-00 for icar-web-archive@ietf.org; Thu, 08 Jan 2004 12:06:41 -0500 Received: from [132.151.1.19] (helo=optimus.ietf.org) by ietf-mx with esmtp (Exim 4.12) id 1Aedaf-0000br-00 for icar-web-archive@ietf.org; Thu, 08 Jan 2004 12:05:01 -0500 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1Aedaf-0002uj-Ja; Thu, 08 Jan 2004 12:05:01 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AedaG-0002tw-8h for icar@optimus.ietf.org; Thu, 08 Jan 2004 12:04:36 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA02768 for ; Thu, 8 Jan 2004 12:04:33 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1AedaE-0000Ze-00 for icar@ietf.org; Thu, 08 Jan 2004 12:04:35 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1AedYN-0000Uw-00 for icar@ietf.org; Thu, 08 Jan 2004 12:02:40 -0500 Received: from f070.brocade.com ([66.243.153.70] helo=blasphemy.brocade.com) by ietf-mx with esmtp (Exim 4.12) id 1AedX5-0000QJ-00 for icar@ietf.org; Thu, 08 Jan 2004 12:01:19 -0500 Received: from hq-ex-3.corp.brocade.com (hq-ex-3 [192.168.38.35]) by blasphemy.brocade.com (Postfix) with ESMTP id BC2291415E; Thu, 8 Jan 2004 09:00:46 -0800 (PST) X-MimeOLE: Produced By Microsoft Exchange V6.0.6249.0 content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Subject: RE: [Icar] ICAR draft charter Date: Thu, 8 Jan 2004 09:00:46 -0800 Message-ID: Thread-Topic: [Icar] ICAR draft charter Thread-Index: AcPVXjCXeH4rRKCPSsSKgR83nl29SwAqQ0xQ From: "Robert Snively" To: "Alex Zinin" , Content-Transfer-Encoding: quoted-printable Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.0 required=5.0 tests=AWL autolearn=no version=2.60 Content-Transfer-Encoding: quoted-printable Content-Transfer-Encoding: quoted-printable As I thought about this, there is one other thing that should be addressed. It seems to me to be important to include at least the obvious liaison requirements explicitly in the charter. Certainly, the "problem" working group's results=20 should be explicitly included as inputs to the working group. There may be others that I am not aware of that should also be included. If we leave out these known liaison activities, we run the risk of replicating already completed work (and even coming out with conflicting conclusions). I would propose that the last paragraph of the WG description be extended as follows: The WG will also coordinate with other WGs on proposed changes to the IETF Working Group operations and Standards process if those are considered necessary. The working group will, as part of its charter, address the relevant parts of the "problem" working group's results. Bob Snively +1 408 333 8135 > -----Original Message----- > From: Alex Zinin [mailto:zinin@psg.com] > Sent: Wednesday, January 07, 2004 12:25 PM > To: icar@ietf.org > Subject: [Icar] ICAR draft charter >=20 >=20 > Folks- >=20 > Draft charter below. Please read. I'd like to hear from as many people > as possible. >=20 > Thanks. >=20 > --=20 > Alex >=20 >=20 > WG name: improved cross-area review (icar) >=20 > Chairs: >=20 > General Area Director(s): > Harald Alvestrand >=20 > General Area Advisor: > Harald Alvestrand > =20 > Mailing list: icar@ietf.org > Subscription: icar-request@ietf.org > Archives :=20 > =20 > https://www1.ietf.org/mail-archive/working-groups/icar/current > /maillist.html >=20 > WG Description: >=20 > Work out mechanisms for improved cross-functional review within > the IETF. This includes a better community review, as well as > more structured (formal and role-based) pre-IESG review that may > be used to improve scalability of the IESG review function. It is > an explicit goal of the WG to come up with mechanisms encouraging > earlier review of the documents. An early review is best for > catching architectural problems while they're still relatively > easy to solve. In particular, many cross-area interactions can be > spotted and dealt with, thus avoiding many "late surprises". A > final review can catch remaining cross-area interactions, as well > as deal with overall quality issues. > =20 > The WG will cooperate with others in starting and evaluating > experiments with both early reviews and structured reviews. The > evaluation of such experiments may be published as Informational > RFCs if the group so desires. >=20 > The WG will also coordinate with other WGs on proposed changes to > the IETF Working Group operations and Standards process if those > are considered necessary. >=20 > WG milestones: >=20 > FEB 2004: Submit -00 draft on improved community review > FEB 2004: Submit -00 draft on improved structured review > SEP 2004: Submit draft on improved community review to > the IESG for publication as BCP > SEP 2004: Submit draft on improved structured community review to > the IESG for publication as BCP > SEP 2005: Evaluate WG progress and potential; close or recharter >=20 >=20 > _______________________________________________ > Icar mailing list > Icar@ietf.org > https://www1.ietf.org/mailman/listinfo/icar >=20 >=20 _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Thu Jan 8 20:25:08 2004 Received: from optimus.ietf.org ([132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id UAA21604 for ; Thu, 8 Jan 2004 20:25:07 -0500 (EST) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AelOB-0000LX-Ir for icar-archive@odin.ietf.org; Thu, 08 Jan 2004 20:24:39 -0500 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i091Ode6001322 for icar-archive@odin.ietf.org; Thu, 8 Jan 2004 20:24:39 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AelO9-0000LD-Ty for icar-web-archive@optimus.ietf.org; Thu, 08 Jan 2004 20:24:37 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id UAA21580 for ; Thu, 8 Jan 2004 20:24:35 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1AelO7-0005un-00 for icar-web-archive@ietf.org; Thu, 08 Jan 2004 20:24:35 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1AelMJ-0005rE-00 for icar-web-archive@ietf.org; Thu, 08 Jan 2004 20:22:45 -0500 Received: from [132.151.1.19] (helo=optimus.ietf.org) by ietf-mx with esmtp (Exim 4.12) id 1AelKe-0005n9-00 for icar-web-archive@ietf.org; Thu, 08 Jan 2004 20:21:00 -0500 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AelKf-0000Hw-QJ; Thu, 08 Jan 2004 20:21:01 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AelKW-0000Gx-Uk for icar@optimus.ietf.org; Thu, 08 Jan 2004 20:20:52 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id UAA21479 for ; Thu, 8 Jan 2004 20:20:50 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1AelKU-0005lh-00 for icar@ietf.org; Thu, 08 Jan 2004 20:20:50 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1AelIm-0005fM-00 for icar@ietf.org; Thu, 08 Jan 2004 20:19:05 -0500 Received: from psg.com ([147.28.0.62] ident=mailnull) by ietf-mx with esmtp (Exim 4.12) id 1AelHE-0005YR-00 for icar@ietf.org; Thu, 08 Jan 2004 20:17:28 -0500 Received: from [147.28.0.62] (helo=127.0.0.1) by psg.com with esmtp (Exim 4.24; FreeBSD) id 1AelHE-000IKn-5s for icar@ietf.org; Fri, 09 Jan 2004 01:17:28 +0000 Date: Thu, 8 Jan 2004 17:14:53 -0800 From: Alex Zinin X-Mailer: The Bat! (v1.62i) Personal Reply-To: Alex Zinin X-Priority: 3 (Normal) Message-ID: <50108437695.20040108171453@psg.com> To: icar@ietf.org Subject: Re: [Icar] ICAR draft charter In-Reply-To: <30863767-415A-11D8-AFFF-000A95E35274@cisco.com> References: <30863767-415A-11D8-AFFF-000A95E35274@cisco.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Content-Transfer-Encoding: 7bit Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.4 required=5.0 tests=AWL,RCVD_NUMERIC_HELO autolearn=no version=2.60 Content-Transfer-Encoding: 7bit Content-Transfer-Encoding: 7bit Melinda, Alex, Joel, Robert- I will post an updated charter in the next message, inline answers below. Wednesday, January 7, 2004, 1:41:02 PM, Melinda Shore wrote: > I prefer the new, terser version. There's still some material > in it that could be clearer, though. You use a lot of different > terms to describe various kinds of reviews and it's not always > obvious what you mean by them. I think it would be useful to > consolidate where possible and then define each of the terms and > what their relationship is (I assume that some are classes of > cross-area review, but I don't know that). I changed the text to say "cross-functional" everywhere, leaving cross-area only in the title, mainly because of the acronym. My personal interpretation of the difference between the two is that cross-functional includes cross-area and cross-WG within an area. I also included definitons in the beginning of the charter. > Similarly, you say > that the wg will cooperate with "others," but I don't know who > "others" means. I think we should leave this open, since those "others" may be any WG that are willing to try and experiment with proposed mechanisms, seems we shouldn't specify them now. > Also, have we decided to use a wg process to > make broader changes to IETF structure and processes? I don't know any other one, and, as far as I know, a WG process is how we did it the past (Scott will correct me if I'm wrong--I'm admittedly new to the whole "process changing" saga). > The last > paragraph may need to change as the superstructure is defined > more clearly by the IESG. Sure. Wednesday, January 7, 2004, 2:15:51 PM, Alex Rousskov wrote: > Cross-functional is not self-explanatory to me. I cannot send the text > since I do not know what you mean by cross-functional. I do not see > why adding a definition of the primary scoping term to the charter is > "too much", especially if the definition is clear/obvious to you. I changed the text to include definitions. Please check if it is any better this way and propose improvements if you think they are needed. >> > b) The playing space for the WG is undefined: it is not clear whether >> > the WG is allowed to propose changes to core IETF processes, rules, >> > and roles and, if yes, which processes/rules/roles are in change >> > scope. >> >> The charter says: >> >> The WG will also coordinate with other WGs on proposed changes to >> the IETF Working Group operations and Standards process if those >> are considered necessary. > The above does not clarify the scope. It is not clear whether the WG > is allowed to propose changes that alter core IETF processes, > especially if those processes affect things other than just > cross-functional review. We must be explicit about what changes, of > any are "allowed" or are in the work scope. Coordination with other > WGs is fine, but does not define the scope. I think we will need changes in the IETF review process anyways. However, it is hard to tell now how major they will be, whether they will affect the core IETF processes, and how those core processes should be defined. Proposing reasonable changes should be fine as long as we keep in mind the primary goal of this WG--to improve the review process. [...] >> > c) The charter assumes that "IESG review function" remains mostly >> > unchanged. There is currently no IETF consensus whether that >> > function needs to change, IMO. Even some of the quoted drafts >> > seem to imply significant changes to IESG review function. >> >> I don't see how I could address this comment. Since there's no >> consensus on this issue, we can't say the WG will change the IESG >> review function. However, in the process of discussion, the WG may >> very well reach consensus that such change is a good idea and it may >> recommend a certain process change. > This needs to be documented explicitly. For example, "The WG is to > evaluate IESG review function and propose IESG role and process > changes as and if necessary". Or, alternatively, "The WG must operate > under the assumption that the current IESG review function and role > are not going to change much. Any changes to IESG functions and roles > are, hence, out of the WG scope." The approach should be, I believe, to improve the pre-IESG review process in such a way that the IESG can make the decision quickly. On the other hand, I could see how some details of IESG review process may change to accommodate the new review process. So, for this question and the question above, I don't think we should put an explicit guideline in the charter. >> > d) The implied difference between "peer" and "structural" is unclear >> > to me. Define and contrast both terms better since your milestones >> > depend on the difference. Furthermore, an a priory assumption of >> > separation between "peer" and "structural" review may hinder WG >> > creativity. >> >> I changed the text to say: >> >> This includes a better community review, as well as >> ^^^^^^^^^^^^^^^^ >> more structured (formal and role-based) pre-IESG review that may >> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ >> be used to improve scalability of the IESG review function. > I still do not know exactly what the difference is and why the two > are a priori separated from each other. I tried to explain this in my earlier message to the list: http://www1.ietf.org/mail-archive/working-groups/icar/current/msg00020.html The new text also contains some info in the definitions. >> If you have a suggestion that would improve the text, please >> send your wording. > Since I am unsure of the intent, it is difficult for me to > suggest specific improvements. Said that, you can consider > replacing > Submit -00 draft on improved community review > Submit -00 draft on improved structured review > with > Submit -00 draft(s) on improved IETF review I did something like this to address this and Joel's comments. Wednesday, January 7, 2004, 2:16:01 PM, Joel M. Halpern wrote: > Looking at the milestones, I actually hope that we will have several drafts > on review for consideration by the working group in February. I would not > expect to see working group consensus on a single starting point. I also > would not be surprised if some drafts cover both community and structured > review. > Hence, I would suggest that the first two milestones be combined into: > FEB 2004: at least two drafts on improved review covering community > and structured reviews. I changed the milestones. Wednesday, January 7, 2004, 2:24:16 PM, Robert Snively wrote: [...] > I like this approach. > I think you might be able to clarify the first paragraph a little > bit. As I understand it, you are actually focusing on two > separate (and probably separable) issues with separate drafts. > I had a bit of a problem understanding what kinds of things might > be included in each of the drafts. [...] Please check the new version, it may explain things better now. Also, please see my earlier message: http://www1.ietf.org/mail-archive/working-groups/icar/current/msg00020.html Regarding guidelines for specific drafts, I think the charter should not have those as different people may have different ideas in mind. I also addressed your second comment. Please see the 1st para in the new charter. Alex _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Thu Jan 8 20:27:10 2004 Received: from optimus.ietf.org ([132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id UAA21661 for ; Thu, 8 Jan 2004 20:27:10 -0500 (EST) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AelQ9-0000N2-QT for icar-archive@odin.ietf.org; Thu, 08 Jan 2004 20:26:41 -0500 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i091QfK0001418 for icar-archive@odin.ietf.org; Thu, 8 Jan 2004 20:26:41 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AelQ9-0000Mn-M0 for icar-web-archive@optimus.ietf.org; Thu, 08 Jan 2004 20:26:41 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id UAA21637 for ; Thu, 8 Jan 2004 20:26:39 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1AelQ7-00060S-00 for icar-web-archive@ietf.org; Thu, 08 Jan 2004 20:26:39 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1AelOF-0005vt-00 for icar-web-archive@ietf.org; Thu, 08 Jan 2004 20:24:44 -0500 Received: from [132.151.1.19] (helo=optimus.ietf.org) by ietf-mx with esmtp (Exim 4.12) id 1AelMZ-0005sQ-00 for icar-web-archive@ietf.org; Thu, 08 Jan 2004 20:22:59 -0500 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AelMb-0000Jy-6Y; Thu, 08 Jan 2004 20:23:01 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AelMK-0000JU-3V for icar@optimus.ietf.org; Thu, 08 Jan 2004 20:22:44 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id UAA21550 for ; Thu, 8 Jan 2004 20:22:42 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1AelMI-0005qr-00 for icar@ietf.org; Thu, 08 Jan 2004 20:22:42 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1AelKa-0005ml-00 for icar@ietf.org; Thu, 08 Jan 2004 20:20:57 -0500 Received: from psg.com ([147.28.0.62] ident=mailnull) by ietf-mx with esmtp (Exim 4.12) id 1AelJY-0005gy-00 for icar@ietf.org; Thu, 08 Jan 2004 20:19:52 -0500 Received: from [147.28.0.62] (helo=127.0.0.1) by psg.com with esmtp (Exim 4.24; FreeBSD) id 1AelJX-000Ic0-Uf for icar@ietf.org; Fri, 09 Jan 2004 01:19:52 +0000 Date: Thu, 8 Jan 2004 17:17:16 -0800 From: Alex Zinin X-Mailer: The Bat! (v1.62i) Personal Reply-To: Alex Zinin X-Priority: 3 (Normal) Message-ID: <196108581021.20040108171716@psg.com> To: icar@ietf.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Content-Transfer-Encoding: 7bit Subject: [Icar] ICAR draft charter rev 2 Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.4 required=5.0 tests=AWL,RCVD_NUMERIC_HELO autolearn=no version=2.60 Content-Transfer-Encoding: 7bit Content-Transfer-Encoding: 7bit Revision 2 below, please. -- Alex WG name: improved cross-area review (icar) Chairs: General Area Director(s): Harald Alvestrand General Area Advisor: Harald Alvestrand Mailing list: icar@ietf.org Subscription: icar-request@ietf.org Archives : https://www1.ietf.org/mail-archive/working-groups/icar/current/maillist.html WG Description: The WG will work out mechanisms for improved cross-functional review within the IETF, addressing the issues identified by the PROBLEM WG. This includes a better community review, as well as more structured pre-IESG review that may be used to improve scalability of the IESG review function. Definitions: o Cross-functional review: document review covering different aspects of Internet technology, including those represented by WGs within different IETF areas. o Community review: document review performed by individual IETF participants and not caused by their responsibilities within the IETF management structure. o Structured review: more formal and role-based document review performed by individuals assuming certain responsibilities (such as by WG chairs, directorate and IESG members.) It is an explicit goal of the WG to come up with mechanisms encouraging earlier review of the documents. An early review is best for catching architectural problems while they're still relatively easy to solve. In particular, many cross-functional interactions can be spotted and dealt with, thus avoiding many "late surprises". A final review (currently done by the IESG) can catch remaining cross-functional interactions, as well as deal with overall quality issues. The WG will cooperate with others in starting and evaluating experiments with early community and structured reviews. The evaluation of such experiments may be published as Informational RFCs if the group so desires. The WG will also coordinate with other WGs involved in the IETF reform process on proposed changes to the IETF Working Group operations and Standards process if those are considered necessary. WG milestones: FEB 2004: Submit drafts on improved community and structured reviews SEP 2004: Submit draft on improved community review to the IESG for publication as BCP SEP 2004: Submit draft on improved structured community review to the IESG for publication as BCP SEP 2005: Evaluate WG progress and potential; close or recharter _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Fri Jan 9 09:51:20 2004 Received: from optimus.ietf.org ([132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id JAA02414 for ; Fri, 9 Jan 2004 09:51:20 -0500 (EST) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AexyP-0000l0-5R for icar-archive@odin.ietf.org; Fri, 09 Jan 2004 09:50:53 -0500 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i09EordI002904 for icar-archive@odin.ietf.org; Fri, 9 Jan 2004 09:50:53 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AexyO-0000kl-WD for icar-web-archive@optimus.ietf.org; Fri, 09 Jan 2004 09:50:53 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id JAA02395 for ; Fri, 9 Jan 2004 09:50:50 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1AexyN-0005eZ-00 for icar-web-archive@ietf.org; Fri, 09 Jan 2004 09:50:51 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1AexwT-0005bA-00 for icar-web-archive@ietf.org; Fri, 09 Jan 2004 09:48:54 -0500 Received: from [132.151.1.19] (helo=optimus.ietf.org) by ietf-mx with esmtp (Exim 4.12) id 1Aexue-0005YD-00 for icar-web-archive@ietf.org; Fri, 09 Jan 2004 09:47:00 -0500 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1Aexue-0000iH-KG; Fri, 09 Jan 2004 09:47:00 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AexuY-0000hq-OQ for icar@optimus.ietf.org; Fri, 09 Jan 2004 09:46:56 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id JAA02361 for ; Fri, 9 Jan 2004 09:46:52 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1AexuW-0005XC-00 for icar@ietf.org; Fri, 09 Jan 2004 09:46:52 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1Aexsh-0005UO-00 for icar@ietf.org; Fri, 09 Jan 2004 09:44:59 -0500 Received: from auemail1.lucent.com ([192.11.223.161] helo=auemail1.firewall.lucent.com) by ietf-mx with esmtp (Exim 4.12) id 1AexsV-0005Qp-00 for icar@ietf.org; Fri, 09 Jan 2004 09:44:47 -0500 Received: from nl0006exch001h.wins.lucent.com (h135-85-76-62.lucent.com [135.85.76.62]) by auemail1.firewall.lucent.com (Switch-2.2.8/Switch-2.2.0) with ESMTP id i09Ei9g29737 for ; Fri, 9 Jan 2004 08:44:11 -0600 (CST) Received: by nl0006exch001h.nl.lucent.com with Internet Mail Service (5.5.2657.72) id ; Fri, 9 Jan 2004 15:44:07 +0100 Message-ID: <7D5D48D2CAA3D84C813F5B154F43B155033D3AAA@nl0006exch001u.nl.lucent.com> From: "Wijnen, Bert (Bert)" To: "Icar (E-mail)" Date: Fri, 9 Jan 2004 15:44:02 +0100 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2657.72) Content-Type: text/plain Subject: [Icar] Input based on SIRS experience Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.0 required=5.0 tests=AWL autolearn=no version=2.60 Forwarded with permission. But probably only usefull after a WG gets instantiated. > -----Original Message----- > From: Brian E Carpenter [mailto:brc@zurich.ibm.com] > Sent: vrijdag 9 januari 2004 11:40 > To: James Kempf > Cc: dcrocker@brandenburg.com; solutions@alvestrand.no > Subject: Re: [Solutions] SIRS dismissed as a failure > > > My opinion is that we didn't generate enough interaction with > the ADs over SIRs, and part of this was due to a technical gap - > there was no link between the I-D tracker and the reviews. So I > think it is very important for the real review system that it is > somehow linked to the I-D (and hence to the I-D tracker) via > the ietf.org web site. So that if you have the name of an I-D, > you can not only find its status from the tracker, but also > its reviews. They then become a real evaluation tool. > > Brian _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Fri Jan 9 10:35:56 2004 Received: from optimus.ietf.org ([132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id KAA05758 for ; Fri, 9 Jan 2004 10:35:56 -0500 (EST) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AeyfY-00030k-TP for icar-archive@odin.ietf.org; Fri, 09 Jan 2004 10:35:29 -0500 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i09FZSPF011573 for icar-archive@odin.ietf.org; Fri, 9 Jan 2004 10:35:28 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AeyfY-00030a-P5 for icar-web-archive@optimus.ietf.org; Fri, 09 Jan 2004 10:35:28 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id KAA05664 for ; Fri, 9 Jan 2004 10:35:25 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1Aeyd0-0007fq-00 for icar-web-archive@ietf.org; Fri, 09 Jan 2004 10:32:50 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1AeyPJ-0006ur-00 for icar-web-archive@ietf.org; Fri, 09 Jan 2004 10:18:42 -0500 Received: from [132.151.1.19] (helo=optimus.ietf.org) by ietf-mx with esmtp (Exim 4.12) id 1AeyL5-0006et-00 for icar-web-archive@ietf.org; Fri, 09 Jan 2004 10:14:19 -0500 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AeyKn-00024L-1w; Fri, 09 Jan 2004 10:14:01 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AeyKY-000243-2F for icar@optimus.ietf.org; Fri, 09 Jan 2004 10:13:51 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id KAA03682 for ; Fri, 9 Jan 2004 10:13:42 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1AeyKQ-0006dn-00 for icar@ietf.org; Fri, 09 Jan 2004 10:13:38 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1AeyEB-0006LY-00 for icar@ietf.org; Fri, 09 Jan 2004 10:07:12 -0500 Received: from netcore.fi ([193.94.160.1]) by ietf-mx with esmtp (Exim 4.12) id 1Aey8s-00064C-00 for icar@ietf.org; Fri, 09 Jan 2004 10:01:43 -0500 Received: from localhost (pekkas@localhost) by netcore.fi (8.11.6/8.11.6) with ESMTP id i09F0tS30729; Fri, 9 Jan 2004 17:00:55 +0200 Date: Fri, 9 Jan 2004 17:00:55 +0200 (EET) From: Pekka Savola To: Alex Zinin cc: icar@ietf.org Subject: Re: [Icar] ICAR draft charter rev 2 In-Reply-To: <196108581021.20040108171716@psg.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.0 required=5.0 tests=none autolearn=no version=2.60 On Thu, 8 Jan 2004, Alex Zinin wrote: > Revision 2 below, please. Yes, this seems to be good. Even though I'm not sure whether forming the WG is necessarily the required condition for progressing Improved Cross-Area Review, it is probably a sufficient, and the most logical one (unless the IESG just decided to adopt a model, but as there was all kinds of weird pushback in Minneapolis, perhaps this is required :-). As for the milestones: FEB 2004: Submit drafts on improved community and structured reviews ==> what does "Submit" mean here? Submit as personal I-Ds? Publish as WG I-D's (if so, say so)? For what it's worth, I'm not 100% sure we can decide, with this aggressive timescale, which of the current individual submissions would be best adopted as WG items (or combined as new proposals to be submitted as WG items)? Maybe you could clarify what kind of process you had in mind here? SEP 2004: Submit draft on improved community review to the IESG for publication as BCP SEP 2004: Submit draft on improved structured community review to the IESG for publication as BCP SEP 2005: Evaluate WG progress and potential; close or recharter ==> Was the last one meant to be SEP 200_4_? For what it's worth, it may be very difficult to find ways to improve community review as it is -- I don't believe there have been many proposals on that yet. Luckily, finding a solution for structured community review (e.g., by adding those responsibilities for reviewing documents) has already sparked at least a couple of good proposals, and progressing with that should be much easier. -- Pekka Savola "You each name yourselves king, yet the Netcore Oy kingdom bleeds." Systems. Networks. Security. -- George R.R. Martin: A Clash of Kings _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Fri Jan 9 15:25:21 2004 Received: from optimus.ietf.org ([132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA09234 for ; Fri, 9 Jan 2004 15:25:20 -0500 (EST) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1Af3Bb-0004JT-Aq for icar-archive@odin.ietf.org; Fri, 09 Jan 2004 15:24:51 -0500 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i09KOpRA016573 for icar-archive@odin.ietf.org; Fri, 9 Jan 2004 15:24:51 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1Af3Bb-0004JE-1Q for icar-web-archive@optimus.ietf.org; Fri, 09 Jan 2004 15:24:51 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA08865 for ; Fri, 9 Jan 2004 15:24:49 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1Af3BZ-000081-00 for icar-web-archive@ietf.org; Fri, 09 Jan 2004 15:24:49 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1Af2cb-0002Ol-00 for icar-web-archive@ietf.org; Fri, 09 Jan 2004 14:48:45 -0500 Received: from [132.151.1.19] (helo=optimus.ietf.org) by ietf-mx with esmtp (Exim 4.12) id 1Af22S-0000un-00 for icar-web-archive@ietf.org; Fri, 09 Jan 2004 14:11:20 -0500 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1Af229-0002Ft-NR; Fri, 09 Jan 2004 14:11:01 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1Af222-0002Fe-H0 for icar@optimus.ietf.org; Fri, 09 Jan 2004 14:10:54 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA24336 for ; Fri, 9 Jan 2004 14:10:41 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1Af21k-0000sk-00 for icar@ietf.org; Fri, 09 Jan 2004 14:10:36 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1Af1JB-0000XJ-00 for icar@ietf.org; Fri, 09 Jan 2004 13:24:37 -0500 Received: from f070.brocade.com ([66.243.153.70] helo=blasphemy.brocade.com) by ietf-mx with esmtp (Exim 4.12) id 1Af0xg-0005o6-00 for icar@ietf.org; Fri, 09 Jan 2004 13:02:20 -0500 Received: from hq-ex-3.corp.brocade.com (hq-ex-3 [192.168.38.35]) by blasphemy.brocade.com (Postfix) with ESMTP id 1C3B71434E; Fri, 9 Jan 2004 10:01:23 -0800 (PST) X-MimeOLE: Produced By Microsoft Exchange V6.0.6249.0 content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Subject: RE: [Icar] ICAR draft charter rev 2 Date: Fri, 9 Jan 2004 10:01:23 -0800 Message-ID: Thread-Topic: [Icar] ICAR draft charter rev 2 Thread-Index: AcPWTyZlF2/AqrSZQHGqkeqCljJioQAioY7g From: "Robert Snively" To: "Alex Zinin" , Content-Transfer-Encoding: quoted-printable Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.0 required=5.0 tests=AWL autolearn=no version=2.60 Content-Transfer-Encoding: quoted-printable Content-Transfer-Encoding: quoted-printable I like this much better. I am assuming that "structured review" will include some work on=20 the review process itself, including perhaps things like the addressing of problems identified by the review, a formal response to the reviewers, solicitation of both "yes this is perfect" and "no this needs the following change" responses, etc. Is that correct? Should we make it explicit? Bob > -----Original Message----- > From: Alex Zinin [mailto:zinin@psg.com] > Sent: Thursday, January 08, 2004 5:17 PM > To: icar@ietf.org > Subject: [Icar] ICAR draft charter rev 2 >=20 >=20 >=20 > Revision 2 below, please. >=20 > --=20 > Alex >=20 > WG name: improved cross-area review (icar) >=20 > Chairs: >=20 > General Area Director(s): > Harald Alvestrand >=20 > General Area Advisor: > Harald Alvestrand > =20 > Mailing list: icar@ietf.org > Subscription: icar-request@ietf.org > Archives :=20 > https://www1.ietf.org/mail-archive/working-groups/icar/current > /maillist.html >=20 > WG Description: >=20 > The WG will work out mechanisms for improved cross-functional > review within the IETF, addressing the issues identified by the > PROBLEM WG. This includes a better community review, as well as > more structured pre-IESG review that may be used to improve > scalability of the IESG review function. >=20 > Definitions: >=20 > o Cross-functional review: document review covering different > aspects of Internet technology, including those represented by > WGs within different IETF areas. >=20 > o Community review: document review performed by individual IETF > participants and not caused by their responsibilities within > the IETF management structure. >=20 > o Structured review: more formal and role-based document review > performed by individuals assuming certain responsibilities (such > as by WG chairs, directorate and IESG members.) >=20 > It is an explicit goal of the WG to come up with mechanisms > encouraging earlier review of the documents. An early review is > best for catching architectural problems while they're still > relatively easy to solve. In particular, many cross-functional > interactions can be spotted and dealt with, thus avoiding many > "late surprises". A final review (currently done by the IESG) can > catch remaining cross-functional interactions, as well as deal > with overall quality issues. > =20 > The WG will cooperate with others in starting and evaluating > experiments with early community and structured reviews. The > evaluation of such experiments may be published as Informational > RFCs if the group so desires. >=20 > The WG will also coordinate with other WGs involved in the IETF > reform process on proposed changes to the IETF Working Group > operations and Standards process if those are considered > necessary. >=20 > WG milestones: >=20 > FEB 2004: Submit drafts on improved community and=20 > structured reviews > SEP 2004: Submit draft on improved community review to > the IESG for publication as BCP > SEP 2004: Submit draft on improved structured community review to > the IESG for publication as BCP > SEP 2005: Evaluate WG progress and potential; close or recharter >=20 >=20 > _______________________________________________ > Icar mailing list > Icar@ietf.org > https://www1.ietf.org/mailman/listinfo/icar >=20 >=20 _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Fri Jan 9 15:36:08 2004 Received: from optimus.ietf.org ([132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA12059 for ; Fri, 9 Jan 2004 15:36:08 -0500 (EST) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1Af3M3-0005PP-BS for icar-archive@odin.ietf.org; Fri, 09 Jan 2004 15:35:39 -0500 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i09KZdsT020785 for icar-archive@odin.ietf.org; Fri, 9 Jan 2004 15:35:39 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1Af3M3-0005P9-2Y for icar-web-archive@optimus.ietf.org; Fri, 09 Jan 2004 15:35:39 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA11885 for ; Fri, 9 Jan 2004 15:35:37 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1Af3M1-0002gP-00 for icar-web-archive@ietf.org; Fri, 09 Jan 2004 15:35:37 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1Af3HY-0001Tf-00 for icar-web-archive@ietf.org; Fri, 09 Jan 2004 15:31:02 -0500 Received: from [132.151.1.19] (helo=optimus.ietf.org) by ietf-mx with esmtp (Exim 4.12) id 1Af3An-0007dP-00 for icar-web-archive@ietf.org; Fri, 09 Jan 2004 15:24:01 -0500 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1Af3An-0004CF-Sh; Fri, 09 Jan 2004 15:24:01 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1Af3Ak-0004Ag-6l for icar@optimus.ietf.org; Fri, 09 Jan 2004 15:23:58 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA08405 for ; Fri, 9 Jan 2004 15:23:56 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1Af3Ai-0007ba-00 for icar@ietf.org; Fri, 09 Jan 2004 15:23:57 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1Af2Yb-0000ir-00 for icar@ietf.org; Fri, 09 Jan 2004 14:44:35 -0500 Received: from m106.maoz.com ([205.167.76.9]) by ietf-mx with esmtp (Exim 4.12) id 1Af1mG-0006nL-00 for icar@ietf.org; Fri, 09 Jan 2004 13:54:36 -0500 Received: from m106.maoz.com (localhost.localdomain [127.0.0.1]) by m106.maoz.com (8.12.10/8.12.10) with ESMTP id i09Iro6f012654; Fri, 9 Jan 2004 10:53:50 -0800 Received: (from dmm@localhost) by m106.maoz.com (8.12.10/8.12.10/Submit) id i09IrocR012653; Fri, 9 Jan 2004 10:53:50 -0800 X-Authentication-Warning: m106.maoz.com: dmm set sender to dmm@1-4-5.net using -f Date: Fri, 9 Jan 2004 10:53:50 -0800 From: David Meyer To: Pekka Savola Cc: Alex Zinin , icar@ietf.org Subject: Re: [Icar] ICAR draft charter rev 2 Message-ID: <20040109185350.GA12600@1-4-5.net> References: <196108581021.20040108171716@psg.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.1i X-public-key: http://www.1-4-5.net/~dmm/public-key.asc X-philosophy: "I just had to let it go" -- John Lennon Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.0 required=5.0 tests=AWL autolearn=no version=2.60 On Fri, Jan 09, 2004 at 05:00:55PM +0200, Pekka Savola wrote: >> On Thu, 8 Jan 2004, Alex Zinin wrote: >> > Revision 2 below, please. >> >> Yes, this seems to be good. >> >> Even though I'm not sure whether forming the WG is necessarily the >> required condition for progressing Improved Cross-Area Review, it is >> probably a sufficient, and the most logical one (unless the IESG just >> decided to adopt a model, but as there was all kinds of weird pushback >> in Minneapolis, perhaps this is required :-). >> >> As for the milestones: >> >> FEB 2004: Submit drafts on improved community and structured reviews >> >> ==> what does "Submit" mean here? Submit as personal I-Ds? Publish >> as WG I-D's (if so, say so)? As we just had out over on the MBONED list, we might consider using language such as "Publish I-D" to mean what I think you mean by "Submit drafts" (as WG document), and perhaps use "Submit to IESG..." to mean that an existing draft with be ready to go to the IESG? Dave _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Fri Jan 9 18:20:25 2004 Received: from optimus.ietf.org ([132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id SAA28430 for ; Fri, 9 Jan 2004 18:20:25 -0500 (EST) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1Af5v4-00050j-QI for icar-archive@odin.ietf.org; Fri, 09 Jan 2004 18:19:59 -0500 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i09NJwxo019255 for icar-archive@odin.ietf.org; Fri, 9 Jan 2004 18:19:58 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1Af5v4-00050U-Ey for icar-web-archive@optimus.ietf.org; Fri, 09 Jan 2004 18:19:58 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id SAA28325 for ; Fri, 9 Jan 2004 18:19:54 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1Af5v1-0003bC-00 for icar-web-archive@ietf.org; Fri, 09 Jan 2004 18:19:55 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1Af5t4-0003T2-00 for icar-web-archive@ietf.org; Fri, 09 Jan 2004 18:17:55 -0500 Received: from [132.151.1.19] (helo=optimus.ietf.org) by ietf-mx with esmtp (Exim 4.12) id 1Af5rE-0003OG-00 for icar-web-archive@ietf.org; Fri, 09 Jan 2004 18:16:00 -0500 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1Af5rG-0004wY-9P; Fri, 09 Jan 2004 18:16:02 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1Af5rE-0004w0-H1 for icar@optimus.ietf.org; Fri, 09 Jan 2004 18:16:00 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id SAA28100 for ; Fri, 9 Jan 2004 18:15:56 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1Af5rB-0003Nj-00 for icar@ietf.org; Fri, 09 Jan 2004 18:15:57 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1Af5pH-0003K4-00 for icar@ietf.org; Fri, 09 Jan 2004 18:14:00 -0500 Received: from colt-na165.alcatel.fr ([62.23.212.165] helo=smail.alcatel.fr) by ietf-mx with esmtp (Exim 4.12) id 1Af5og-0003GU-00 for icar@ietf.org; Fri, 09 Jan 2004 18:13:23 -0500 Received: from bemail05.netfr.alcatel.fr (bemail05.netfr.alcatel.fr [155.132.251.11]) by smail.alcatel.fr (ALCANET/NETFR) with ESMTP id i09NDLbN029356; Sat, 10 Jan 2004 00:13:21 +0100 Received: from alcatel.be ([138.203.64.171]) by bemail05.netfr.alcatel.fr (Lotus Domino Release 5.0.11) with ESMTP id 2004011000132061:93 ; Sat, 10 Jan 2004 00:13:20 +0100 Message-ID: <3FFF35DF.1080400@alcatel.be> Date: Sat, 10 Jan 2004 00:14:39 +0100 From: Dimitri.Papadimitriou@alcatel.be User-Agent: Mozilla/5.0 (Windows; U; WinNT4.0; en-US; rv:1.5) Gecko/20030925 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Alex Zinin CC: icar@ietf.org Subject: Re: [Icar] ICAR draft charter rev 2 References: In-Reply-To: X-MIMETrack: Itemize by SMTP Server on BEMAIL05/BE/ALCATEL(Release 5.0.11 |July 24, 2002) at 01/10/2004 00:13:20, Serialize by Router on BEMAIL05/BE/ALCATEL(Release 5.0.11 |July 24, 2002) at 01/10/2004 00:13:21, Serialize complete at 01/10/2004 00:13:21 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=us-ascii; format=flowed X-Alcanet-MTA-scanned-and-authorized: yes Content-Transfer-Encoding: 7bit Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.3 required=5.0 tests=AWL,NO_REAL_NAME autolearn=no version=2.60 Content-Transfer-Encoding: 7bit Content-Transfer-Encoding: 7bit alex, this revision of the charter proposal looks pretty good in the last sentence, the word "potential" in the last sentence might be further detailed thanks, - dimitri. >>-----Original Message----- >>From: Alex Zinin [mailto:zinin@psg.com] >>Sent: Thursday, January 08, 2004 5:17 PM >>To: icar@ietf.org >>Subject: [Icar] ICAR draft charter rev 2 >> >> >> >>Revision 2 below, please. >> >>-- >>Alex >> >> WG name: improved cross-area review (icar) >> >> Chairs: >> >> General Area Director(s): >> Harald Alvestrand >> >> General Area Advisor: >> Harald Alvestrand >> >> Mailing list: icar@ietf.org >> Subscription: icar-request@ietf.org >> Archives : >>https://www1.ietf.org/mail-archive/working-groups/icar/current >>/maillist.html >> >> WG Description: >> >> The WG will work out mechanisms for improved cross-functional >> review within the IETF, addressing the issues identified by the >> PROBLEM WG. This includes a better community review, as well as >> more structured pre-IESG review that may be used to improve >> scalability of the IESG review function. >> >> Definitions: >> >> o Cross-functional review: document review covering different >> aspects of Internet technology, including those represented by >> WGs within different IETF areas. >> >> o Community review: document review performed by individual IETF >> participants and not caused by their responsibilities within >> the IETF management structure. >> >> o Structured review: more formal and role-based document review >> performed by individuals assuming certain responsibilities (such >> as by WG chairs, directorate and IESG members.) >> >> It is an explicit goal of the WG to come up with mechanisms >> encouraging earlier review of the documents. An early review is >> best for catching architectural problems while they're still >> relatively easy to solve. In particular, many cross-functional >> interactions can be spotted and dealt with, thus avoiding many >> "late surprises". A final review (currently done by the IESG) can >> catch remaining cross-functional interactions, as well as deal >> with overall quality issues. >> >> The WG will cooperate with others in starting and evaluating >> experiments with early community and structured reviews. The >> evaluation of such experiments may be published as Informational >> RFCs if the group so desires. >> >> The WG will also coordinate with other WGs involved in the IETF >> reform process on proposed changes to the IETF Working Group >> operations and Standards process if those are considered >> necessary. >> >> WG milestones: >> >> FEB 2004: Submit drafts on improved community and >>structured reviews >> SEP 2004: Submit draft on improved community review to >> the IESG for publication as BCP >> SEP 2004: Submit draft on improved structured community review to >> the IESG for publication as BCP >> SEP 2005: Evaluate WG progress and potential; close or recharter >> >> >>_______________________________________________ >>Icar mailing list >>Icar@ietf.org >>https://www1.ietf.org/mailman/listinfo/icar >> >> > > > _______________________________________________ > Icar mailing list > Icar@ietf.org > https://www1.ietf.org/mailman/listinfo/icar -- Papadimitriou Dimitri E-mail : dimitri.papadimitriou@alcatel.be E-mail : dpapadimitriou@psg.com Webpage: http://psg.com/~dpapadimitriou/ Address: Fr. Wellesplein 1, B-2018 Antwerpen, Belgium Phone : +32 3 240-8491 _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Fri Jan 9 20:04:23 2004 Received: from optimus.ietf.org ([132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id UAA02627 for ; Fri, 9 Jan 2004 20:04:23 -0500 (EST) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1Af7Xe-0000yh-87 for icar-archive@odin.ietf.org; Fri, 09 Jan 2004 20:03:54 -0500 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i0A13scO003758 for icar-archive@odin.ietf.org; Fri, 9 Jan 2004 20:03:54 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1Af7Xe-0000yX-2K for icar-web-archive@optimus.ietf.org; Fri, 09 Jan 2004 20:03:54 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id UAA02594 for ; Fri, 9 Jan 2004 20:03:52 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1Af7Xc-0001g7-00 for icar-web-archive@ietf.org; Fri, 09 Jan 2004 20:03:52 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1Af7Vi-0001bK-00 for icar-web-archive@ietf.org; Fri, 09 Jan 2004 20:01:54 -0500 Received: from [132.151.1.19] (helo=optimus.ietf.org) by ietf-mx with esmtp (Exim 4.12) id 1Af7Tu-0001X8-00 for icar-web-archive@ietf.org; Fri, 09 Jan 2004 20:00:02 -0500 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1Af7Tt-0000tf-Fu; Fri, 09 Jan 2004 20:00:01 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1Af7Cb-0000Xy-KR for icar@optimus.ietf.org; Fri, 09 Jan 2004 19:42:09 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id TAA02010 for ; Fri, 9 Jan 2004 19:42:06 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1Af7CZ-0000ne-00 for icar@ietf.org; Fri, 09 Jan 2004 19:42:07 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1Af7Aj-0000i8-00 for icar@ietf.org; Fri, 09 Jan 2004 19:40:13 -0500 Received: from key1.docomolabs-usa.com ([216.98.102.225] helo=fridge.docomolabs-usa.com ident=fwuser) by ietf-mx with esmtp (Exim 4.12) id 1Af7AA-0000by-00; Fri, 09 Jan 2004 19:39:38 -0500 Message-ID: <029201c3d712$48293870$606015ac@dclkempt40> From: "James Kempf" To: , "MPowr" , Date: Fri, 9 Jan 2004 16:40:01 -0800 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Transfer-Encoding: 7bit Subject: [Icar] Summary of Discussion on Reforming IETF Quality Control Process Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.0 required=5.0 tests=AWL autolearn=no version=2.60 Content-Transfer-Encoding: 7bit Content-Transfer-Encoding: 7bit I've written a draft summarizing the discussions on reforming the IETF quality control process that started on mpowr in December and wandered to solutions in January. It's here: http://www.geocities.com/kempf42/draft-list-quality-control-00.txt I am not exactly sure what to do with the draft at this point, but would be interested in hearing feedback. There has been a preference expressed for discussing the issue on solutions rather than on either mpowr or icar, but I'm on all three lists and I'll try to respond as time allows. I'd also be interested in hearing suggestions about how to proceed. If this doesn't look like a valuable activity, we should apply the procedure described in the draft and terminate it early. :-) jak _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Fri Jan 9 20:54:30 2004 Received: from optimus.ietf.org ([132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id UAA05015 for ; Fri, 9 Jan 2004 20:54:30 -0500 (EST) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1Af8K9-0002mm-QZ for icar-archive@odin.ietf.org; Fri, 09 Jan 2004 20:54:01 -0500 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i0A1s1sq010708 for icar-archive@odin.ietf.org; Fri, 9 Jan 2004 20:54:01 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1Af8K9-0002md-HR for icar-web-archive@optimus.ietf.org; Fri, 09 Jan 2004 20:54:01 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id UAA05002 for ; Fri, 9 Jan 2004 20:53:59 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1Af8K7-0004Qs-00 for icar-web-archive@ietf.org; Fri, 09 Jan 2004 20:53:59 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1Af8IK-0004NH-00 for icar-web-archive@ietf.org; Fri, 09 Jan 2004 20:52:09 -0500 Received: from [132.151.1.19] (helo=optimus.ietf.org) by ietf-mx with esmtp (Exim 4.12) id 1Af8HE-0004Io-00 for icar-web-archive@ietf.org; Fri, 09 Jan 2004 20:51:00 -0500 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1Af8HF-0002d3-G8; Fri, 09 Jan 2004 20:51:01 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1Af8GJ-0002a1-J4 for icar@optimus.ietf.org; Fri, 09 Jan 2004 20:50:03 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id UAA04885 for ; Fri, 9 Jan 2004 20:50:01 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1Af8GH-0004Gt-00 for icar@ietf.org; Fri, 09 Jan 2004 20:50:01 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1Af8EY-0004BC-00 for icar@ietf.org; Fri, 09 Jan 2004 20:48:15 -0500 Received: from smtp.exodus.net ([66.35.230.236]) by ietf-mx with esmtp (Exim 4.12) id 1Af8Cs-0003yg-00; Fri, 09 Jan 2004 20:46:30 -0500 Received: from ms101.mail1.com (ms101.mail1.com [209.1.5.174]) by smtp.exodus.net (8.12.8/8.12.8) with ESMTP id i0A3Prw3006651; Fri, 9 Jan 2004 19:25:53 -0800 Received: from ala-mrwtemp.thingmagic.com (unverified [24.61.30.237]) by accounting.espmail.com (Rockliffe SMTPRA 5.2.5) with ESMTP id ; Fri, 9 Jan 2004 17:46:00 -0800 Message-Id: <5.1.0.14.2.20040109203410.04552a28@ms101.mail1.com> X-Sender: margaret@thingmagic.com@ms101.mail1.com X-Mailer: QUALCOMM Windows Eudora Version 5.1 Date: Fri, 09 Jan 2004 20:41:09 -0500 To: "James Kempf" From: Margaret Wasserman Cc: , "MPowr" , In-Reply-To: <029201c3d712$48293870$606015ac@dclkempt40> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed Subject: [Icar] Re: [Solutions] Summary of Discussion on Reforming IETF Quality Control Process Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.0 required=5.0 tests=AWL autolearn=no version=2.60 Hi James, I just read your proposal. Thanks for posting it. Although I may have more detailed questions later, I have a couple of quick clarification questions now... What are the criteria for deciding whether a particular document requires full IESG review? And, when would this be decided -- at charter time, when the document is accepted as a WG item, during WG Last Call, when the shepherding AD reviews the status of the document, or at some other time? In dispute resolution, you seem to indicate that the shepherding AD will take all disputes to the full IESG. Currently, the shepherding AD (or two ADs in an area) make an attempt to resolve any dispute before it is brought to the full IESG. Is it your intention to change that? Do you have any sort of transition plan for how we could move to this approach? Are there useful experiments that we could run to see if it would work? Margaret At 04:40 PM 1/9/2004 -0800, James Kempf wrote: >I've written a draft summarizing the discussions on reforming the IETF >quality control process that started on mpowr in December and wandered to >solutions in January. It's here: > >http://www.geocities.com/kempf42/draft-list-quality-control-00.txt > >I am not exactly sure what to do with the draft at this point, but would be >interested in hearing feedback. There has been a preference expressed for >discussing the issue on solutions rather than on either mpowr or icar, but >I'm on all three lists and I'll try to respond as time allows. I'd also be >interested in hearing suggestions about how to proceed. If this doesn't look >like a valuable activity, we should apply the procedure described in the >draft and terminate it early. :-) > > jak > >_______________________________________________ >Solutions mailing list >Solutions@alvestrand.no >http://eikenes.alvestrand.no/mailman/listinfo/solutions _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Fri Jan 9 21:32:25 2004 Received: from optimus.ietf.org ([132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id VAA05707 for ; Fri, 9 Jan 2004 21:32:25 -0500 (EST) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1Af8ur-00048r-JA for icar-archive@odin.ietf.org; Fri, 09 Jan 2004 21:31:57 -0500 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i0A2VvL4015915 for icar-archive@odin.ietf.org; Fri, 9 Jan 2004 21:31:57 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1Af8ur-00048c-FH for icar-web-archive@optimus.ietf.org; Fri, 09 Jan 2004 21:31:57 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id VAA05688 for ; Fri, 9 Jan 2004 21:31:54 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1Af8uo-0005p2-00 for icar-web-archive@ietf.org; Fri, 09 Jan 2004 21:31:54 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1Af8sw-0005lC-00 for icar-web-archive@ietf.org; Fri, 09 Jan 2004 21:29:59 -0500 Received: from [132.151.1.19] (helo=optimus.ietf.org) by ietf-mx with esmtp (Exim 4.12) id 1Af8r1-0005hW-00 for icar-web-archive@ietf.org; Fri, 09 Jan 2004 21:27:59 -0500 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1Af8r2-00041f-Rp; Fri, 09 Jan 2004 21:28:00 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1Af8r1-00041O-2k for icar@optimus.ietf.org; Fri, 09 Jan 2004 21:27:59 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id VAA05644 for ; Fri, 9 Jan 2004 21:27:56 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1Af8qy-0005gv-00 for icar@ietf.org; Fri, 09 Jan 2004 21:27:56 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1Af8p6-0005eD-00 for icar@ietf.org; Fri, 09 Jan 2004 21:26:01 -0500 Received: from psg.com ([147.28.0.62] ident=mailnull) by ietf-mx with esmtp (Exim 4.12) id 1Af8oe-0005av-00 for icar@ietf.org; Fri, 09 Jan 2004 21:25:32 -0500 Received: from [147.28.0.62] (helo=127.0.0.1) by psg.com with esmtp (Exim 4.24; FreeBSD) id 1Af8oe-000NYO-MC for icar@ietf.org; Sat, 10 Jan 2004 02:25:32 +0000 Date: Fri, 9 Jan 2004 18:22:44 -0800 From: Alex Zinin X-Mailer: The Bat! (v1.62i) Personal Reply-To: Alex Zinin X-Priority: 3 (Normal) Message-ID: <12198909396.20040109182244@psg.com> To: icar@ietf.org Subject: Re: [Icar] ICAR draft charter rev 2 In-Reply-To: References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Content-Transfer-Encoding: 7bit Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.4 required=5.0 tests=AWL,RCVD_NUMERIC_HELO autolearn=no version=2.60 Content-Transfer-Encoding: 7bit Content-Transfer-Encoding: 7bit Pekka, Robert, Dimitri- Thanks for the comments, see below, please. Friday, January 9, 2004, 7:00:55 AM, Pekka Savola wrote: [...] > As for the milestones: > FEB 2004: Submit drafts on improved community and structured reviews ==>> what does "Submit" mean here? Submit as personal I-Ds? Publish > as WG I-D's (if so, say so)? Since this is a WG milestone, a WG document is implied. Strictly speaking, we don't publish, the secretariat does, after the authors/editors submit. > For what it's worth, I'm not 100% sure we can decide, with this > aggressive timescale, which of the current individual submissions > would be best adopted as WG items (or combined as new proposals to be > submitted as WG items)? Maybe you could clarify what kind of process > you had in mind here? I am not sure either, but we should try to get something ready by Seoul for a substantial discussion. The way I see it working (though I'm open to other ideas) is we do the analysis of problems with each type of review at hand, decide which we problem we can solve, which we can't, then go back to the features of the proposals on the table, compare and discuss those and finally assemble solutions that are documented in WG documents edited by appointed editors. > SEP 2004: Submit draft on improved community review to > the IESG for publication as BCP > SEP 2004: Submit draft on improved structured community review to > the IESG for publication as BCP > SEP 2005: Evaluate WG progress and potential; close or recharter ==>> Was the last one meant to be SEP 200_4_? I think it was supposed to be JAN 2005--we need to give some time for the WG follow up on the possible IETF LC and IESG comments. Thanks for catching this, I'll update the text. > For what it's worth, it may be very difficult to find ways to improve > community review as it is -- I don't believe there have been many > proposals on that yet. Yes, I think the community review problem is not 100% solvable, mainly because we are dealing with sociological aspects, which we are not good at. On the other hand, if there's something we can improve, we should do it. > Luckily, finding a solution for structured community review (e.g., by > adding those responsibilities for reviewing documents) has already > sparked at least a couple of good proposals, and progressing with that > should be much easier. Indeed. BTW, the difference between these types of review and how they can potentially be addressed was my motivation behind splitting the work items. Friday, January 9, 2004, 10:01:23 AM, Robert Snively wrote: > I like this much better. > I am assuming that "structured review" will include some work on the > review process itself, including perhaps things like the addressing > of problems identified by the review, a formal response to the > reviewers, solicitation of both "yes this is perfect" and "no this > needs the following change" responses, etc. > Is that correct? Should we make it explicit? Yes, I would expect the final product to at least make suggestions on these aspects. However, I don't think details of this sort should be in the charter--I consider these part of the "mechanisms" called out in it. Friday, January 9, 2004, 3:14:39 PM, Dimitri.Papadimitriou@alcatel.be wrote: > alex, this revision of the charter proposal looks > pretty good > in the last sentence, the word "potential" in the > last sentence might be further detailed I suppose you mean in the last milestone. If so, this is just a standard alias to "do we have useful stuff to work on, or do we close?" Alex _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Sat Jan 10 00:36:28 2004 Received: from optimus.ietf.org ([132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id AAA10617 for ; Sat, 10 Jan 2004 00:36:27 -0500 (EST) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AfBmy-0000yy-JZ for icar-archive@odin.ietf.org; Sat, 10 Jan 2004 00:36:00 -0500 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i0A5a0fL003774 for icar-archive@odin.ietf.org; Sat, 10 Jan 2004 00:36:00 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AfBmx-0000yj-JH for icar-web-archive@optimus.ietf.org; Sat, 10 Jan 2004 00:35:59 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id AAA10595 for ; Sat, 10 Jan 2004 00:35:56 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1AfBmu-0006L6-00 for icar-web-archive@ietf.org; Sat, 10 Jan 2004 00:35:56 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1AfBkx-0006FI-00 for icar-web-archive@ietf.org; Sat, 10 Jan 2004 00:33:56 -0500 Received: from [132.151.1.19] (helo=optimus.ietf.org) by ietf-mx with esmtp (Exim 4.12) id 1AfBj5-00067s-00 for icar-web-archive@ietf.org; Sat, 10 Jan 2004 00:31:59 -0500 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AfBj6-0000uR-Me; Sat, 10 Jan 2004 00:32:00 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AfBLu-0000Ue-Gt for icar@optimus.ietf.org; Sat, 10 Jan 2004 00:08:02 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id AAA09680 for ; Sat, 10 Jan 2004 00:07:59 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1AfBLr-00052O-00 for icar@ietf.org; Sat, 10 Jan 2004 00:07:59 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1AfBK3-0004yn-00 for icar@ietf.org; Sat, 10 Jan 2004 00:06:08 -0500 Received: from key1.docomolabs-usa.com ([216.98.102.225] helo=fridge.docomolabs-usa.com ident=fwuser) by ietf-mx with esmtp (Exim 4.12) id 1AfBIx-0004v7-00; Sat, 10 Jan 2004 00:05:00 -0500 Message-ID: <003701c3d737$5b361530$386015ac@dclkempt40> From: "James Kempf" To: "Margaret Wasserman" Cc: , "MPowr" , References: <5.1.0.14.2.20040109203410.04552a28@ms101.mail1.com> Date: Fri, 9 Jan 2004 21:05:24 -0800 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Transfer-Encoding: 7bit Subject: [Icar] Re: [Solutions] Summary of Discussion on Reforming IETF Quality Control Process Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.0 required=5.0 tests=AWL autolearn=no version=2.60 Content-Transfer-Encoding: 7bit Content-Transfer-Encoding: 7bit Hi Margaret, That was quick! Here are a couple of quick answers, but if you think this draft might have legs, I'll get an issues page up on it. > What are the criteria for deciding whether a particular document > requires full IESG review? It is up to the IESG to decide that. The WG can ask for a full review if they think it might be necessary, but the IESG makes the decision. Also, the shepherding AD can ask for a full review if s(he) thinks its necessary. In general, I think the IESG would want to review drafts from WGs that are proposing new technology or major changes in old technology, but I did not think it would be such a good idea to be that explicit because it would decrease the attractiveness of serving on the review board, and, as a practical matter, the IESG may want at some point to have the review board do a review on some draft that proposes new technology if it's a small change or a major change in old technology if it isn't all the different or if the WG is composed of people who are known to be reliable and world class experts in the field. >And, when would this be decided -- at > charter time, when the document is accepted as a WG item, during > WG Last Call, when the shepherding AD reviews the status of the > document, or at some other time? > In the normal case, as soon as the draft showed up as a work item on the WG's goals list, which is what the process draft should say because some WG drafts are added after the charter has been written. A review would have to be done before the draft became a WG draft in any case. The idea here is to inject a note of predictability into the review process, so the chair, WG, and shepherding AD know exactly what to do when the time comes for the review. However, in exceptional cases, typically when the shepherding AD has detected that the WG is off track or has reason to believe that the solutions being discussed by the WG may be impractical or otherwise flawed, the shepherding AD can call for a review and include an IESG review of documents, if talking with the WG fails to reorient the WG. > In dispute resolution, you seem to indicate that the shepherding > AD will take all disputes to the full IESG. Currently, the > shepherding AD (or two ADs in an area) make an attempt to resolve > any dispute before it is brought to the full IESG. Is it your > intention to change that? > Yes, I believe that is probably a better idea and I will add that to the document. No reason for full IESG intervention if the WG and reviewers can come to an accommodation with the AD's help. > Do you have any sort of transition plan for how we could move > to this approach? Are there useful experiments that we could > run to see if it would work? > One way to start would be to pick a few WGs and give it a try. The list of SIRS are obvious candidates for initial Reviewers. Obviously, if the process is applied to some WGs that exist initially, they will not be able to set up the review process when they are initially formed. I'd say picking groups such as DHC where many of the drafts involve proposals for new options or other small changes would be one way to test the review board process, I'm not sure which ones would be appropriate for the new/deeply changed technology case. The experiment could be run for, say, a year and closely monitored to see how much work it really did eliminate for the IESG and whether it really did free up more time for ADs to spend working with their WGs, then tuned as appropriate or even discarded if it isn't working. jak _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Sat Jan 10 00:52:27 2004 Received: from optimus.ietf.org ([132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id AAA11162 for ; Sat, 10 Jan 2004 00:52:27 -0500 (EST) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AfC2S-0001cP-CJ for icar-archive@odin.ietf.org; Sat, 10 Jan 2004 00:52:00 -0500 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i0A5q05B006215 for icar-archive@odin.ietf.org; Sat, 10 Jan 2004 00:52:00 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AfC2S-0001cA-3Q for icar-web-archive@optimus.ietf.org; Sat, 10 Jan 2004 00:52:00 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id AAA11084 for ; Sat, 10 Jan 2004 00:51:56 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1AfC2O-0007EW-00 for icar-web-archive@ietf.org; Sat, 10 Jan 2004 00:51:57 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1AfC0T-00079R-00 for icar-web-archive@ietf.org; Sat, 10 Jan 2004 00:49:58 -0500 Received: from [132.151.1.19] (helo=optimus.ietf.org) by ietf-mx with esmtp (Exim 4.12) id 1AfByY-000757-00 for icar-web-archive@ietf.org; Sat, 10 Jan 2004 00:47:58 -0500 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AfBya-0001Wo-Ow; Sat, 10 Jan 2004 00:48:00 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AfByY-0001WY-Kf for icar@optimus.ietf.org; Sat, 10 Jan 2004 00:47:58 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id AAA10988 for ; Sat, 10 Jan 2004 00:47:55 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1AfByV-00074W-00 for icar@ietf.org; Sat, 10 Jan 2004 00:47:56 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1AfBwb-0006zs-00 for icar@ietf.org; Sat, 10 Jan 2004 00:45:58 -0500 Received: from measurement-factory.com ([206.168.0.5]) by ietf-mx with esmtp (Exim 4.12) id 1AfBuh-0006tJ-00; Sat, 10 Jan 2004 00:44:00 -0500 Received: from measurement-factory.com (localhost [127.0.0.1]) by measurement-factory.com (8.12.9/8.12.9) with ESMTP id i0A5hvk3094168; Fri, 9 Jan 2004 22:43:57 -0700 (MST) (envelope-from rousskov@measurement-factory.com) Received: (from rousskov@localhost) by measurement-factory.com (8.12.9/8.12.9/Submit) id i0A5hvoK094167; Fri, 9 Jan 2004 22:43:57 -0700 (MST) (envelope-from rousskov) Date: Fri, 9 Jan 2004 22:43:57 -0700 (MST) From: Alex Rousskov To: James Kempf cc: Margaret Wasserman , MPowr , icar@ietf.org, solutions@alvestrand.no In-Reply-To: <003701c3d737$5b361530$386015ac@dclkempt40> Message-ID: References: <5.1.0.14.2.20040109203410.04552a28@ms101.mail1.com> <003701c3d737$5b361530$386015ac@dclkempt40> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Subject: [Icar] Re: [Solutions] Summary of Discussion on Reforming IETF Quality Control Process Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.0 required=5.0 tests=AWL autolearn=no version=2.60 On Fri, 9 Jan 2004, James Kempf wrote: > > What are the criteria for deciding whether a particular document > > requires full IESG review? > > It is up to the IESG to decide that. I want to clarify this because Margeret's question might be based on the wrong assumption, possibly causing misinterpretation of the correct answer. There should be no "requires full IESG review" flag or state for any document. IESG is not a special case when it comes to review. IESG or any single AD can submit a review for any document that is up for review, at any time. This is no different from any IETF participant submitting a review. If IESG feels that a particular document needs full IESG review, it is their internal business, invisible and unpredictable to others (in general), until they submit a review. In general, one does not know a priori whether a document will be reviewed by IESG until the IESG submits the review. Now, if there is a conflict that WG and a reviewer cannot resolve despite all the negotiation efforts, then the document automatically goes to IESG for the final conflict resolution. Such resolution may require full IESG review, but it is internal IESG business how to approach that. The final IESG decision is documented, of course. Note that "a reviewer" above may be IESG (but it is not a special case). Thus, IESG has special formal powers when it comes to conflict resolution, but does not have (and does not need!) any special powers or exceptions when it comes to document review. This scheme is both simple and gives IESG full flexibility when it comes to selecting "full IESG review" targets. We do not need to spend time documenting what those targets might be. Alex. _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Sat Jan 10 01:50:42 2004 Received: from optimus.ietf.org ([132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id BAA12518 for ; Sat, 10 Jan 2004 01:50:42 -0500 (EST) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AfCwm-0003dn-42 for icar-archive@odin.ietf.org; Sat, 10 Jan 2004 01:50:14 -0500 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i0A6oCee013992 for icar-archive@odin.ietf.org; Sat, 10 Jan 2004 01:50:12 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AfCwl-0003db-U5 for icar-web-archive@optimus.ietf.org; Sat, 10 Jan 2004 01:50:11 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id BAA12508 for ; Sat, 10 Jan 2004 01:50:09 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1AfCwi-0001rm-00 for icar-web-archive@ietf.org; Sat, 10 Jan 2004 01:50:08 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1AfCuz-0001nA-00 for icar-web-archive@ietf.org; Sat, 10 Jan 2004 01:48:22 -0500 Received: from [132.151.1.19] (helo=optimus.ietf.org) by ietf-mx with esmtp (Exim 4.12) id 1AfCte-0001h4-00 for icar-web-archive@ietf.org; Sat, 10 Jan 2004 01:46:58 -0500 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AfCth-0003Wk-5M; Sat, 10 Jan 2004 01:47:01 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AfCsn-0003Vd-Hp for icar@optimus.ietf.org; Sat, 10 Jan 2004 01:46:05 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id BAA12389 for ; Sat, 10 Jan 2004 01:46:02 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1AfCsk-0001dD-00 for icar@ietf.org; Sat, 10 Jan 2004 01:46:02 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1AfCqy-0001Ys-00 for icar@ietf.org; Sat, 10 Jan 2004 01:44:13 -0500 Received: from ns.execdsl.net ([208.184.15.238] helo=EXECDSL.COM) by ietf-mx with esmtp (Exim 4.12) id 1AfCpo-0001Ts-00; Sat, 10 Jan 2004 01:43:00 -0500 Received: from [66.95.38.74] (HELO JLaptop.stevecrocker.com) by EXECDSL.COM (CommuniGate Pro SMTP 3.3) with ESMTP id 6063177; Sat, 10 Jan 2004 01:42:52 -0500 Message-Id: <5.1.0.14.0.20040110013737.018ef758@localhost> X-Sender: joel@stevecrocker.com@localhost X-Mailer: QUALCOMM Windows Eudora Version 5.1 Date: Sat, 10 Jan 2004 01:42:43 -0500 To: MPowr , icar@ietf.org, solutions@alvestrand.no From: "Joel M. Halpern" In-Reply-To: References: <003701c3d737$5b361530$386015ac@dclkempt40> <5.1.0.14.2.20040109203410.04552a28@ms101.mail1.com> <003701c3d737$5b361530$386015ac@dclkempt40> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed Subject: [Icar] Re: [Solutions] Summary of Discussion on Reforming IETF Quality Control Process Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.0 required=5.0 tests=none autolearn=no version=2.60 I may be missing something, but the problem I see reading this is that this gives the IESG no assistance in deciding which documents it needs to review. There was another proposal which I read to say ~the IESG needs to review documents for which there is disagreement about the results of other reviews.~ While one can argue about whether that is good or not, it at least spells out when the IESG needs to review a document, and when it does not need to. Obviously, if we can lighten the load enough that the IESG has time to review more than it needs to, that is great and they should do so. Equally, some IESG members will review some documents over and above such a "requirement", since they will be interested. But my understanding of one aspect of this discussion is to help replace the current ~the IESG must review everything~ with a useful alternative review procedure, and a useful definition of which things the IESG needs to review. (Among other things, such a reduction should help enable the necessary IESG reviews to occur in a more timely fashion.) The text below seems to say taht there are no documents the IESG needs to review. While it would be nice to arrive in such a state, I doubt we can get there in one jump. Yours, Joel M. Halpern PS: Can we pick one of the three lists for this? At 10:43 PM 1/9/2004 -0700, Alex Rousskov wrote: >There should be no "requires full IESG review" flag or state for any >document. IESG is not a special case when it comes to review. IESG or >any single AD can submit a review for any document that is up for >review, at any time. This is no different from any IETF participant >submitting a review. If IESG feels that a particular document needs >full IESG review, it is their internal business, invisible and >unpredictable to others (in general), until they submit a review. In >general, one does not know a priori whether a document will be >reviewed by IESG until the IESG submits the review. _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Sat Jan 10 12:16:37 2004 Received: from optimus.ietf.org ([132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA11610 for ; Sat, 10 Jan 2004 12:16:37 -0500 (EST) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AfMiZ-0008BV-03 for icar-archive@odin.ietf.org; Sat, 10 Jan 2004 12:16:11 -0500 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i0AHGAAd031455 for icar-archive@odin.ietf.org; Sat, 10 Jan 2004 12:16:10 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AfMiY-0008BG-Se for icar-web-archive@optimus.ietf.org; Sat, 10 Jan 2004 12:16:10 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA11566 for ; Sat, 10 Jan 2004 12:16:07 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1AfMiX-0002Vb-00 for icar-web-archive@ietf.org; Sat, 10 Jan 2004 12:16:09 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1AfMge-0002QH-00 for icar-web-archive@ietf.org; Sat, 10 Jan 2004 12:14:13 -0500 Received: from [132.151.1.19] (helo=optimus.ietf.org) by ietf-mx with esmtp (Exim 4.12) id 1AfMfT-0002Nt-00 for icar-web-archive@ietf.org; Sat, 10 Jan 2004 12:12:59 -0500 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AfMfU-00084P-Kq; Sat, 10 Jan 2004 12:13:00 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AfMeg-00083F-KZ for icar@optimus.ietf.org; Sat, 10 Jan 2004 12:12:10 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA11537 for ; Sat, 10 Jan 2004 12:12:07 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1AfMef-0002Mj-00 for icar@ietf.org; Sat, 10 Jan 2004 12:12:09 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1AfMcl-0002JR-00 for icar@ietf.org; Sat, 10 Jan 2004 12:10:12 -0500 Received: from joy.songbird.com ([208.184.79.7]) by ietf-mx with esmtp (Exim 4.12) id 1AfMbM-0002F4-00 for icar@ietf.org; Sat, 10 Jan 2004 12:08:44 -0500 Received: from bbprime (jay.songbird.com [208.184.79.253]) by joy.songbird.com (8.11.6/8.11.6) with SMTP id i0AHFrc25784 for ; Sat, 10 Jan 2004 09:15:53 -0800 From: Dave Crocker To: Icar (E-mail) X-Mailer: PocoMail 3.03 (1740) - EVALUATION VERSION X-URL: http://www.pocomail.com/ Date: Sat, 10 Jan 2004 09:08:08 -0800 Message-ID: <2004110988.424309@bbprime> In-Reply-To: <7D5D48D2CAA3D84C813F5B154F43B155033D3AAA@nl0006exch001u.nl.lucent.com> Subject: Re: [Icar] Input based on SIRS experience Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Content-Transfer-Encoding: 7bit Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.1 required=5.0 tests=AWL autolearn=no version=2.60 Content-Transfer-Encoding: 7bit Content-Transfer-Encoding: 7bit Adding my own thoughts to the archive, since the topic came up, and with the hope it can added to the discussion: It was not much effort to get a core of reviewers signed up. This suggests a good core of community motivation to help. We have not needed more, so we do not know whether an effort like this can attract _enough_ qualified reviewers. I am not sure how to interpret the low number of review requests. It is easy to say that it is due to lack of community awareness or interest or trust in the SIRs effort. Certainly we have not done as much marketing as we might have wished. However this is view is confounded by the very slow metabolic rate of the IETF and the fact that the effort began at the beginning of a summer. No doubt there probably are other factors. Cliches like 'tipping point' and 'crossing the chasm' were created for just this issue. For example, it may be that it is more difficult to change existing working group efforts than it is to get integrated in with new(er) ones. Certainly we have not done any serious, sustained effort to get working groups to request reviews. On doing the administrative side of this effort, there are some things that I suspect are strategic, besides the relatively obvious fact that this really does take some ongoing, administrative effort, more than I had expected. (But, then, I always underestimate these things.) It needs to be easy to have a record of the review process for a document and, by derivation, an ability to see the whole set of review efforts. There should be a central point for tracking review efforts and a central point for accessing reviews. Besides the obvious, local and tactical benefit to the individual working group, this permits the community to get the ability to review the _set _of reviews and get a sense of the effort and result of doing reviews. In other words, it has marketing value. It makes it easy for others to see that there is review activity and it makes it easy for them to evaluate the nature and quality of those reviews. This will make them more comfortable getting reviews for their own efforts and will help them decide on what reviewers to ask for. Whether this needs to be a permanent, formal archive, like RFCs, an ephemeral, formal archive, like I-Ds, or an unofficial, ephemeral archive like a WG mailing list, has not been resolved. My own thought is that formal publication (RFC and even I-D) is far too much overhead. The only existing mechanism is the mailing list. Creating a new mechanism strikes me as too much effort, and added effort is a very large barrier to adoption of this process. Opening up the IETF Tracker to working group chairs, and adding the review construct and/or allowing free-form entries would help this enormously. (Free-form means that the IETF does not have to formally adopt the thing being tracked. This means reviews -- and other activities -- could be tracked much sooner than would be possible if we first have to get IETF-wide approval of the review process. I am not sure whether selection of reviewers, by requesters, needs to be different than we set up. A web page with a list of bios, and a mailing list for sending requests to is probably fine. The public discussion about reviewing shows that the community is still working on clarifying what types of reviews are needed, when, and how to use them. There is also the small matter of the SIRS reviews that have been done. Have they been they type that are needed? Should they be done differently? We have not tried to assess this. Certainly we cannot expect regular, widespread use of a review process until working groups know what to ask for, or at least what to expect they will get and how they should use the results. d/ -- Dave Crocker Brandenburg InternetWorking _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Sat Jan 10 12:20:52 2004 Received: from optimus.ietf.org ([132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA11765 for ; Sat, 10 Jan 2004 12:20:52 -0500 (EST) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AfMme-0008MC-Vk for icar-archive@odin.ietf.org; Sat, 10 Jan 2004 12:20:25 -0500 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i0AHKNMf032113 for icar-archive@odin.ietf.org; Sat, 10 Jan 2004 12:20:23 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AfMmd-0008Lr-0N for icar-web-archive@optimus.ietf.org; Sat, 10 Jan 2004 12:20:23 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA11734 for ; Sat, 10 Jan 2004 12:20:19 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1AfMmb-0002ku-00 for icar-web-archive@ietf.org; Sat, 10 Jan 2004 12:20:21 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1AfMki-0002dV-00 for icar-web-archive@ietf.org; Sat, 10 Jan 2004 12:18:26 -0500 Received: from [132.151.1.19] (helo=optimus.ietf.org) by ietf-mx with esmtp (Exim 4.12) id 1AfMjM-0002Yk-00 for icar-web-archive@ietf.org; Sat, 10 Jan 2004 12:17:00 -0500 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AfMjN-0008DR-7i; Sat, 10 Jan 2004 12:17:01 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AfMib-0008Be-D2 for icar@optimus.ietf.org; Sat, 10 Jan 2004 12:16:13 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA11573 for ; Sat, 10 Jan 2004 12:16:09 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1AfMiZ-0002Vt-00 for icar@ietf.org; Sat, 10 Jan 2004 12:16:11 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1AfMgg-0002QY-00 for icar@ietf.org; Sat, 10 Jan 2004 12:14:16 -0500 Received: from smtp.exodus.net ([66.35.230.236]) by ietf-mx with esmtp (Exim 4.12) id 1AfMfk-0002Np-00 for icar@ietf.org; Sat, 10 Jan 2004 12:13:17 -0500 Received: from ms101.mail1.com (ms101.mail1.com [209.1.5.174]) by smtp.exodus.net (8.12.8/8.12.8) with ESMTP id i0AIqWw3003021 for ; Sat, 10 Jan 2004 10:52:35 -0800 Received: from ala-mrwtemp.thingmagic.com (unverified [24.61.30.237]) by accounting.espmail.com (Rockliffe SMTPRA 5.2.5) with ESMTP id ; Sat, 10 Jan 2004 09:12:40 -0800 Message-Id: <5.1.0.14.2.20040110075158.0385bbd0@ms101.mail1.com> X-Sender: margaret@thingmagic.com@ms101.mail1.com X-Mailer: QUALCOMM Windows Eudora Version 5.1 Date: Sat, 10 Jan 2004 12:09:52 -0500 To: "James Kempf" From: Margaret Wasserman Cc: icar@ietf.org, solutions@alvestrand.no In-Reply-To: <003701c3d737$5b361530$386015ac@dclkempt40> References: <5.1.0.14.2.20040109203410.04552a28@ms101.mail1.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed Subject: [Icar] Re: [mpowr] Re: [Solutions] Summary of Discussion on Reforming IETF Quality Control Process Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.0 required=5.0 tests=AWL autolearn=no version=2.60 Hi James, [I'm only responding on solutions and icar (not mpowr). Most of this discussion happened on solutions, and your proposal is largely about review/approval structures, so I think it is more applicable to the proposed ICAR effort than to the proposed MPOWR effort.] I've had some time to think about (and re-read) your draft, so I now I will provide my feedback, rather than just clarifying questions. I do support the idea of a tiered review/approval structure within the IETF, but I have many concerns with the specifics of your proposal, several of which I also expressed regarding the SIRS proposal. IMO, any proposal in this area needs meet certain criteria for scalability, consistency, cross-area coverage, efficiency, manageability and accountability. I have some concerns with your proposal in each of these areas and I have explained them below. I also hope that these criteria and my explanations may be useful to assessing other review/approval proposals, as well. These concerns are what lead me to originally prefer the type of per-area review board that Alex Zinin has suggested, rather than a single large (SIRS-like) review board as you suggest. I believe that it would be possible to give single ADs the ability to approve non-critical documents (criteria TBD) based on reviews by all of the per-area boards -- in other words, I think that we can somewhat consider the structure/management of the review board(s) to be orthogonal to whether some documents can be approved without full IESG review. So, on to the criteria... SCALABILITY =========== Like the SIRS proposal, I believe that this proposal is quite naive about the scale of this particular problem. Let's take a few numbers: We currently approve about 200 RFCs per year. Each of these RFCs receives (on average) ~2-1/2 review cycles from the IESG plus a full AD Review. So, let's assume that we will continue to produce 200 documents per year, and that each will be subjected to 3 cross-area review cycles. BTW, if the same group of people reviews the document all three times (also see section on consistency below), the later reviews will take much less time than the earlier reviews. The IESG consists of 13 people, not all of whom carefully review each document during IESG review (for various reason). So, let's assume that we can get adequate cross-area coverage (see section on cross-area coverage) by having each document reviewed by 8 properly selected members of the Quality Review Board. That's the number of ADs that it currently takes (today, with one slot empty) to approve publication of a document. So, the number of individual document reviews required would be 200 * 3 * 8 == 4800. Let's assume that we can find 200 people who are willing and qualified to serve on the Quality Review Board. I don't know that this is possible, and it means that someone will have to manage a function that involves 200 people (see manageability section below), but let's assume... In that case, each member of the review board would need to do an average of 24 reviews per year -- so a minimum of 3 is misleadingly low. Ideally, this means that each member of the review board would do a full 3-cycle review for 8 documents per year. If we expect this system to result in a 50% improvement in document throughput, we will need to handle 300 documents per year, which requires 36 reviews/board member/year (or 12 documents). Doable? Maybe. If we can find 200 people willing to do this, figure out how to train and organize them (see sections on preparation/training and management below), and sustain that number over time. The sign-up rate for SIRS does not make me confident that this is possible, but we could try.... CONSISTENCY =========== There are two types of consistency that are important for IETF document reviews: (1) Consistency across the different reviews done for the same document (to avoid thrash). (2) Using a consistent set of acceptance criteria for each document that is reviewed/approved. Although we don't always achieve both type of consistency today (due to personnel changes on the IESG, for example), we usually do quite well in this area. The fact that the same group of people reviews/approves every document is heavily optimized for consistency (over efficiency, scalability, etc.), and because we don't suffer too much is this area today, it is easy to underrate the benefits and importance of consistency. Type (1) consistency could be achieved by having the same set of reviewers (to whatever extent possible) perform all of the reviews for a given document. In other words, each document would have its own mini-board of reviewers (8 reviewers, in my previous example) who would do the initial review and perform whatever subsequent reviews are needed to determine that new versions of the document correct problems found in earlier reviews and do not introduce new problems. So, this is fairly easily handled, assuming that we have a manageable method for assigning reviewers to a particular document and replacing them as needed, etc. (see manageability section below). Type (2) consistency is _much_ harder to achieve across a large board (~200 people in my example above) than it is with a group of 13 people who spend several hours per week on the phone with each other, hold retreats and communicate daily... In order to achieve even a reasonable level of consistency across a large group (200 people?), I believe that we would need most or all of the following things: - Written review criteria for each type of document. - Documented per-area review criteria (like the MIB nits) for every area or technical sub-area. - A record of architectural/policy decisions that were made that can be searched and used by subsequent reviewers. - A mandatory preparation/training program (in-person or on-line) for all new members of the review board. - Some type of mentoring or monitoring program for new members during the first NN reviews (most easily achieved, perhaps, by having some sort of structure within the board?). Without a plan (including committed resources) to achieve these things, I believe that a large review board would devolve into chaos. There may also be an issue with documents that are passed to the IESG for final approval. The IESG's review may not be consistent with earlier reviews, perhaps resulting in "late surprises" or demotivation of the WG. This problem is most likely in a situation where the IESG has no influence over what reviewers are chosen to do the initial review, and where the IESG does not feel accountable for the quality/correctness of the initial review (see section on accountability below). CROSS-AREA COVERAGE =================== The IESG believes that Cross-Area Review is a core value of the IETF, as well as one of the key properties that differentiates us from other standards bodies. Obviously the folks involved in developing this plan think so, too, because otherwise we would simply ship documents based on WG technical review. However, this proposal doesn't seem to do much to support or facilitate cross-area review. In order for a document to receive sufficient cross-area review, it is not sufficient to establish a pool of (perhaps 200) reviewers and ensure that the document is reviewed by a certain number (perhaps 8) of those reviewers. We would need to make sure that the document is reviewed by a range of people who, together, have an appropriate breadth of expertise to provide adequate cross-area coverage. It isn't easy to put together a small group of people who can cover all of the technology areas represented in the IETF (as I'm sure any nomcom member could tell you), and it certainly isn't possible to populate the Quality Review Team with people who each have knowledge of the whole set of IETF technologies and their possible interactions. As far as I know, there isn't anyone who could provide a full cross-area review alone (although Allison can do a passable imitation) -- that's why we have documents reviewed by IESG members from all areas before they are approved. A lot of details are swept under the rug of the statement "The Working Group Chair is responsible for ensuring that all work done by the Working Group receives adequate and timely review". How are you picturing that this would work? How would the WG chair identify the areas of expertise represented by each reviewer and gauge their level of expertise in each area? How would s/he make sure that the appropriate inter-area issues are adequately covered by the chosen reviewers? This proposal is largely predicated on the belief that WG chairs should not be responsible for document quality because their involvement in the technology may blind them to its technical flaws. However, it seems to assume that WG chairs will be aware of all possible interactions that this technology may have with other IETF areas and be able (and willing) to solicit review feedback from the right sort of people to uncover all of those problems, which IMO won't work. For example, if the WG is unaware that their work has major impact on the application layer, the WG chair may not think it is important that the work be reviewed by an applications- knowledgeable reviewer! This seems, to me, to be a pretty big flaw. Per-area review boards have a major advantage in this area. If a document is passed by every per-area review board (particularly if those boards are carefully chosen to cover their area well and provide coverage for inter-area gaps), with the reviewers chosen by the ADs or other area review board leaders who aren't active in the WG, then it is more likely that a document will have received adequate cross-area review than if it is reviewed by a set of reviewers chosen from a large pool by the WG chair. EFFICIENCY ========== One goal of any new review/approval process should be to increase our document throughput -- publishing documents faster than we do today, without a reduction in quality. In general, we should try to avoid a situation where a document needs to go through two (possibly inconsistent, see above) review/approval processes. This proposal might achieve this goal for some documents, particularly those that do not require IESG review. However, the most important documents might be subjected to two rounds of review/approval -- one by the Quality Review Board and another by the IESG. This issue might be resolved by better defining how/when a decision is made regarding whether a document will be reviewed/approved by the Quality Review Board or the IESG. MANAGEABILITY ============= In order to run an orderly review process, to assign reviewers with appropriate (combinations of) expertise, to adquately training reviewers and to maintain the quality of the review process, the Quality Review Board would need to be managed. This proposal does not indicate how the board would be managed, who would manage it, how they would be chosen, etc. This is a major omission that makes it difficult to evaluate many aspects of this proposal. Passive voice is used throughout this document, and that tends to obscure the fact that there is no management structure defined. For example, it says "Reviewers can be assigned to particular Working Groups and have their names listed on the Working Group Web page as technical advisors". By whom? By the WG chair? This document does not mention any preparation or training for members of the Quality Review Board, nor does it specify any period during which a member's initial reviews will be monitored by experienced members, etc. I think that some type of preparation/training is essential to asking people to assume any role in the IETF process. The per-area review board proposal relies on the ADs to select reviewers and manage the process as appropriate to each area. The AD would be responsible for ensuring that members of the area boards are adequately trained and prepared, and would be held accountable for the quality of their work. ACCOUNTABILITY ============== This document seems to be based on the concept that quality review is performed for the WGs, and that the review process should be accountable to the WGs. Ultimately, reviewers report to WG chairs, because they if they are not asked (by WG chairs) to review at least 3 document per year, they are removed. I believe that is wrong... The current AD Review/IETF Last Call/IESG Review system is not intended to be accountable to a WG. The IESG (as a review board) exists outside of the WG structure, as a group that is accountable to the community. AD/IESG reviews and IETF Last Calls serve as a mechanism for the interests of the IETF community (and perhaps the Internet as a whole) to be balanced against the interests of a particular WG, and I don't see this balance adequately represented in this proposal, at least not for documents that don't receive full IESG review. I realize that some members of this discussion would like to move away from the hierarchical structure represented by the current IESG. I'm not sure I agree with that approach, but I'm also not sure that I'm against it. So far, though, I have seen any proposal that gives the IETF/Internet community a strong enough voice that doesn't involve giving some authority to some selected representatives of that community... This doesn't mean that the IESG is the only group that can review/approve documents, and I do strongly believe that we should build a more scalable review/approval structure that does not rely solely on the IESG. What it does mean (to me) is that reviewers should be chosen, managed, evaluated and held accountable by someone other than WG chairs. This leads to two choices: - Have the IESG manage the review board(s) and the review/approval process. - Build a different group to manage this process that is selected by and accountable to the community. The second choice has some known weaknesses. In the past, we have found that separating review/approval from the group that was ultimately responsible for WG management lead to serious problems with the review/approval group becoming out of touch with the community, etc. (at least that's a very short summary of what I've been told about the pre-Kobe IESG/IAB split). Perhaps we could do this in a way that wouldn't have those flaws, but most people I've spoken to seem to be against the idea of splitting review/approval and WG management at the highest level, as the review/approval group has no motivation to help the WGs to achieve their chartered goals... So, perhaps the first choice makes more sense? That is the choice that is represented in the per-area review board proposal that Alex has submitted. In addition to properly balancing the interests of the WG and the interests of the IETF/Internet community, the per-area review board approach makes the IESG directly accountable to the community for the results of the initial review/approval process, making it more likely that those reviews will be consistent with a later IESG review, if one is deemed necessary. --- Another advantage of the per-area review board proposal over this proposal is that it builds upon, rather than replacing, the work that has already been done to build per-area review teams such as the MIB doctors, the ops-dir, Transport doctors, the rtg-dir, etc. At this point, I think that my comments on the proposal may be almost as long as the proposal itself, so I'll stop now. I hope that these comments will be taken as constructive input to the effort of building a scalable IETF review/approval process, even though I don't favor the specific type of review mechanism proposed in this document. Margaret _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Sat Jan 10 19:57:08 2004 Received: from optimus.ietf.org ([132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id TAA27211 for ; Sat, 10 Jan 2004 19:57:08 -0500 (EST) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AfTuC-0003ih-5D for icar-archive@odin.ietf.org; Sat, 10 Jan 2004 19:56:40 -0500 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i0B0ueu9014295 for icar-archive@odin.ietf.org; Sat, 10 Jan 2004 19:56:40 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AfTuB-0003iU-VD for icar-web-archive@optimus.ietf.org; Sat, 10 Jan 2004 19:56:40 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id TAA27051 for ; Sat, 10 Jan 2004 19:56:37 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1AfTuA-0001WJ-00 for icar-web-archive@ietf.org; Sat, 10 Jan 2004 19:56:38 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1AfToY-0001GM-00 for icar-web-archive@ietf.org; Sat, 10 Jan 2004 19:50:52 -0500 Received: from [65.246.255.50] (helo=mx2.foretec.com) by ietf-mx with esmtp (Exim 4.12) id 1AfTln-00017D-00 for icar-web-archive@ietf.org; Sat, 10 Jan 2004 19:47:59 -0500 Received: from optimus22.ietf.org ([132.151.6.22] helo=optimus.ietf.org) by mx2.foretec.com with esmtp (Exim 4.24) id 1AfTjV-00061m-Hs for icar-web-archive@ietf.org; Sat, 10 Jan 2004 19:45:37 -0500 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AfTiu-0003LV-Ia; Sat, 10 Jan 2004 19:45:00 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AfTiP-0003L6-Qz for icar@optimus.ietf.org; Sat, 10 Jan 2004 19:44:34 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id TAA26882 for ; Sat, 10 Jan 2004 19:44:22 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1AfTiE-00011L-00 for icar@ietf.org; Sat, 10 Jan 2004 19:44:18 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1AfTbM-0000pn-00 for icar@ietf.org; Sat, 10 Jan 2004 19:37:12 -0500 Received: from joy.songbird.com ([208.184.79.7]) by ietf-mx with esmtp (Exim 4.12) id 1AfTZB-0000f3-00 for icar@ietf.org; Sat, 10 Jan 2004 19:34:57 -0500 Received: from bbprime (jay.songbird.com [208.184.79.253]) by joy.songbird.com (8.11.6/8.11.6) with SMTP id i0B0fcc14026; Sat, 10 Jan 2004 16:41:41 -0800 From: Dave Crocker To: CC: X-Mailer: PocoMail 3.03 (1740) - EVALUATION VERSION X-URL: http://www.pocomail.com/ Date: Sat, 10 Jan 2004 16:33:53 -0800 Message-ID: <2004110163353.077341@bbprime> In-Reply-To: <5.1.0.14.0.20040110013737.018ef758@localhost> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Content-Transfer-Encoding: 7bit Subject: [Icar] Assessing wg risk and criticality Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.1 required=5.0 tests=AWL autolearn=no version=2.60 Content-Transfer-Encoding: 7bit Content-Transfer-Encoding: 7bit Folks, > I may be missing something, but the problem I see reading this is that this > gives the IESG no assistance in deciding which documents it needs to > review. There was another proposal which I read to say ~the IESG needs to > review documents for which there is disagreement about the results of other > reviews Joel is certainly correct. If we expect to have the IESG treat working groups differentially, then there needs to be community-wide understanding of the differences they should filter on. I think there are a number of factors that ought to be considered: 1. Infrastructure If the work is going to hit the core of the Internet, it needs to have high-level and broad-based review. Period. (In spite of being up in Apps space, something like imposing a system-wide spam control mechanism hits the email infrastructure and probably should count in this category.) 2. Risk Some work is clearly pretty straightforward. The work, itself, is well understood, and/or the core team working on it are known to do this sort of work really well. When the work is not straightforward and/or the team is not known to be experienced with this IETF territory, then the IESG should give it more attention. 3. Urgency When there is rough consensus that useful work is needed Real Soon, then the IESG should endeavor to help that work get done quickly. This does not mean that working groups which do not qualify under any of these concerns should be ignored, but it does provide a meaningful basis for prioritizing oversight. It also has the advantage of requiring working groups and the IESG to assess the factors at the outset (and perhaps on a continuing basis?) d/ -- Dave Crocker Brandenburg InternetWorking _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Sun Jan 11 21:59:22 2004 Received: from optimus.ietf.org ([132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id VAA27371 for ; Sun, 11 Jan 2004 21:59:22 -0500 (EST) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AfsI2-00058S-Bv for icar-archive@odin.ietf.org; Sun, 11 Jan 2004 21:58:54 -0500 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i0C2ws6o019739 for icar-archive@odin.ietf.org; Sun, 11 Jan 2004 21:58:54 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AfsI2-00058I-7Z for icar-web-archive@optimus.ietf.org; Sun, 11 Jan 2004 21:58:54 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id VAA27304 for ; Sun, 11 Jan 2004 21:58:51 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1AfsHz-0002ac-00 for icar-web-archive@ietf.org; Sun, 11 Jan 2004 21:58:51 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1AfsGA-0002UH-00 for icar-web-archive@ietf.org; Sun, 11 Jan 2004 21:56:59 -0500 Received: from [132.151.1.19] (helo=optimus.ietf.org) by ietf-mx with esmtp (Exim 4.12) id 1AfsEK-0002OD-00 for icar-web-archive@ietf.org; Sun, 11 Jan 2004 21:55:04 -0500 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AfsEI-0004rk-Q7; Sun, 11 Jan 2004 21:55:02 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AfsE6-0004q8-AU for icar@optimus.ietf.org; Sun, 11 Jan 2004 21:54:50 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id VAA27164 for ; Sun, 11 Jan 2004 21:54:47 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1AfsE3-0002MJ-00 for icar@ietf.org; Sun, 11 Jan 2004 21:54:47 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1AfsC5-0002Ec-00 for icar@ietf.org; Sun, 11 Jan 2004 21:52:46 -0500 Received: from joy.songbird.com ([208.184.79.7]) by ietf-mx with esmtp (Exim 4.12) id 1AfsAg-00026k-00 for icar@ietf.org; Sun, 11 Jan 2004 21:51:18 -0500 Received: from bbprime (jay.songbird.com [208.184.79.253]) by joy.songbird.com (8.11.6/8.11.6) with SMTP id i0C2w4c22425 for ; Sun, 11 Jan 2004 18:58:04 -0800 From: Dave Crocker To: X-Mailer: PocoMail 3.03 (1740) - EVALUATION VERSION X-URL: http://www.pocomail.com/ Reply-To: dcrocker@brandenburg.com Date: Sun, 11 Jan 2004 18:50:20 -0800 Message-ID: <2004111185020.818584@bbprime> Subject: Re: [Icar] Input based on SIRS experience Mime-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Transfer-Encoding: 7bit Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.1 required=5.0 tests=AWL autolearn=no version=2.60 Content-Transfer-Encoding: 7bit Content-Transfer-Encoding: 7bit (How embarrassing. I've no idea why my previous posting wrapped so strangely, other than the quirkiness of a new mail client. Here's an attempt to re-post, maybe without the problem... /d) Adding my own thoughts to the archive, since the topic came up, and with the hope it can added to the discussion: - It was not much effort to get a core of reviewers signed up. This suggests a good core of community motivation to help. We have not needed more, so we do not know whether an effort like this can attract _enough_ qualified reviewers. - I am not sure how to interpret the low number of review requests. It is easy to say that it is due to lack of community awareness or interest or trust in the SIRs effort. Certainly we have not done as much marketing as we might have wished. However this view is confounded by the very slow metabolic rate of the IETF and the fact that the effort began at the beginning of a summer. No doubt there probably are other factors. Cliches like 'tipping point' and 'crossing the chasm' were created for just this sort of issue in adopting change. For example, it may be that it is more difficult to change existing working group efforts than it is to get integrated in with new(er) ones. Certainly we have not done any serious, sustained effort to get working groups to request reviews. On doing the administrative side of this effort, there are some things that I suspect are strategic, besides the relatively obvious fact that this really does take some ongoing, administrative effort, more than I had expected. (But, then, I always underestimate these things.) - It needs to be easy to have a record of the review process for a document and, by derivation, an ability to see the whole set of review efforts. - There should be a central point for tracking review efforts and a central point for accessing reviews. Besides the obvious, local and tactical benefit to the individual working group, this permits the community to get the ability to review the _set _of reviews and get a sense of the effort and result of doing reviews. In other words, it has marketing value. It makes it easy for others to see that there is review activity and it makes it easy for them to evaluate the nature and quality of those reviews. This will make them more comfortable getting reviews for their own efforts and will help them decide on what reviewers to ask for. Whether this needs to be a permanent, formal archive, like RFCs, an ephemeral, formal archive, like I-Ds, or an unofficial, ephemeral archive like a WG mailing list, has not been resolved. My own thought is that formal publication (RFC and even I-D) is far too much overhead. The only existing mechanism is the mailing list. Creating a new mechanism strikes me as too much effort, and added effort is a very large barrier to adoption of this process. Opening up the IETF Tracker to working group chairs, and adding the review construct and/or allowing free-form entries would help this enormously. (Free-form means that the IETF does not have to formally adopt the thing being tracked.) This means reviews -- and other activities -- could be tracked much sooner than would be possible if we first have to get IETF-wide approval of the review process. - I am not sure whether selection of reviewers, by requesters, needs to be different than we set up. A web page with a list of bios, and a mailing list for sending requests to is probably fine. The public discussion about reviewing shows that the community is still working on clarifying what types of reviews are needed, when, and how to use them. - There is also the small matter of the SIRS reviews that have been done. Have they been they type that are needed? Should they be done differently? We have not tried to assess this. Certainly we cannot expect regular, widespread use of a review process until working groups know what to ask for, or at least what to expect they will get and how they should use the results. d/ -- Dave Crocker Brandenburg InternetWorking _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Sun Jan 11 22:59:44 2004 Received: from optimus.ietf.org ([132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id WAA28857 for ; Sun, 11 Jan 2004 22:59:44 -0500 (EST) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AftET-0006pO-SS for icar-archive@odin.ietf.org; Sun, 11 Jan 2004 22:59:17 -0500 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i0C3xH2X026240 for icar-archive@odin.ietf.org; Sun, 11 Jan 2004 22:59:17 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AftET-0006p9-OV for icar-web-archive@optimus.ietf.org; Sun, 11 Jan 2004 22:59:17 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id WAA28790 for ; Sun, 11 Jan 2004 22:59:13 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1AftEQ-0004hy-00 for icar-web-archive@ietf.org; Sun, 11 Jan 2004 22:59:14 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1AftCd-0004aD-00 for icar-web-archive@ietf.org; Sun, 11 Jan 2004 22:57:24 -0500 Received: from [132.151.1.19] (helo=optimus.ietf.org) by ietf-mx with esmtp (Exim 4.12) id 1AftBH-0004T4-00 for icar-web-archive@ietf.org; Sun, 11 Jan 2004 22:55:59 -0500 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AftBJ-0006hS-UO; Sun, 11 Jan 2004 22:56:01 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AftAW-0006gM-5B for icar@optimus.ietf.org; Sun, 11 Jan 2004 22:55:12 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id WAA28613 for ; Sun, 11 Jan 2004 22:55:08 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1AftAS-0004NZ-00 for icar@ietf.org; Sun, 11 Jan 2004 22:55:08 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1Aft8P-0004Ho-00 for icar@ietf.org; Sun, 11 Jan 2004 22:53:02 -0500 Received: from sccrmhc13.comcast.net ([204.127.202.64]) by ietf-mx with esmtp (Exim 4.12) id 1Aft7r-0004A3-00 for icar@ietf.org; Sun, 11 Jan 2004 22:52:27 -0500 Received: from dfnjgl21 (c-24-1-97-129.client.comcast.net[24.1.97.129]) by comcast.net (sccrmhc13) with SMTP id <2004011203515801600nj51te> (Authid: sdawkins@comcast.net); Mon, 12 Jan 2004 03:51:58 +0000 Message-ID: <01e501c3d8bf$7e382df0$0400a8c0@DFNJGL21> Reply-To: "Spencer Dawkins" From: "Spencer Dawkins" To: "Icar \(E-mail\)" References: <2004110988.424309@bbprime> Subject: Re: [Icar] Input based on SIRS experience Date: Sun, 11 Jan 2004 21:52:25 -0600 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2800.1158 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1165 Content-Transfer-Encoding: 7bit Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.0 required=5.0 tests=none autolearn=no version=2.60 Content-Transfer-Encoding: 7bit Content-Transfer-Encoding: 7bit Dave, Thanks for providing this input. Have I ever thanked you and Brian publically for taking a swing at SIRS? It was an omission. Thank you. I have a couple of comments inline. Spencer ----- Original Message ----- From: "Dave Crocker" To: "Icar (E-mail)" Sent: Saturday, January 10, 2004 11:08 AM Subject: Re: [Icar] Input based on SIRS experience > Adding my own thoughts to the archive, since the topic came up, and with the > hope it can added to the discussion: > > It was not much effort to get a core of reviewers signed up. > > This suggests a good core of community motivation to help. We have not needed > more, so we do not know whether an effort like this can attract _enough_ > qualified reviewers. Yeah, Margaret is also coming up with 200-300 reviewers needed, and we still have no idea how many we can get. :-{ > > I am not sure how to interpret the low number of review requests. > > It is easy to say that it is due to lack of community awareness or interest or > trust in the SIRs effort. Certainly we have not done as much marketing as we > might have wished. However this is view is confounded by the very slow metabolic > rate of the IETF and the fact that the effort began at the beginning of a > summer. I concur completely with each point. > > No doubt there probably are other factors. Cliches like 'tipping point' and > 'crossing the chasm' were created for just this issue. For example, it may be > that it is more difficult to change existing working group efforts than it is to > get integrated in with new(er) ones. Certainly we have not done any serious, > sustained effort to get working groups to request reviews. Yes. > > On doing the administrative side of this effort, there are some things that I > suspect are strategic, besides the relatively obvious fact that this really does > take some ongoing, administrative effort, more than I had expected. (But, then, > I always underestimate these things.) > > It needs to be easy to have a record of the review process for a document and, > by derivation, an ability to see the whole set of review efforts. There should > be a central point for tracking review efforts and a central point for > accessing reviews. I agree, and reviews should be tightly tied to documents (easy to see what reviewers think). > > Besides the obvious, local and tactical benefit to the individual working group, > this permits the community to get the ability to review the _set _of reviews and > get a sense of the effort and result of doing reviews. In other words, it has > marketing value. It makes it easy for others to see that there is review > activity and it makes it easy for them to evaluate the nature and quality of > those reviews. This will make them more comfortable getting reviews for their > own efforts and will help them decide on what reviewers to ask for. Whether this > needs to be a permanent, formal archive, like RFCs, an ephemeral, formal > archive, like I-Ds, or an unofficial, ephemeral archive like a WG mailing list, > has not been resolved. > > My own thought is that formal publication (RFC and even I-D) is far too much > overhead. The only existing mechanism is the mailing list. Creating a new > mechanism strikes me as too much effort, and added effort is a very large > barrier to adoption of this process. This may be correct, but it's tragic, if so. Most of our archives aren't that tractable, and some are REALLY grim (whether because of spam or simply because of posting volume). > > Opening up the IETF Tracker to working group chairs, and adding the review > construct and/or allowing free-form entries would help this enormously. > (Free-form means that the IETF does not have to formally adopt the thing being > tracked. This means reviews -- and other activities -- could be tracked much > sooner than would be possible if we first have to get IETF-wide approval of the > review process. I guessed that by "free-form" you were thinking of using the mechanism to track reviews of documents that hadn't been adopted as WG documents yet. That's also a question. > > I am not sure whether selection of reviewers, by requesters, needs to be > different than we set up. > > A web page with a list of bios, and a mailing list for sending requests to is > probably fine. The mailing list violates a concept Ted Hardie talked about in Minneapolis - that appeals to individuals work better than appeals to groups in the IETF. I think Ted is right. Requesters will really have to be bothered to wonder who needs to review a document, rather than posting a request and wishing reviewers would materialize. > > The public discussion about reviewing shows that the community is still working > on clarifying what types of reviews are needed, when, and how to use them. > > There is also the small matter of the SIRS reviews that have been done. > > Have they been they type that are needed? Should they be done differently? We > have not tried to assess this. Certainly we cannot expect regular, widespread > use of a review process until working groups know what to ask for, or at least > what to expect they will get and how they should use the results. You are such a social scientist. How inconvenient that you should ask a question when we have no idea what the answer is! _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Sun Jan 11 23:46:00 2004 Received: from optimus.ietf.org ([132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id XAA29927 for ; Sun, 11 Jan 2004 23:46:00 -0500 (EST) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AftxE-0008Pp-Lu for icar-archive@odin.ietf.org; Sun, 11 Jan 2004 23:45:32 -0500 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i0C4jWFm032343 for icar-archive@odin.ietf.org; Sun, 11 Jan 2004 23:45:32 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AftxE-0008Pa-H4 for icar-web-archive@optimus.ietf.org; Sun, 11 Jan 2004 23:45:32 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id XAA29921 for ; Sun, 11 Jan 2004 23:45:29 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1Aftx7-0006F6-00 for icar-web-archive@ietf.org; Sun, 11 Jan 2004 23:45:25 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1AftsZ-00067R-00 for icar-web-archive@ietf.org; Sun, 11 Jan 2004 23:40:43 -0500 Received: from [132.151.1.19] (helo=optimus.ietf.org) by ietf-mx with esmtp (Exim 4.12) id 1AftoL-0005yK-00 for icar-web-archive@ietf.org; Sun, 11 Jan 2004 23:36:21 -0500 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1Afto2-0007yJ-Ro; Sun, 11 Jan 2004 23:36:02 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1Aftno-0007xn-R9 for icar@optimus.ietf.org; Sun, 11 Jan 2004 23:35:48 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id XAA29708 for ; Sun, 11 Jan 2004 23:35:46 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1Aftnh-0005xB-00 for icar@ietf.org; Sun, 11 Jan 2004 23:35:41 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1AfthU-0005qZ-00 for icar@ietf.org; Sun, 11 Jan 2004 23:29:17 -0500 Received: from joy.songbird.com ([208.184.79.7]) by ietf-mx with esmtp (Exim 4.12) id 1Aftfs-0005kN-00 for icar@ietf.org; Sun, 11 Jan 2004 23:27:36 -0500 Received: from bbprime (jay.songbird.com [208.184.79.253]) by joy.songbird.com (8.11.6/8.11.6) with SMTP id i0C4YMc27689; Sun, 11 Jan 2004 20:34:22 -0800 From: Dave Crocker To: , "Icar \(E-mail\)" X-Mailer: PocoMail 3.03 (1740) - EVALUATION VERSION X-URL: http://www.pocomail.com/ Date: Sun, 11 Jan 2004 20:26:37 -0800 Message-ID: <2004111202637.788100@bbprime> In-Reply-To: <01e501c3d8bf$7e382df0$0400a8c0@DFNJGL21> Subject: Re: Re: [Icar] Input based on SIRS experience Mime-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Transfer-Encoding: 7bit Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.1 required=5.0 tests=AWL autolearn=no version=2.60 Content-Transfer-Encoding: 7bit Content-Transfer-Encoding: 7bit Spencer, > Yeah, Margaret is also coming up with 200-300 reviewers needed, and > we still have no idea how many we can get. :-{ The calculations along those lines all look reasonable. At the moment, I'd class it as a problem we'd love to have. More realistically, yes, we need to make sure we design a mechanism that is both useful and scalable. One thought is that good early reviewing might substantially alter what is needed later. Another is to note a separate line of discussion that essentially distinguishes between some working group efforts that get more attention, versus others that might get less. > > My own thought is that formal publication (RFC and even I-D) is far > > too much overhead. The only existing mechanism is the mailing > > list. Creating a new > > mechanism strikes me as too much effort, and added effort is a very > > large barrier to adoption of this process. > > This may be correct, but it's tragic, if so. Most of our archives > aren't that tractable, and some are REALLY grim I think that long-term access to useful copies of all IETF working materials -- other than the stuff we formally publish -- has major benefit, for legal and academic purposes. But I view this as a distinct issue. My current concern is that the review stuff not create a new distribution and/or storage mechanism. > > Opening up the IETF Tracker to working group chairs, and adding the > > review construct and/or allowing free-form entries would help this > > enormously. (Free-form means that the IETF does not have to > > formally adopt the thing being > > tracked. This means reviews -- and other activities -- could be > > tracked much sooner than would be possible if we first have to get > > IETF-wide approval of the review process. > > I guessed that by "free-form" you were thinking of using the > mechanism to track reviews of documents that hadn't been adopted as > WG documents yet. That's also a question. It isn't the status of the document that I was thinking about. It was the status of the _type_ of data being entered, in terms of IETF formality and officialness. If we simply let a working group chair put in any entry they want, the tracker won't know what the state transitions need to be, but the working group chair can set the state manually. This means they do not need to wait for the IETF to define and approve the details beforehand. > > I am not sure whether selection of reviewers, by requesters, needs > > to be different than we set up. > > > > A web page with a list of bios, and a mailing list for sending > > requests to is probably fine. > > The mailing list violates a concept Ted Hardie talked about in > Minneapolis - that appeals to individuals work better than appeals to > groups in the IETF. I think Ted is right. Requesters will really have > to be bothered to wonder who needs to review a document, rather than > posting a request and wishing reviewers would materialize. The phenomenon of dissipated responsibility was first noted with the murder of Kitty Genovese, in New York, with lots of bystanders who could see each other, none of whom called the police. That said, the queries that have come into the SIRS list have gotten good response. However my point was not that all requests should come to the list, but that the list and the biographies with email addresses ought to be sufficient. > > There is also the small matter of the SIRS reviews that have been > > done... > > You are such a social scientist. How inconvenient that you should ask > a question when we have no idea what the answer is! The purpose of an experiment is to spawn more experiments... d/ -- Dave Crocker Brandenburg InternetWorking _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Mon Jan 12 12:27:42 2004 Received: from optimus.ietf.org ([132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA06793 for ; Mon, 12 Jan 2004 12:27:42 -0500 (EST) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1Ag5qM-0004hy-PQ for icar-archive@odin.ietf.org; Mon, 12 Jan 2004 12:27:15 -0500 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i0CHRElW018094 for icar-archive@odin.ietf.org; Mon, 12 Jan 2004 12:27:14 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1Ag5qM-0004hl-Ka for icar-web-archive@optimus.ietf.org; Mon, 12 Jan 2004 12:27:14 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA06753 for ; Mon, 12 Jan 2004 12:27:11 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1Ag5qL-0004iY-00 for icar-web-archive@ietf.org; Mon, 12 Jan 2004 12:27:13 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1Ag5oS-0004fQ-00 for icar-web-archive@ietf.org; Mon, 12 Jan 2004 12:25:17 -0500 Received: from [132.151.1.19] (helo=optimus.ietf.org) by ietf-mx with esmtp (Exim 4.12) id 1Ag5nE-0004cw-00 for icar-web-archive@ietf.org; Mon, 12 Jan 2004 12:24:00 -0500 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1Ag5nF-0004ei-9W; Mon, 12 Jan 2004 12:24:01 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1Ag5mL-0004dq-KQ for icar@optimus.ietf.org; Mon, 12 Jan 2004 12:23:05 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA06646 for ; Mon, 12 Jan 2004 12:23:02 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1Ag5mK-0004ao-00 for icar@ietf.org; Mon, 12 Jan 2004 12:23:04 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1Ag5kR-0004Yn-00 for icar@ietf.org; Mon, 12 Jan 2004 12:21:08 -0500 Received: from measurement-factory.com ([206.168.0.5]) by ietf-mx with esmtp (Exim 4.12) id 1Ag5jB-0004Wn-00 for icar@ietf.org; Mon, 12 Jan 2004 12:19:49 -0500 Received: from measurement-factory.com (localhost [127.0.0.1]) by measurement-factory.com (8.12.9/8.12.9) with ESMTP id i0CHJhk3017481; Mon, 12 Jan 2004 10:19:43 -0700 (MST) (envelope-from rousskov@measurement-factory.com) Received: (from rousskov@localhost) by measurement-factory.com (8.12.9/8.12.9/Submit) id i0CHJhWG017480; Mon, 12 Jan 2004 10:19:43 -0700 (MST) (envelope-from rousskov) Date: Mon, 12 Jan 2004 10:19:43 -0700 (MST) From: Alex Rousskov To: Alex Zinin cc: icar@ietf.org Subject: Re: [Icar] ICAR draft charter rev 2 In-Reply-To: <196108581021.20040108171716@psg.com> Message-ID: References: <196108581021.20040108171716@psg.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.0 required=5.0 tests=AWL autolearn=no version=2.60 Alex, I am sorry, but I still do not like the proposed WG charter/direction/milestones. Your Version2 edits were very useful in that they made old problems with scope and authority more explicit and visible. Perhaps charters are not perceived as important WG documents in IETF, but I think they should be reviewed and polished as rigorously as protocols. My specific comments and wording suggestions are inlined below, but I am waiting for somebody to say that "rough consensus does not require unanimity" and keep going in the direction you want. On Thu, 8 Jan 2004, Alex Zinin wrote: > WG Description: > > The WG will work out mechanisms for improved cross-functional > review within the IETF, addressing the issues identified by the > PROBLEM WG. What is the rationale behind trying to limit the scope of the WG to cross-functional review? Given your definition of "cross-functional" as "document review covering different aspects of Internet technology", the above scope definition includes virtually _any_ review within the IETF. IMO, "any review" should be the explicit scope. We should not try to limit the scope using unbounded concepts like cross-functional review (as currently defined). I cannot think of any significant review activity that the above scope statement (along with the cross-functional definition) would clearly _exclude_. Also, let's not imply that we will solve all PROBLEM WG problems and let's not mix scope and an incomplete problem list together. Suggested wording: The WG will improve IETF review mechanisms by adjusting existing mechanisms and proposing new ones. The improvements will address at least the following known issues: - provide a list of issues - quoting PROBLEM WG output as needed > This includes a better community review, as well as > more structured pre-IESG review that may be used to improve > scalability of the IESG review function. I do not understand why this WG must stop at the IESG border. Are we saying that IESG review mechanisms are already good enough and just do not scale well? What if the best solution will require some IESG-level changes? How about IAB involvement? Is it necessarily post-IESG? Suggested wording: Review mechanisms under consideration cover both community and structured review, may span all levels of IETF structure, and may require changes to IETF entity roles or structure. > Definitions: > > o Cross-functional review: document review covering different > aspects of Internet technology, including those represented by > WGs within different IETF areas. If my changes are accepted, the above definition becomes unnecessary. > o Community review: document review performed by individual IETF > participants and not caused by their responsibilities within > the IETF management structure. > > o Structured review: more formal and role-based document review > performed by individuals assuming certain responsibilities (such > as by WG chairs, directorate and IESG members.) Good. > It is an explicit goal of the WG to come up with mechanisms > encouraging earlier review of the documents. It is trivial to come up with encouraging mechanisms. We should aim higher and also be more specific about what "earlier" means. Earlier than what? Suggested wording: It is an explicit goal of the WG to design "early review" mechanisms. These mechanisms should ensure that most IETF documents are reviewed and corrected before the estimated cost of correction becomes comparable with the cost of leaving the review-identified defect unfixed. The above suggested wording needs further polishing, but I hope it gives enough idea on how I think we should introduce early review into the charter. > An early review is > best for catching architectural problems while they're still > relatively easy to solve. In particular, many cross-functional > interactions can be spotted and dealt with, thus avoiding many > "late surprises". A final review (currently done by the IESG) can > catch remaining cross-functional interactions, as well as deal > with overall quality issues. I would suggest to remove the above paragraph as unnecessary and even too suggestive about a stage-like approach and final review role. > The WG will cooperate with others in starting and evaluating > experiments with early community and structured reviews. The > evaluation of such experiments may be published as Informational > RFCs if the group so desires. Does the above wording change anything for this WG? In other words, what would change if we remove the above paragraph? To me, it sounds lile a non-statement that is true for every WG. > The WG will also coordinate with other WGs involved in the IETF > reform process on proposed changes to the IETF Working Group > operations and Standards process if those are considered > necessary. Same comment: What would change if we remove the above paragraph? It is so vague that it seems useless to me. If we can identify specific WGs and liaisons, then we should do so. If we cannot, we are not really providing any useful information here. All IETF WGs should cooperate, of course. > WG milestones: > > FEB 2004: Submit drafts on improved community and structured reviews 40 days sounds like an unnecessary aggressive deadline, especially if the WG has to submit all drafts by then. It would take at least 20-30 days to carefully review and polish existing drafts, and we do not have any of decent quality to start with. I would suggest to add at least one more month. Also, "submitting a draft" is trivial because one can submit a placeholder. See specific wording suggestions below. > SEP 2004: Submit draft on improved community review to > the IESG for publication as BCP > SEP 2004: Submit draft on improved structured community review to > the IESG for publication as BCP Again, I am against separating the two drafts in the charter. Perhaps we will have one draft addressing both areas with a single, unified solution. Or maybe we will have three drafts that split the work horizontally and not vertically (whatever that means). Also, it is not clear whether we can plan that well in advance. How about replacing all currently proposed deadlines with: FEB 2004: Decide on a comprehensive set of review mechanisms. MAR 2004: Document alpha versions of all review mechanisms. APR 2004: Evaluate WG progress and potential; close or recharter. Thank you, Alex. _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Mon Jan 12 13:08:24 2004 Received: from optimus.ietf.org ([132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA08935 for ; Mon, 12 Jan 2004 13:08:24 -0500 (EST) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1Ag6Tj-0007Oj-RM for icar-archive@odin.ietf.org; Mon, 12 Jan 2004 13:07:56 -0500 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i0CI7sNg028430 for icar-archive@odin.ietf.org; Mon, 12 Jan 2004 13:07:54 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1Ag6Th-0007OT-VY for icar-web-archive@optimus.ietf.org; Mon, 12 Jan 2004 13:07:54 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA08869 for ; Mon, 12 Jan 2004 13:07:50 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1Ag6Tg-0006jr-00 for icar-web-archive@ietf.org; Mon, 12 Jan 2004 13:07:52 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1Ag6Rl-0006ej-00 for icar-web-archive@ietf.org; Mon, 12 Jan 2004 13:05:54 -0500 Received: from [132.151.1.19] (helo=optimus.ietf.org) by ietf-mx with esmtp (Exim 4.12) id 1Ag6Pv-0006bt-00 for icar-web-archive@ietf.org; Mon, 12 Jan 2004 13:03:59 -0500 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1Ag6Pw-0006sq-NY; Mon, 12 Jan 2004 13:04:00 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1Ag6PO-0006rv-Qx for icar@optimus.ietf.org; Mon, 12 Jan 2004 13:03:26 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA08558 for ; Mon, 12 Jan 2004 13:03:23 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1Ag6PN-0006VL-00 for icar@ietf.org; Mon, 12 Jan 2004 13:03:25 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1Ag6OF-0006NT-00 for icar@ietf.org; Mon, 12 Jan 2004 13:02:17 -0500 Received: from key1.docomolabs-usa.com ([216.98.102.225] helo=fridge.docomolabs-usa.com ident=fwuser) by ietf-mx with esmtp (Exim 4.12) id 1Ag6N2-0006IQ-00 for icar@ietf.org; Mon, 12 Jan 2004 13:01:01 -0500 Message-ID: <01c401c3d936$19675690$606015ac@dclkempt40> From: "James Kempf" To: References: <2004111185020.818584@bbprime> Subject: Review Board Scalability (was: Re: [Icar] Input based on SIRS experience) Date: Mon, 12 Jan 2004 10:01:27 -0800 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Transfer-Encoding: 7bit Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.0 required=5.0 tests=AWL autolearn=no version=2.60 Content-Transfer-Encoding: 7bit Content-Transfer-Encoding: 7bit Margaret Wasserman said: >Like the SIRS proposal, I believe that this proposal is quite >naive about the scale of this particular problem. Let's take >a few numbers: > >We currently approve about 200 RFCs per year. Each of these >RFCs receives (on average) ~2-1/2 review cycles from the IESG >plus a full AD Review. So, let's assume that we will continue to >produce 200 documents per year, and that each will be subjected >to 3 cross-area review cycles. > >BTW, if the same group of people reviews the document all three >times (also see section on consistency below), the later reviews >will take much less time than the earlier reviews. > >The IESG consists of 13 people, not all of whom carefully review >each document during IESG review (for various reason). So, let's >assume that we can get adequate cross-area coverage (see section >on cross-area coverage) by having each document reviewed by >8 properly selected members of the Quality Review Board. That's >the number of ADs that it currently takes (today, with one slot >empty) to approve publication of a document. > >So, the number of individual document reviews required would be >200 * 3 * 8 == 4800. > >Let's assume that we can find 200 people who are willing and >qualified to serve on the Quality Review Board. I don't know >that this is possible, and it means that someone will have to >manage a function that involves 200 people (see manageability >section below), but let's assume... In that case, each member >of the review board would need to do an average of 24 reviews >per year -- so a minimum of 3 is misleadingly low. Ideally, >this means that each member of the review board would do a >full 3-cycle review for 8 documents per year. > >If we expect this system to result in a 50% improvement in >document throughput, we will need to handle 300 documents per >year, which requires 36 reviews/board member/year (or 12 >documents). > >Doable? Maybe. If we can find 200 people willing to do this, >figure out how to train and organize them (see sections on >preparation/training and management below), and sustain that >number over time. The sign-up rate for SIRS does not make me >confident that this is possible, but we could try.... and Spencer Dawkins said: > Yeah, Margaret is also coming up with 200-300 reviewers needed, and > we still have no idea how many we can get. :-{ It seems to me that, in principle, the only difference between an IETF review board and the program committee for a conference is that the members of an IETF review board would be expected to stick with a draft for more than just one review, and possibly in the number of reviewers per paper. So it seems like we might be able to look at existing experience with conference program committees for some idea about how to make this work. As an example, I'm on the program committee for Mobihoc this year. There are 42 program committee members and 230 papers which works out to a ratio of about 6 papers per reviewer. I was assigned 11 papers, from which I conclude that the program chairs have decided to have about 2 reviews per paper. If one accepts Margaret's numbers of about 24 reviews per year, that would imply that the number of reviews for IETF drafts would be about twice what Mobihoc members are expected to do. This doesn't seem all that bad to me considering that I'll probably do at least 11 additional reviews this year, of IETF drafts and papers for other conferences, etc. Working backwards from the Mobihoc program committee, that would reduce the number of review board members to maybe 90 (twice the Mobihoc number) rather than 200. To do this, however, we would have to reduce the number of independent _solicited_ reviewers to 2 rather than Margaret's suggestion of 8. Would this cause a reduction of quality detection problems? Unclear. Many drafts now receive unsolicited review as part of WG last call and IETF last call and presumably that would not change. Margaret's number of 8 reviews was based on an approximation of how many ADs review drafts, so YMMV. And, even with 8 people in the IESG reviewing, there have been occasions when problems have been caught later (MIPv6 binding update security comes to mind). Also, with an issue tracking system in place, the reviewers may be able to determine whether issues are resolved more easily without having to wade through the text in the spec, so reviews might become less time consuming. I think there is a different issue that is possibly more of a barrier, getting people to volunteer by providing the right incentive. People volunteer for a conference program commitee because they want to see what other people in the area are up to from a reseach standpoint, and they want to make sure that published results are of high quaility. The program committee's comments are used to determine what papers are ultimately published, that is, their reviews make a real difference in the conference content. If the review board's work is just advisory, as the ART and SIRS proposals do, then I think we will have a hard time getting people to volunteer. Imagine a program committee for a conference in which the reviews produced by the program committee could be overridden by the program's organizing committee, or even by the authors (this would be the equivalent of the IESG or the WGs themselves, respectively, being allowed to override a review board opinion). Who would volunteer? There is another kind of conference program committee in which the authors themselves are required to do reviews. This could be another way to solicit reviews, people who are draft editors are required to provide reviews. Obviously, since draft editors are less experienced, their reviews could not be as highly rated as the review board's reviews. In such conferences, I presume the organizing committee has the final say, and typically these kinds of conferences are not first tier. jak _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Mon Jan 12 13:16:31 2004 Received: from optimus.ietf.org ([132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA09266 for ; Mon, 12 Jan 2004 13:16:31 -0500 (EST) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1Ag6bc-0007cc-0t for icar-archive@odin.ietf.org; Mon, 12 Jan 2004 13:16:04 -0500 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i0CIG4uP029292 for icar-archive@odin.ietf.org; Mon, 12 Jan 2004 13:16:04 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1Ag6bb-0007cN-Ro for icar-web-archive@optimus.ietf.org; Mon, 12 Jan 2004 13:16:03 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA09231 for ; Mon, 12 Jan 2004 13:16:00 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1Ag6bZ-00075I-00 for icar-web-archive@ietf.org; Mon, 12 Jan 2004 13:16:02 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1Ag6Zc-000733-00 for icar-web-archive@ietf.org; Mon, 12 Jan 2004 13:14:01 -0500 Received: from [132.151.1.19] (helo=optimus.ietf.org) by ietf-mx with esmtp (Exim 4.12) id 1Ag6Yd-0006zy-00 for icar-web-archive@ietf.org; Mon, 12 Jan 2004 13:12:59 -0500 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1Ag6Ye-0007YU-IP; Mon, 12 Jan 2004 13:13:00 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1Ag6Xo-0007Xs-0i for icar@optimus.ietf.org; Mon, 12 Jan 2004 13:12:08 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA09108 for ; Mon, 12 Jan 2004 13:12:04 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1Ag6Xm-0006x9-00 for icar@ietf.org; Mon, 12 Jan 2004 13:12:06 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1Ag6W2-0006tY-00 for icar@ietf.org; Mon, 12 Jan 2004 13:10:19 -0500 Received: from measurement-factory.com ([206.168.0.5]) by ietf-mx with esmtp (Exim 4.12) id 1Ag6Ur-0006oh-00 for icar@ietf.org; Mon, 12 Jan 2004 13:09:05 -0500 Received: from measurement-factory.com (localhost [127.0.0.1]) by measurement-factory.com (8.12.9/8.12.9) with ESMTP id i0CI94k3019210; Mon, 12 Jan 2004 11:09:04 -0700 (MST) (envelope-from rousskov@measurement-factory.com) Received: (from rousskov@localhost) by measurement-factory.com (8.12.9/8.12.9/Submit) id i0CI9410019209; Mon, 12 Jan 2004 11:09:04 -0700 (MST) (envelope-from rousskov) Date: Mon, 12 Jan 2004 11:09:04 -0700 (MST) From: Alex Rousskov To: Dave Crocker cc: "Icar (E-mail)" Subject: Re: Re: [Icar] Input based on SIRS experience In-Reply-To: <2004111202637.788100@bbprime> Message-ID: References: <2004111202637.788100@bbprime> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.0 required=5.0 tests=AWL autolearn=no version=2.60 On Sun, 11 Jan 2004, Dave Crocker wrote: > the list and the biographies with email addresses ought to be > sufficient. In my experience as a SIRS user (though I was not able to solicit a review), I found these two items highly desirable if not essential: - reviewer status (available immediately, busy with N reviews, come back in 3 months, on vacation, etc.) - browseable collection of reviewer's past reviews SIRs interface currently lacks them. Thanks, Alex. _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Mon Jan 12 14:00:33 2004 Received: from optimus.ietf.org ([132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA11385 for ; Mon, 12 Jan 2004 14:00:32 -0500 (EST) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1Ag7IC-0000Sy-GJ for icar-archive@odin.ietf.org; Mon, 12 Jan 2004 14:00:04 -0500 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i0CJ04FZ001789 for icar-archive@odin.ietf.org; Mon, 12 Jan 2004 14:00:04 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1Ag7IB-0000Sk-Mq for icar-web-archive@optimus.ietf.org; Mon, 12 Jan 2004 14:00:03 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA11362 for ; Mon, 12 Jan 2004 14:00:01 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1Ag7I9-0001JA-00 for icar-web-archive@ietf.org; Mon, 12 Jan 2004 14:00:01 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1Ag7GQ-0001Gb-00 for icar-web-archive@ietf.org; Mon, 12 Jan 2004 13:58:15 -0500 Received: from [132.151.1.19] (helo=optimus.ietf.org) by ietf-mx with esmtp (Exim 4.12) id 1Ag7FD-0001Ce-00 for icar-web-archive@ietf.org; Mon, 12 Jan 2004 13:56:59 -0500 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1Ag7FF-0000OQ-4E; Mon, 12 Jan 2004 13:57:01 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1Ag7Eh-0000Np-Gs for icar@optimus.ietf.org; Mon, 12 Jan 2004 13:56:27 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA11116 for ; Mon, 12 Jan 2004 13:56:25 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1Ag7Ef-00019J-00 for icar@ietf.org; Mon, 12 Jan 2004 13:56:25 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1Ag7Cx-00011P-00 for icar@ietf.org; Mon, 12 Jan 2004 13:54:40 -0500 Received: from netcore.fi ([193.94.160.1]) by ietf-mx with esmtp (Exim 4.12) id 1Ag7BS-0000sc-00 for icar@ietf.org; Mon, 12 Jan 2004 13:53:07 -0500 Received: from localhost (pekkas@localhost) by netcore.fi (8.11.6/8.11.6) with ESMTP id i0CIqWn01740; Mon, 12 Jan 2004 20:52:32 +0200 Date: Mon, 12 Jan 2004 20:52:32 +0200 (EET) From: Pekka Savola To: James Kempf cc: icar@ietf.org Subject: Re: Review Board Scalability (was: Re: [Icar] Input based on SIRS experience) In-Reply-To: <01c401c3d936$19675690$606015ac@dclkempt40> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.0 required=5.0 tests=none autolearn=no version=2.60 On Mon, 12 Jan 2004, James Kempf wrote: > It seems to me that, in principle, the only difference between an IETF > review board and the program committee for a conference is that the members > of an IETF review board would be expected to stick with a draft for more > than just one review, and possibly in the number of reviewers per paper. So > it seems like we might be able to look at existing experience with > conference program committees for some idea about how to make this work. Note that there may be some unstated assumptions here. For example, that such an IETF Review Board would replace some functions of the IESG, such as: - AD Evaluation - Review for RFC-Editor submissions - Review of WG submission of Info/Experimental documents - Review of individual submissions of --""-- - Review of standards track documents as well If you assume that (or all of these), you will undoubtedly arrive at a conclusion that any proposal will lack manpower. However, if the idea is to improve cross-area or cross-functional review so that some of the above will go *easier*, requiring much smaller amount of time, we might be getting somewhere. So, when proposing a review board(s), it's useful to explicitly state which functions we want to replace (or augment). > If the review board's work is just advisory, as the ART and SIRS proposals > do, then I think we will have a hard time getting people to volunteer. I'm not sure if this is necessarily the case. Speaking from personal experience, there is several kinds of responses a reviewer will not want to see: - "it's too late to make reviews [especially on this specific issue [which may have had consensus at X]], but thanks anyway", " - (comments are received, but they're not taken into account) * at least in a timely fashion (following up within days is easier than after months) > Imagine a program committee for a conference in which the reviews produced > by the program committee could be overridden by the program's organizing > committee, Organizing committee would not be a problem, at least to me -- you can consider it as "they know best" -- at least after discussion. > or even by the authors (this would be the equivalent of the IESG > or the WGs themselves, respectively, being allowed to override a review > board opinion). Who would volunteer? The authors and WGs are a different thing entirely. If they failed to see a problem in the first place, I would not count on them to judge whether a problem needs to solved or not. > There is another kind of conference program committee in which the authors > themselves are required to do reviews. This could be another way to solicit > reviews, people who are draft editors are required to provide reviews. Heh, an interesting idea :-) Along the same lines, you could also require that each WG chair must make N reviews every year. An instant base of 200+ reviewers :-) -- Pekka Savola "You each name yourselves king, yet the Netcore Oy kingdom bleeds." Systems. Networks. Security. -- George R.R. Martin: A Clash of Kings _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Mon Jan 12 14:30:40 2004 Received: from optimus.ietf.org ([132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA12536 for ; Mon, 12 Jan 2004 14:30:40 -0500 (EST) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1Ag7lM-0001OB-1j for icar-archive@odin.ietf.org; Mon, 12 Jan 2004 14:30:12 -0500 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i0CJUCCv005338 for icar-archive@odin.ietf.org; Mon, 12 Jan 2004 14:30:12 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1Ag7lK-0001O0-CQ for icar-web-archive@optimus.ietf.org; Mon, 12 Jan 2004 14:30:10 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA12514 for ; Mon, 12 Jan 2004 14:30:07 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1Ag7lH-0002Y1-00 for icar-web-archive@ietf.org; Mon, 12 Jan 2004 14:30:07 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1Ag7jd-0002Uq-00 for icar-web-archive@ietf.org; Mon, 12 Jan 2004 14:28:27 -0500 Received: from [132.151.1.19] (helo=optimus.ietf.org) by ietf-mx with esmtp (Exim 4.12) id 1Ag7iF-0002Pf-00 for icar-web-archive@ietf.org; Mon, 12 Jan 2004 14:26:59 -0500 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1Ag7iG-0001KA-Rp; Mon, 12 Jan 2004 14:27:00 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1Ag7hW-0001JE-Q7 for icar@optimus.ietf.org; Mon, 12 Jan 2004 14:26:14 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA12349 for ; Mon, 12 Jan 2004 14:26:12 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1Ag7hU-0002MY-00 for icar@ietf.org; Mon, 12 Jan 2004 14:26:12 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1Ag7fd-0002Ho-00 for icar@ietf.org; Mon, 12 Jan 2004 14:24:18 -0500 Received: from measurement-factory.com ([206.168.0.5]) by ietf-mx with esmtp (Exim 4.12) id 1Ag7ef-0002Dq-00 for icar@ietf.org; Mon, 12 Jan 2004 14:23:18 -0500 Received: from measurement-factory.com (localhost [127.0.0.1]) by measurement-factory.com (8.12.9/8.12.9) with ESMTP id i0CJN5k3023074; Mon, 12 Jan 2004 12:23:06 -0700 (MST) (envelope-from rousskov@measurement-factory.com) Received: (from rousskov@localhost) by measurement-factory.com (8.12.9/8.12.9/Submit) id i0CJN52I023073; Mon, 12 Jan 2004 12:23:05 -0700 (MST) (envelope-from rousskov) Date: Mon, 12 Jan 2004 12:23:05 -0700 (MST) From: Alex Rousskov To: James Kempf cc: solutions@alvestrand.no, icar@ietf.org In-Reply-To: <029201c3d712$48293870$606015ac@dclkempt40> Message-ID: References: <029201c3d712$48293870$606015ac@dclkempt40> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Subject: [Icar] Re: [Solutions] Summary of Discussion on Reforming IETF Quality Control Process Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.0 required=5.0 tests=AWL autolearn=no version=2.60 On Fri, 9 Jan 2004, James Kempf wrote: > I've written a draft summarizing the discussions on reforming the > IETF quality control process that started on mpowr in December and > wandered to solutions in January. It's here: > > http://www.geocities.com/kempf42/draft-list-quality-control-00.txt James, Thank you for writing the draft. Here is my high-level feedback. I assume we are not interested in low-level polishing at this time. 1. "The ultimate responsibility for ensuring the quality of technical work done in the IETF lies with the Working Group..." I suspect you meant: "The ultimate responsibility for producing quality technical work in the IETF lies with the Working Group..." It is correct that WG/authors are responsible for producing quality output. It is wrong to imply that WG can ensure that. We need outside controls to ensure quality and your wording does not reflect that intent. Also, how does the above statement is affected by the fact that IESG can make WG-unapproved changes to WG documents? Do such changes make IESG responsible? Would it be more accurate to say that WG is responsible only up to a point where IESG takes control and does its conflict resolution thing? 2. "The ultimate responsibility for the quality of editorial content of the document describing the work lies with the authors and editors of the document." Sounds nice, but how does this reconcile with #1 above? The Author cannot overrule WG consensus. Are you implying that the author should resign if she disagrees with the WG consensus regarding her document? 3. "The IESG is responsible for deciding which Working Groups will require full IESG review on their documents" Wrong granularity, IMO. IESG decision should be made on document basis not WG basis, in general. That does not prevent IESG from reviewing every document of a given WG, of course, but it does not require that either. 4. "For documents that require IESG review or IETF management actions involving quality issues, the Area Director is responsible for ensuring that ... the [IESG] results are communicated to the Working Group, in as prompt a fashion as possible." A good review mechanism should not require manual involvement to communicate IESG (or any reviewer) actions on a document. We should avoid problems that manual forwarding of decisions always causes. 5. "Members of the board (called "Reviewers") may be subject experts with deep but fairly narrow expertise in one particular area, but more often they will have broad experience in a variety of areas involving Internet technology." I do not understand why deep expertise is worse than broad one. Why are you making this distinction? Is this sentence important? 6. Quality Review Board membership. I believe that any Board membership approach based on formal AD approval and/or popularity contest (IETF voting) is inferior to an "open pool of reviewers" approach. I hate to see IETF using such rigid mechanisms of "traditional" SDOs in a what is supposed to be an open volunteer organization. More specifically, I believe that the review pool/board should be open to anybody who volunteers and plays by the rules. There has to be a mechanism that identifies reviewers currently "trusted" by IESG, of course. Such mechanism will ensure consistency of reviews and IESG actions should the latter be necessary. Designing this mechanism is not trivial, but we can succeed if we focus on that rather than "membership rules". There should be no automatic inclusion or exclusion from the pool/board. We should not drag senior IETFers into the pool just because we can identify them by their IESG memberships or RFC counts. If they want to help, they will volunteer. We should not exclude idle reviewers from the pool because they did nothing wrong per se (rejecting review opportunities should not be counted _against_ anybody; it should be counted for solicitors to see and interpret). 7. "A determination of whether a full IESG review is necessary should be included in the Working Group Charter in order that the review process can proceed predictably" I do not understand the justification of this heavy requirement. Why is it important to know a priori that all WG documents will be fully reviewed by IESG? Will it make the WG work harder? Will it make the Chair look more important? Will it make other WG feel less important? What is the value to be gained from knowing that all WG documents will be reviewed by IESG, especially since IESG can review any document without a prior warning? This design also assumes that we are good at predicting what needs a full review even before the work has started. What problem does this WG flag solve?? 8. Review Process The AD should not remove reviewers from the proposed list. The only thing an AD can do is to add reviewers if he thinks that proposed list does not provide enough trusted coverage. The review request should be sent to all reviewers accepting such requests, not just the ones on the AD-approved list. The whole design should make unknown but good reviewers known. This feature is essential for attracting always new/fresh reviewers and ensuring smooth scale. Full IESG review should not be a special case as far as notification and "handling" is concerned. All reviews must be archived and cross-referenced with documents and reviewers. 9. Disposition of Review Results The reviewers should be responsible for classifying their own comments and entering them into the review database. The Chair should not be involved at that stage. Again, we should not create a forwarding dependency/role when none is required. The above suggestion does create some extra work for some reviewers, but it is "good" work. It keeps reviews cleaner and more constructive. It also avoids problems with Chairs missing a comment or mis-classifying it. Finally, it actually saves reviewers time in the long run because they can check how their comments were addressed without mapping their comments with Chair's rendering of their comments. The review database/interface should be provided on IETF web site for everybody to use. The "issues page" is generated by IETF software, not Chairs. This provides everybody with a familiar interface and, again, solves "forwarding" problems like a forgotten, misrepresented, or even changed-after-final-reviewer-check issue on a Chair-controlled web page. IMO, we need an IETF-wide review management system to facilitate key review-related improvements we want. Thank you, Alex. _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Mon Jan 12 16:00:23 2004 Received: from optimus.ietf.org ([132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA18808 for ; Mon, 12 Jan 2004 16:00:23 -0500 (EST) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1Ag9AA-0004kO-RD for icar-archive@odin.ietf.org; Mon, 12 Jan 2004 15:59:54 -0500 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i0CKxsJx018242 for icar-archive@odin.ietf.org; Mon, 12 Jan 2004 15:59:54 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1Ag9AA-0004k9-MK for icar-web-archive@optimus.ietf.org; Mon, 12 Jan 2004 15:59:54 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA18750 for ; Mon, 12 Jan 2004 15:59:52 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1Ag9A9-0007QA-00 for icar-web-archive@ietf.org; Mon, 12 Jan 2004 15:59:53 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1Ag98F-0007N2-00 for icar-web-archive@ietf.org; Mon, 12 Jan 2004 15:57:55 -0500 Received: from [132.151.1.19] (helo=optimus.ietf.org) by ietf-mx with esmtp (Exim 4.12) id 1Ag96O-0007Jr-00 for icar-web-archive@ietf.org; Mon, 12 Jan 2004 15:56:00 -0500 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1Ag96P-0004gD-Jl; Mon, 12 Jan 2004 15:56:01 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1Ag96K-0004f3-R1 for icar@optimus.ietf.org; Mon, 12 Jan 2004 15:55:56 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA18593 for ; Mon, 12 Jan 2004 15:55:54 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1Ag96J-0007Ix-00 for icar@ietf.org; Mon, 12 Jan 2004 15:55:55 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1Ag94P-0007GF-00 for icar@ietf.org; Mon, 12 Jan 2004 15:53:57 -0500 Received: from sj-iport-1-in.cisco.com ([171.71.176.70] helo=sj-iport-1.cisco.com) by ietf-mx with esmtp (Exim 4.12) id 1Ag92Y-0007CV-00 for icar@ietf.org; Mon, 12 Jan 2004 15:52:02 -0500 Received: from mira-sjc5-c.cisco.com (IDENT:mirapoint@mira-sjc5-c.cisco.com [171.71.163.17]) by sj-core-1.cisco.com (8.12.9/8.12.6) with ESMTP id i0CKpUGN018520; Mon, 12 Jan 2004 12:51:31 -0800 (PST) Received: from cisco.com ([10.25.65.181]) by mira-sjc5-c.cisco.com (Mirapoint Messaging Server MOS 3.3.6-GR) with SMTP id APU80587; Mon, 12 Jan 2004 12:51:29 -0800 (PST) Date: Mon, 12 Jan 2004 15:51:26 -0500 Subject: Re: [Icar] ICAR draft charter rev 2 Content-Type: text/plain; charset=US-ASCII; format=flowed Mime-Version: 1.0 (Apple Message framework v553) From: Melinda Shore To: Alex Zinin , icar@ietf.org Content-Transfer-Encoding: 7bit In-Reply-To: Message-Id: <16A45988-4541-11D8-9AF0-000A95E35274@cisco.com> X-Mailer: Apple Mail (2.553) Content-Transfer-Encoding: 7bit Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.0 required=5.0 tests=AWL autolearn=no version=2.60 Content-Transfer-Encoding: 7bit Content-Transfer-Encoding: 7bit I think the proposed charter is in very good shape, in terms of providing the basis for a "contract" between the IETF and a new working group. It's clear, it's crisp, it's focused, it's unambiguous, and most importantly it describes work that we know needs to be done. Because chartering is an iterative (sometimes highly iterative) process I think it's time to try to get some feedback on this version from the probable area director (Harald) and the rest of the IESG. Melinda _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Mon Jan 12 16:00:32 2004 Received: from optimus.ietf.org ([132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA18825 for ; Mon, 12 Jan 2004 16:00:32 -0500 (EST) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1Ag9AA-0004k6-6N for icar-archive@odin.ietf.org; Mon, 12 Jan 2004 16:00:00 -0500 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i0CKxs49018226 for icar-archive@odin.ietf.org; Mon, 12 Jan 2004 15:59:54 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1Ag9AA-0004jt-1s for icar-web-archive@optimus.ietf.org; Mon, 12 Jan 2004 15:59:54 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA18747 for ; Mon, 12 Jan 2004 15:59:52 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1Ag9A8-0007Q5-00 for icar-web-archive@ietf.org; Mon, 12 Jan 2004 15:59:52 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1Ag98E-0007Mu-00 for icar-web-archive@ietf.org; Mon, 12 Jan 2004 15:57:55 -0500 Received: from [132.151.1.19] (helo=optimus.ietf.org) by ietf-mx with esmtp (Exim 4.12) id 1Ag96O-0007Jm-00 for icar-web-archive@ietf.org; Mon, 12 Jan 2004 15:56:00 -0500 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1Ag96O-0004fL-Rb; Mon, 12 Jan 2004 15:56:00 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1Ag96I-0004ey-NL for icar@optimus.ietf.org; Mon, 12 Jan 2004 15:55:54 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA18590 for ; Mon, 12 Jan 2004 15:55:52 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1Ag96H-0007Ih-00 for icar@ietf.org; Mon, 12 Jan 2004 15:55:53 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1Ag94N-0007Fx-00 for icar@ietf.org; Mon, 12 Jan 2004 15:53:56 -0500 Received: from ns.execdsl.net ([208.184.15.238] helo=EXECDSL.COM) by ietf-mx with esmtp (Exim 4.12) id 1Ag92W-0007Db-00 for icar@ietf.org; Mon, 12 Jan 2004 15:52:00 -0500 Received: from [64.254.114.114] (HELO JLaptop.stevecrocker.com) by EXECDSL.COM (CommuniGate Pro SMTP 3.3) with ESMTP id 6067979; Mon, 12 Jan 2004 15:51:59 -0500 Message-Id: <5.1.0.14.0.20040112154956.01a86878@localhost> X-Sender: joel@stevecrocker.com@localhost X-Mailer: QUALCOMM Windows Eudora Version 5.1 Date: Mon, 12 Jan 2004 15:51:42 -0500 To: Alex Zinin , icar@ietf.org From: "Joel M. Halpern" Subject: Re: [Icar] ICAR draft charter rev 2 In-Reply-To: <196108581021.20040108171716@psg.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.0 required=5.0 tests=none autolearn=no version=2.60 Having seen the discussion since this was posted, I thought it sensible to comment that I thought this revision provided a good frame for the working group. It defines the goals, some of the approaches, and a set of observable milestones in reasonable time frames. Yours, Joel M. Halpern At 05:17 PM 1/8/2004 -0800, Alex Zinin wrote: >Revision 2 below, please. > >-- >Alex > > WG name: improved cross-area review (icar) > > Chairs: > > General Area Director(s): > Harald Alvestrand > > General Area Advisor: > Harald Alvestrand > > Mailing list: icar@ietf.org > Subscription: icar-request@ietf.org > Archives : > https://www1.ietf.org/mail-archive/working-groups/icar/current/maillist.html > > WG Description: > > The WG will work out mechanisms for improved cross-functional > review within the IETF, addressing the issues identified by the > PROBLEM WG. This includes a better community review, as well as > more structured pre-IESG review that may be used to improve > scalability of the IESG review function. > > Definitions: > > o Cross-functional review: document review covering different > aspects of Internet technology, including those represented by > WGs within different IETF areas. > > o Community review: document review performed by individual IETF > participants and not caused by their responsibilities within > the IETF management structure. > > o Structured review: more formal and role-based document review > performed by individuals assuming certain responsibilities (such > as by WG chairs, directorate and IESG members.) > > It is an explicit goal of the WG to come up with mechanisms > encouraging earlier review of the documents. An early review is > best for catching architectural problems while they're still > relatively easy to solve. In particular, many cross-functional > interactions can be spotted and dealt with, thus avoiding many > "late surprises". A final review (currently done by the IESG) can > catch remaining cross-functional interactions, as well as deal > with overall quality issues. > > The WG will cooperate with others in starting and evaluating > experiments with early community and structured reviews. The > evaluation of such experiments may be published as Informational > RFCs if the group so desires. > > The WG will also coordinate with other WGs involved in the IETF > reform process on proposed changes to the IETF Working Group > operations and Standards process if those are considered > necessary. > > WG milestones: > > FEB 2004: Submit drafts on improved community and structured reviews > SEP 2004: Submit draft on improved community review to > the IESG for publication as BCP > SEP 2004: Submit draft on improved structured community review to > the IESG for publication as BCP > SEP 2005: Evaluate WG progress and potential; close or recharter > > >_______________________________________________ >Icar mailing list >Icar@ietf.org >https://www1.ietf.org/mailman/listinfo/icar _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Mon Jan 12 16:08:56 2004 Received: from optimus.ietf.org ([132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA19522 for ; Mon, 12 Jan 2004 16:08:56 -0500 (EST) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1Ag9IR-0005Vb-Im for icar-archive@odin.ietf.org; Mon, 12 Jan 2004 16:08:28 -0500 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i0CL8RGB021169 for icar-archive@odin.ietf.org; Mon, 12 Jan 2004 16:08:27 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1Ag9IR-0005VM-Dd for icar-web-archive@optimus.ietf.org; Mon, 12 Jan 2004 16:08:27 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA19444 for ; Mon, 12 Jan 2004 16:08:25 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1Ag9IP-000019-00 for icar-web-archive@ietf.org; Mon, 12 Jan 2004 16:08:25 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1Ag9GJ-0007gY-00 for icar-web-archive@ietf.org; Mon, 12 Jan 2004 16:06:16 -0500 Received: from [132.151.1.19] (helo=optimus.ietf.org) by ietf-mx with esmtp (Exim 4.12) id 1Ag9F7-0007cS-00 for icar-web-archive@ietf.org; Mon, 12 Jan 2004 16:05:01 -0500 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1Ag9F8-0004s8-Sm; Mon, 12 Jan 2004 16:05:02 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1Ag9EG-0004r4-G4 for icar@optimus.ietf.org; Mon, 12 Jan 2004 16:04:08 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA19023 for ; Mon, 12 Jan 2004 16:04:06 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1Ag9EE-0007Zq-00 for icar@ietf.org; Mon, 12 Jan 2004 16:04:06 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1Ag9CO-0007X3-00 for icar@ietf.org; Mon, 12 Jan 2004 16:02:12 -0500 Received: from measurement-factory.com ([206.168.0.5]) by ietf-mx with esmtp (Exim 4.12) id 1Ag9BS-0007TY-00 for icar@ietf.org; Mon, 12 Jan 2004 16:01:14 -0500 Received: from measurement-factory.com (localhost [127.0.0.1]) by measurement-factory.com (8.12.9/8.12.9) with ESMTP id i0CL1Dk3026752; Mon, 12 Jan 2004 14:01:13 -0700 (MST) (envelope-from rousskov@measurement-factory.com) Received: (from rousskov@localhost) by measurement-factory.com (8.12.9/8.12.9/Submit) id i0CL1CPa026751; Mon, 12 Jan 2004 14:01:12 -0700 (MST) (envelope-from rousskov) Date: Mon, 12 Jan 2004 14:01:12 -0700 (MST) From: Alex Rousskov To: "Joel M. Halpern" cc: icar@ietf.org, solutions@alvestrand.no In-Reply-To: <5.1.0.14.0.20040110013737.018ef758@localhost> Message-ID: References: <003701c3d737$5b361530$386015ac@dclkempt40> <5.1.0.14.2.20040109203410.04552a28@ms101.mail1.com> <003701c3d737$5b361530$386015ac@dclkempt40> <5.1.0.14.0.20040110013737.018ef758@localhost> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Subject: [Icar] Re: [Solutions] Summary of Discussion on Reforming IETF Quality Control Process Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.0 required=5.0 tests=AWL autolearn=no version=2.60 On Sat, 10 Jan 2004, Joel M. Halpern wrote: > I may be missing something, but the problem I see reading this is > that this gives the IESG no assistance in deciding which documents > it needs to review. Why do you think IESG needs an assistance with this? Can IESG decide on a case by case basis, as ADs see new documents popping up in a "now soliciting reviews" state? What kind of assistance do you have in mind? > There was another proposal which I read to say ~the IESG needs to > review documents for which there is disagreement about the results > of other reviews.~ While one can argue about whether that is good or > not, it at least spells out when the IESG needs to review a > document, and when it does not need to. You may be talking about the "IESG must resolve reviewer-WG conflicts" rule I have been pushing for. That rule co-exists nicely with "IESG may review any document as any IETF participant can" rule. Put together, the two rules essentially define how IESG interacts with WGs as far as reviews are concerned: first as a normal reviewer, then as the final conflict resolution authority if needed. > ... my understanding of one aspect of this discussion is to help > replace the current ~the IESG must review everything~ with a useful > alternative review procedure, and a useful definition of which > things the IESG needs to review. I do not think we can pre-define what IESG needs to review without being either too broad or too restrictive. I would rather avoid the whole problem by letting IESG decide what warrants full review. Note that there is some balance in this approach: since IESG is just a "regular" reviewer, they are subject to the same deadlines and timeouts. Thus, IESG (as a whole) should be careful not to overload itself with non-essential reviews because they will start missing deadlines on essential ones. > The text below seems to say taht there are no documents the IESG > needs to review. Not exactly. The text below implies that we cannot define a priori what IESG needs to review, but we give IESG an interface to review anything that IESG thinks is important enough to be worth their review. Alex. > At 10:43 PM 1/9/2004 -0700, Alex Rousskov wrote: > >There should be no "requires full IESG review" flag or state for > >any document. IESG is not a special case when it comes to review. > >IESG or any single AD can submit a review for any document that is > >up for review, at any time. This is no different from any IETF > >participant submitting a review. If IESG feels that a particular > >document needs full IESG review, it is their internal business, > >invisible and unpredictable to others (in general), until they > >submit a review. In general, one does not know a priori whether a > >document will be reviewed by IESG until the IESG submits the > >review. _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Mon Jan 12 17:21:21 2004 Received: from optimus.ietf.org ([132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA25624 for ; Mon, 12 Jan 2004 17:21:20 -0500 (EST) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AgAQV-00089Y-MI for icar-archive@odin.ietf.org; Mon, 12 Jan 2004 17:20:53 -0500 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i0CMKpDU031330 for icar-archive@odin.ietf.org; Mon, 12 Jan 2004 17:20:51 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AgAQQ-00089F-NE for icar-web-archive@optimus.ietf.org; Mon, 12 Jan 2004 17:20:46 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA25605 for ; Mon, 12 Jan 2004 17:20:43 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1AgAQO-0005pp-00 for icar-web-archive@ietf.org; Mon, 12 Jan 2004 17:20:44 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1AgAOb-0005hr-00 for icar-web-archive@ietf.org; Mon, 12 Jan 2004 17:18:54 -0500 Received: from [132.151.1.19] (helo=optimus.ietf.org) by ietf-mx with esmtp (Exim 4.12) id 1AgAMl-0005XY-00 for icar-web-archive@ietf.org; Mon, 12 Jan 2004 17:16:59 -0500 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AgAMm-000838-QQ; Mon, 12 Jan 2004 17:17:00 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AgAMK-00082r-Bl for icar@optimus.ietf.org; Mon, 12 Jan 2004 17:16:32 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA25332 for ; Mon, 12 Jan 2004 17:16:29 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1AgAMI-0005Wa-00 for icar@ietf.org; Mon, 12 Jan 2004 17:16:30 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1AgAKY-0005S2-00 for icar@ietf.org; Mon, 12 Jan 2004 17:14:43 -0500 Received: from measurement-factory.com ([206.168.0.5]) by ietf-mx with esmtp (Exim 4.12) id 1AgAJL-0005Kl-00 for icar@ietf.org; Mon, 12 Jan 2004 17:13:27 -0500 Received: from measurement-factory.com (localhost [127.0.0.1]) by measurement-factory.com (8.12.9/8.12.9) with ESMTP id i0CMDOk3029479; Mon, 12 Jan 2004 15:13:24 -0700 (MST) (envelope-from rousskov@measurement-factory.com) Received: (from rousskov@localhost) by measurement-factory.com (8.12.9/8.12.9/Submit) id i0CMDNjn029478; Mon, 12 Jan 2004 15:13:23 -0700 (MST) (envelope-from rousskov) Date: Mon, 12 Jan 2004 15:13:23 -0700 (MST) From: Alex Rousskov To: Margaret Wasserman cc: James Kempf , icar@ietf.org, solutions@alvestrand.no In-Reply-To: <5.1.0.14.2.20040110075158.0385bbd0@ms101.mail1.com> Message-ID: References: <5.1.0.14.2.20040109203410.04552a28@ms101.mail1.com> <5.1.0.14.2.20040110075158.0385bbd0@ms101.mail1.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Subject: [Icar] Re: [mpowr] Re: [Solutions] Summary of Discussion on Reforming IETF Quality Control Process Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=1.0 required=5.0 tests=AWL,DRASTIC_REDUCED autolearn=no version=2.60 On Sat, 10 Jan 2004, Margaret Wasserman wrote: > These concerns are what lead me to originally prefer the type of > per-area review board that Alex Zinin has suggested, rather than a > single large (SIRS-like) review board as you suggest. I believe > that it would be possible to give single ADs the ability to approve > non-critical documents (criteria TBD) based on reviews by all of the > per-area boards I see no essential difference between a single board with members that specialize in certain areas and multiple boards with members specializing in board-specific area. From the point of view of the important criteria you have been talking about, both designs are essentially the same, IMO. I am not sure what concerns make one design more attractive than another (on this level of abstraction). I suspect the devil is in the details instead of high-level architectures. For example, "what reviews are sufficient to publish a document" is a question that can be answered equally well/poor with both single and multiple boards structures. If we concentrate on cross-area review and possibly lack volunteers, a single board/pool sounds like a more natural framework. > I think that we can somewhat consider the structure/management of > the review board(s) to be orthogonal to whether some documents can > be approved without full IESG review. I agree. > SCALABILITY > =========== > > Like the SIRS proposal, I believe that this proposal is quite > naive about the scale of this particular problem. The draft-list-quality-control scalability depends on how many people the ADs can approve as "certified reviewers" without approving bad reviewers. I do not see how that process is essentially different from any other process where there is a "trusted by final Authority" flag assigned to a reviewer in one way or another. If that flag is required for review consistency, then all approaches would scale equally well/badly. It is not a problem we can "solve" by using a process. We can only eliminate the scalability problem if we remove the flag and, hence, require IESG to trust at least a certain portion of unknown-to-IESG reviewers. For example, we can say that any document that has no review conflicts and has at least K reviews from each Area, with at least N reviews from "trusted by AD/IESG" reviewers is published without IESG involvement. Then you get a natural trade-off between scalability (number of trusted reviewers required) and consistency/quality (probability of IESG last-minute killing a document without a prior review conflict and probability of IETF publishing a bad document without a review conflict). > CONSISTENCY > =========== > > There are two types of consistency that are important for IETF > document reviews: > > (1) Consistency across the different reviews done for the > same document (to avoid thrash). I do not understand what that means. Can you elaborate please? Consistency in reviewer opinions during a round of reviews? Consistency in reviewer opinions during the document lifetime? > (2) Using a consistent set of acceptance criteria for each > document that is reviewed/approved. This is important indeed. > Type (1) consistency could be achieved by having the same set of > reviewers (to whatever extent possible) perform all of the > reviews for a given document. This can be achieved in any proposal I have seen, including draft-list-quality-control-00, right? > Type (2) consistency is _much_ harder to achieve across a large > board (~200 people in my example above) than it is with a group of > 13 people who spend several hours per week on the phone with each > other, hold retreats and communicate daily... I do not see a difference. If you can define acceptance criteria well, then 10, 100, or 1000 people can use them with good-enough consistency. If your acceptance criteria are so vague that they can only exist as an undocumented belief system of 13 close friends (different from the belief system of the other 13 friends), then we probably do not need that kind of consistency. > In order to achieve even a reasonable level of consistency across > a large group (200 people?), I believe that we would need most or > all of the following things: > > - Written review criteria for each type of document. > - Documented per-area review criteria (like the MIB nits) for > every area or technical sub-area. > - A record of architectural/policy decisions that were made > that can be searched and used by subsequent reviewers. > - A mandatory preparation/training program (in-person or > on-line) for all new members of the review board. > - Some type of mentoring or monitoring program for new > members during the first NN reviews (most easily > achieved, perhaps, by having some sort of structure > within the board?). > > Without a plan (including committed resources) to achieve these > things, I believe that a large review board would devolve into > chaos. How is N*13 reviewers groups would be any different? I suspect there is either violation of the energy preservation laws in these arguments or you are comparing apples to oranges. The amount of IETF review work that needs to be done should not change depending on whether you use pre-area boards or one single pool. More precisely, if the amount of work changes, then we are not comparing apples to apples and we should decide on what the review system should accomplish first. > There may also be an issue with documents that are passed to the > IESG for final approval. The IESG's review may not be consistent > with earlier reviews, perhaps resulting in "late surprises" or > demotivation of the WG. This problem is most likely in a situation > where the IESG has no influence over what reviewers are chosen to do > the initial review, and where the IESG does not feel accountable for > the quality/correctness of the initial review (see section on > accountability below). I assumed that consistency with IESG decisions is included in (2). If it is not, I do not understand your classification scheme, but the laws of physics should still apply. > CROSS-AREA COVERAGE > =================== > > this proposal doesn't seem to do much to support or > facilitate cross-area review. I agree. The proposal requires an AD to ensure that all areas are covered. This is a poor design because AD is responsible for one area only. We should find a scheme where all areas are represented when it comes to cross-area review. For example, each area can have a list of "trusted area reviewers" (all members of the IETF review board/pool). And at least one(?) review must come from each area. This will ensure that all areas are represented without relying on one area AD to take care of that. > Per-area review boards have a major advantage in this area. If > a document is passed by every per-area review board (particularly > if those boards are carefully chosen to cover their area well > and provide coverage for inter-area gaps), with the reviewers > chosen by the ADs or other area review board leaders who aren't > active in the WG, then it is more likely that a document will > have received adequate cross-area review than if it is reviewed > by a set of reviewers chosen from a large pool by the WG chair. Indeed. However, what you describe as a "major advantage" can be implemented equally well with a single pool design. Each reviewer has a set of tags, including flags that mark that reviewer as a trusted reviewer by an IETF Area Directorate. In fact, I can argue that a single pool would work marginally better in some cases because Chairs can easily find reviewers that are trusted by several areas and, hence, reduce the number of reviews required. Common pool and interface makes ensuring process rules easier as well. > EFFICIENCY > ========== > > This proposal might achieve this goal for some documents, > particularly those that do not require IESG review. However, the > most important documents might be subjected to two rounds of > review/approval -- one by the Quality Review Board and another by > the IESG. This issue might be resolved by better defining how/when > a decision is made regarding whether a document will be > reviewed/approved by the Quality Review Board or the IESG. I agree and proposed some specific interfaces to solve this problem (to the extent it can be solved when IESG authority differs from the Board authority). > MANAGEABILITY > ============= > > In order to run an orderly review process, to assign reviewers > with appropriate (combinations of) expertise, to adquately > training reviewers and to maintain the quality of the review > process, the Quality Review Board would need to be managed. > This proposal does not indicate how the board would be managed, > who would manage it, how they would be chosen, etc. This is > a major omission that makes it difficult to evaluate many > aspects of this proposal. I tend to agree. However, I do note that the amount of required management can be drastically reduced if we adopt a self-tuning open pool of reviewers instead of going with a "traditional" managed board(s) approach so typical to other SDOs and other non-volunteer organizations. > The per-area review board proposal relies on the ADs to select > reviewers and manage the process as appropriate to each area. > The AD would be responsible for ensuring that members of the > area boards are adequately trained and prepared, and would > be held accountable for the quality of their work. ... which has exactly the same manageability problems and a cross-area consistency problem added on top. IMO, IETF areas are large enough to be comparable (in scale) with IETF as a whole when it comes to manageability > ACCOUNTABILITY > ============== > > This document seems to be based on the concept that quality > review is performed for the WGs, and that the review process > should be accountable to the WGs. Ultimately, reviewers report > to WG chairs, because they if they are not asked (by WG chairs) > to review at least 3 document per year, they are removed. > I believe that is wrong... It would be wrong, but I disagree with your summary. While it is not explicitly documented, the Review Board essentially works for and accountable to IETF as a whole (just like a WG does/is). It should be explicitly documented. There are some minor mechanisms bugs (like requiring the Chair to enter comments) that may create an impression that reviewers report to WG chairs. I already commented on those; they can be fixed. N.B. idle reviewers are _not_ removed. Only those that idle but also rejecting review requests are. > Another advantage of the per-area review board proposal over this > proposal is that it builds upon, rather than replacing, the work > that has already been done to build per-area review teams such as > the MIB doctors, the ops-dir, Transport doctors, the rtg-dir, etc. Members of those teams should be encouraged to volunteer to join the open pool of reviewers, of course. We should not replace them, but provide better access to and visibility for them! Alex. _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Mon Jan 12 17:28:42 2004 Received: from optimus.ietf.org ([132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA25895 for ; Mon, 12 Jan 2004 17:28:42 -0500 (EST) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AgAXd-0008K0-DA for icar-archive@odin.ietf.org; Mon, 12 Jan 2004 17:28:13 -0500 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i0CMSCJM031972 for icar-archive@odin.ietf.org; Mon, 12 Jan 2004 17:28:12 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AgAXa-0008JX-NO for icar-web-archive@optimus.ietf.org; Mon, 12 Jan 2004 17:28:10 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA25873 for ; Mon, 12 Jan 2004 17:28:07 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1AgAXY-00067h-00 for icar-web-archive@ietf.org; Mon, 12 Jan 2004 17:28:08 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1AgAVh-000647-00 for icar-web-archive@ietf.org; Mon, 12 Jan 2004 17:26:14 -0500 Received: from [132.151.1.19] (helo=optimus.ietf.org) by ietf-mx with esmtp (Exim 4.12) id 1AgAUX-000613-00 for icar-web-archive@ietf.org; Mon, 12 Jan 2004 17:25:01 -0500 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AgAUZ-0008Fb-36; Mon, 12 Jan 2004 17:25:03 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AgATg-0008EH-Tg for icar@optimus.ietf.org; Mon, 12 Jan 2004 17:24:08 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA25798 for ; Mon, 12 Jan 2004 17:24:05 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1AgATe-0005xw-00 for icar@ietf.org; Mon, 12 Jan 2004 17:24:06 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1AgARs-0005uW-00 for icar@ietf.org; Mon, 12 Jan 2004 17:22:16 -0500 Received: from m106.maoz.com ([205.167.76.9]) by ietf-mx with esmtp (Exim 4.12) id 1AgAQl-0005or-00 for icar@ietf.org; Mon, 12 Jan 2004 17:21:08 -0500 Received: from m106.maoz.com (localhost.localdomain [127.0.0.1]) by m106.maoz.com (8.12.10/8.12.10) with ESMTP id i0CMKb6f018299; Mon, 12 Jan 2004 14:20:37 -0800 Received: (from dmm@localhost) by m106.maoz.com (8.12.10/8.12.10/Submit) id i0CMKbEx018298; Mon, 12 Jan 2004 14:20:37 -0800 X-Authentication-Warning: m106.maoz.com: dmm set sender to dmm@1-4-5.net using -f Date: Mon, 12 Jan 2004 14:20:37 -0800 From: David Meyer To: icar@ietf.org Subject: Re: [Icar] ICAR draft charter rev 2 Message-ID: <20040112222037.GA18177@1-4-5.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.4.1i X-public-key: http://www.1-4-5.net/~dmm/public-key.asc X-philosophy: "I just had to let it go" -- John Lennon Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.0 required=5.0 tests=AWL autolearn=no version=2.60 This looks good. Just a few nits: (i). What does "submit drafts" mean? I assume (again) that we mean "produce WG document" (which in turn means that the document name is of the form draft-ietf-icar-xxxx, was adopted by WG consensus, and it can be found on ftp://ftp.ietf.org/internet-drafts); this is in contrast to "submit draft ... to IESG..." (all of this is really a nit an belongs in a discussion of how one writes a charter, I suppose), (ii). Can we be more precise about what "earlier" means in "It is an explicit goal of the WG to come up with mechanisms encouraging earlier review of the documents"? My first question when I read this was "earlier than what"? (iii). SEP 2005: Evaluate WG progress and potential; close or recharter What does it mean (what is meant) by evaluating the WG "potential" (i.e., we could be talking about a kind of potential s/a potential energy, but I'm pretty sure that isn't what is entended). Other than that, the charter defines some reasonable goals within a (importantly) constrained scope, and time frames. Dave >> Revision 2 below, please. >> >> -- >> Alex >> >> WG name: improved cross-area review (icar) >> >> Chairs: >> >> General Area Director(s): >> Harald Alvestrand >> >> General Area Advisor: >> Harald Alvestrand >> >> Mailing list: icar@ietf.org >> Subscription: icar-request@ietf.org >> Archives : https://www1.ietf.org/mail-archive/working-groups/icar/current/maillist.html >> >> WG Description: >> >> The WG will work out mechanisms for improved cross-functional >> review within the IETF, addressing the issues identified by the >> PROBLEM WG. This includes a better community review, as well as >> more structured pre-IESG review that may be used to improve >> scalability of the IESG review function. >> >> Definitions: >> >> o Cross-functional review: document review covering different >> aspects of Internet technology, including those represented by >> WGs within different IETF areas. >> >> o Community review: document review performed by individual IETF >> participants and not caused by their responsibilities within >> the IETF management structure. >> >> o Structured review: more formal and role-based document review >> performed by individuals assuming certain responsibilities (such >> as by WG chairs, directorate and IESG members.) >> >> It is an explicit goal of the WG to come up with mechanisms >> encouraging earlier review of the documents. An early review is >> best for catching architectural problems while they're still >> relatively easy to solve. In particular, many cross-functional >> interactions can be spotted and dealt with, thus avoiding many >> "late surprises". A final review (currently done by the IESG) can >> catch remaining cross-functional interactions, as well as deal >> with overall quality issues. >> >> The WG will cooperate with others in starting and evaluating >> experiments with early community and structured reviews. The >> evaluation of such experiments may be published as Informational >> RFCs if the group so desires. >> >> The WG will also coordinate with other WGs involved in the IETF >> reform process on proposed changes to the IETF Working Group >> operations and Standards process if those are considered >> necessary. >> >> WG milestones: >> >> FEB 2004: Submit drafts on improved community and structured reviews >> SEP 2004: Submit draft on improved community review to >> the IESG for publication as BCP >> SEP 2004: Submit draft on improved structured community review to >> the IESG for publication as BCP >> SEP 2005: Evaluate WG progress and potential; close or recharter >> >> >> _______________________________________________ >> Icar mailing list >> Icar@ietf.org >> https://www1.ietf.org/mailman/listinfo/icar >> >> ===8<===========End of original message text=========== _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Mon Jan 12 17:32:44 2004 Received: from optimus.ietf.org ([132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA26036 for ; Mon, 12 Jan 2004 17:32:44 -0500 (EST) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AgAbY-0008Qc-66 for icar-archive@odin.ietf.org; Mon, 12 Jan 2004 17:32:16 -0500 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i0CMWFPc032391 for icar-archive@odin.ietf.org; Mon, 12 Jan 2004 17:32:15 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AgAbV-0008QI-RL for icar-web-archive@optimus.ietf.org; Mon, 12 Jan 2004 17:32:13 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA26024 for ; Mon, 12 Jan 2004 17:32:10 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1AgAbT-0006HY-00 for icar-web-archive@ietf.org; Mon, 12 Jan 2004 17:32:11 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1AgAZf-0006Ek-00 for icar-web-archive@ietf.org; Mon, 12 Jan 2004 17:30:20 -0500 Received: from [132.151.1.19] (helo=optimus.ietf.org) by ietf-mx with esmtp (Exim 4.12) id 1AgAYU-0006An-00 for icar-web-archive@ietf.org; Mon, 12 Jan 2004 17:29:06 -0500 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AgAYV-0008LT-7q; Mon, 12 Jan 2004 17:29:07 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AgAXo-0008Kg-P0 for icar@optimus.ietf.org; Mon, 12 Jan 2004 17:28:24 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA25892 for ; Mon, 12 Jan 2004 17:28:21 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1AgAXm-00069n-00 for icar@ietf.org; Mon, 12 Jan 2004 17:28:22 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1AgAVn-00065R-00 for icar@ietf.org; Mon, 12 Jan 2004 17:26:20 -0500 Received: from key1.docomolabs-usa.com ([216.98.102.225] helo=fridge.docomolabs-usa.com ident=fwuser) by ietf-mx with esmtp (Exim 4.12) id 1AgAV0-000619-00 for icar@ietf.org; Mon, 12 Jan 2004 17:25:30 -0500 Message-ID: <031701c3d95b$0c997220$606015ac@dclkempt40> From: "James Kempf" To: Date: Mon, 12 Jan 2004 14:25:56 -0800 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Transfer-Encoding: 7bit Subject: [Icar] Charter Comments Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.0 required=5.0 tests=AWL autolearn=no version=2.60 Content-Transfer-Encoding: 7bit Content-Transfer-Encoding: 7bit The charter looks fine as far as it goes. It is a good start. jak _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Mon Jan 12 18:06:52 2004 Received: from optimus.ietf.org ([132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id SAA28101 for ; Mon, 12 Jan 2004 18:06:52 -0500 (EST) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AgB8b-0000zP-VR for icar-archive@odin.ietf.org; Mon, 12 Jan 2004 18:06:25 -0500 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i0CN6PpR003803 for icar-archive@odin.ietf.org; Mon, 12 Jan 2004 18:06:25 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AgB8b-0000zG-P7 for icar-web-archive@optimus.ietf.org; Mon, 12 Jan 2004 18:06:25 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id SAA28048 for ; Mon, 12 Jan 2004 18:06:22 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1AgB8Z-0000Ou-00 for icar-web-archive@ietf.org; Mon, 12 Jan 2004 18:06:23 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1AgB72-0000Ij-00 for icar-web-archive@ietf.org; Mon, 12 Jan 2004 18:04:48 -0500 Received: from [132.151.1.19] (helo=optimus.ietf.org) by ietf-mx with esmtp (Exim 4.12) id 1AgB5H-00009O-00 for icar-web-archive@ietf.org; Mon, 12 Jan 2004 18:02:59 -0500 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AgB5I-0000v5-Qd; Mon, 12 Jan 2004 18:03:00 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AgB4W-0000tv-9I for icar@optimus.ietf.org; Mon, 12 Jan 2004 18:02:12 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id SAA27476 for ; Mon, 12 Jan 2004 18:02:08 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1AgB4T-00000X-00 for icar@ietf.org; Mon, 12 Jan 2004 18:02:09 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1AgB2U-0007eG-00 for icar@ietf.org; Mon, 12 Jan 2004 18:00:06 -0500 Received: from ns.execdsl.net ([208.184.15.238] helo=EXECDSL.COM) by ietf-mx with esmtp (Exim 4.12) id 1AgB0f-0007Y3-00 for icar@ietf.org; Mon, 12 Jan 2004 17:58:13 -0500 Received: from [64.254.114.114] (HELO JLaptop.stevecrocker.com) by EXECDSL.COM (CommuniGate Pro SMTP 3.3) with ESMTP id 6068499 for icar@ietf.org; Mon, 12 Jan 2004 17:58:13 -0500 Message-Id: <5.1.0.14.0.20040112175429.01a88040@localhost> X-Sender: joel@stevecrocker.com@localhost X-Mailer: QUALCOMM Windows Eudora Version 5.1 Date: Mon, 12 Jan 2004 17:57:55 -0500 To: icar@ietf.org From: "Joel M. Halpern" Subject: Re: [Icar] ICAR draft charter rev 2 In-Reply-To: <20040112222037.GA18177@1-4-5.net> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.0 required=5.0 tests=none autolearn=no version=2.60 My personal read of "submit drafts" was Have one or more I-Ds in the repository which the working group is discussing. These may not at that stage (I would guess will not) reflect working group consensus. In other groups I have seen the use of the naming scheme draft-person-icar-xxx used to reflect this stage of the process. I would not expect the charter to have to get too specific about this, as the process of marking the documents as under discussion should be anything the WG and chairs are comfortable with, not a chartering issue. Yours, Joel At 02:20 PM 1/12/2004 -0800, David Meyer wrote: > (i). What does "submit drafts" mean? I assume (again) > that we mean "produce WG document" (which in turn > means that the document name is of the form > draft-ietf-icar-xxxx, was adopted by WG > consensus, and it can be found on > ftp://ftp.ietf.org/internet-drafts); this is in > contrast to "submit draft ... to IESG..." (all of > this is really a nit an belongs in a discussion > of how one writes a charter, I suppose), _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Mon Jan 12 18:32:36 2004 Received: from optimus.ietf.org ([132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id SAA29497 for ; Mon, 12 Jan 2004 18:32:36 -0500 (EST) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AgBXV-0001gm-KU for icar-archive@odin.ietf.org; Mon, 12 Jan 2004 18:32:09 -0500 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i0CNW9XE006491 for icar-archive@odin.ietf.org; Mon, 12 Jan 2004 18:32:09 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AgBXV-0001gc-Bk for icar-web-archive@optimus.ietf.org; Mon, 12 Jan 2004 18:32:09 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id SAA29488 for ; Mon, 12 Jan 2004 18:32:05 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1AgBXS-0001O3-00 for icar-web-archive@ietf.org; Mon, 12 Jan 2004 18:32:06 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1AgBVg-0001KH-00 for icar-web-archive@ietf.org; Mon, 12 Jan 2004 18:30:17 -0500 Received: from [132.151.1.19] (helo=optimus.ietf.org) by ietf-mx with esmtp (Exim 4.12) id 1AgBUR-0001G0-00 for icar-web-archive@ietf.org; Mon, 12 Jan 2004 18:28:59 -0500 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AgBUS-0001cb-PG; Mon, 12 Jan 2004 18:29:00 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AgBTb-0001an-CK for icar@optimus.ietf.org; Mon, 12 Jan 2004 18:28:07 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id SAA29353 for ; Mon, 12 Jan 2004 18:28:03 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1AgBTY-0001CU-00 for icar@ietf.org; Mon, 12 Jan 2004 18:28:04 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1AgBRe-00016n-00 for icar@ietf.org; Mon, 12 Jan 2004 18:26:07 -0500 Received: from m106.maoz.com ([205.167.76.9]) by ietf-mx with esmtp (Exim 4.12) id 1AgBQ0-00010M-00 for icar@ietf.org; Mon, 12 Jan 2004 18:24:24 -0500 Received: from m106.maoz.com (localhost.localdomain [127.0.0.1]) by m106.maoz.com (8.12.10/8.12.10) with ESMTP id i0CNNs6f019895; Mon, 12 Jan 2004 15:23:54 -0800 Received: (from dmm@localhost) by m106.maoz.com (8.12.10/8.12.10/Submit) id i0CNNsX3019894; Mon, 12 Jan 2004 15:23:54 -0800 X-Authentication-Warning: m106.maoz.com: dmm set sender to dmm@1-4-5.net using -f Date: Mon, 12 Jan 2004 15:23:54 -0800 From: David Meyer To: "Joel M. Halpern" Cc: icar@ietf.org Subject: Re: [Icar] ICAR draft charter rev 2 Message-ID: <20040112232354.GA19886@1-4-5.net> References: <20040112222037.GA18177@1-4-5.net> <5.1.0.14.0.20040112175429.01a88040@localhost> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <5.1.0.14.0.20040112175429.01a88040@localhost> User-Agent: Mutt/1.4.1i X-public-key: http://www.1-4-5.net/~dmm/public-key.asc X-philosophy: "I just had to let it go" -- John Lennon Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.0 required=5.0 tests=AWL autolearn=no version=2.60 On Mon, Jan 12, 2004 at 05:57:55PM -0500, Joel M. Halpern wrote: >> My personal read of "submit drafts" was >> Have one or more I-Ds in the repository which the working group is >> discussing. >> >> These may not at that stage (I would guess will not) reflect working group >> consensus. In other groups I have seen the use of the naming scheme >> draft-person-icar-xxx used to reflect this stage of the process. Yep. >> I would not expect the charter to have to get too specific about this, as >> the process of marking the documents as under discussion should be anything >> the WG and chairs are comfortable with, not a chartering issue. Agree. As I said, a nit that came to mind as I was reading through the charter. Dave _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Tue Jan 13 10:46:51 2004 Received: from optimus.ietf.org ([132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id KAA19994 for ; Tue, 13 Jan 2004 10:46:51 -0500 (EST) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AgQkK-0002F9-MW for icar-archive@odin.ietf.org; Tue, 13 Jan 2004 10:46:24 -0500 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i0DFkOAK008617 for icar-archive@odin.ietf.org; Tue, 13 Jan 2004 10:46:24 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AgQkK-0002Ep-84 for icar-web-archive@optimus.ietf.org; Tue, 13 Jan 2004 10:46:24 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id KAA19982 for ; Tue, 13 Jan 2004 10:46:20 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1AgQkH-0007Jk-00 for icar-web-archive@ietf.org; Tue, 13 Jan 2004 10:46:21 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1AgQiQ-0007HY-00 for icar-web-archive@ietf.org; Tue, 13 Jan 2004 10:44:26 -0500 Received: from [132.151.1.19] (helo=optimus.ietf.org) by ietf-mx with esmtp (Exim 4.12) id 1AgQh2-0007F3-00 for icar-web-archive@ietf.org; Tue, 13 Jan 2004 10:43:00 -0500 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AgQh3-0002AS-9V; Tue, 13 Jan 2004 10:43:01 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AgQgK-00029U-Nc for icar@optimus.ietf.org; Tue, 13 Jan 2004 10:42:16 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id KAA19790 for ; Tue, 13 Jan 2004 10:42:13 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1AgQgI-0007CX-00 for icar@ietf.org; Tue, 13 Jan 2004 10:42:14 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1AgQeT-0007A3-00 for icar@ietf.org; Tue, 13 Jan 2004 10:40:22 -0500 Received: from sj-iport-4.cisco.com ([171.68.10.86]) by ietf-mx with esmtp (Exim 4.12) id 1AgQcy-00072u-00 for icar@ietf.org; Tue, 13 Jan 2004 10:38:48 -0500 Received: from cisco.com (erosen-u10.cisco.com [161.44.70.36]) by rtp-core-1.cisco.com (8.12.9/8.12.6) with ESMTP id i0DFcDLE029186; Tue, 13 Jan 2004 10:38:13 -0500 (EST) Message-Id: <200401131538.i0DFcDLE029186@rtp-core-1.cisco.com> To: Alex Zinin cc: icar@ietf.org Subject: Re: [Icar] ICAR draft charter In-reply-to: Your message of Wed, 07 Jan 2004 12:25:03 -0800. <1044648133.20040107122503@psg.com> Reply-To: erosen@cisco.com User-Agent: EMH/1.14.1 SEMI/1.14.3 (Ushinoya) FLIM/1.14.3 (=?ISO-8859-4?Q?Unebigory=F2mae?=) APEL/10.3 Emacs/21.3 (sparc-sun-solaris2.8) MULE/5.0 (SAKAKI) MIME-Version: 1.0 (generated by SEMI 1.14.3 - "Ushinoya") Content-Type: text/plain; charset=US-ASCII Date: Tue, 13 Jan 2004 10:38:12 -0500 From: Eric Rosen Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.0 required=5.0 tests=none autolearn=no version=2.60 I think the charter's fine. Whether the group will actually achieve something is another question. Getting early cross-functional review is a great idea, at least in theory, but in practice I have the following concerns: - While I would welcome an APPs guy's perspective on how a routing change might impact APPs, I don't necessarily want the APPs guys telling me how to do routing. For cross-functional review to be effective, the reviewers and reviewees must respect each others' knowledge in their respective areas of expertise. In the absence of this mutual respect, the whole thing is just an exercise in politics. - If one really wants to know, e.g., how a security change will impact routing, it might take a security guy and a routing guy working together to figure out all the inter-relationships. That's a bit different than throwing the spec over the wall for someone in another field to review. - Everybody will want to see the trade-offs made in such a way as to create the simplest result their own area. This will lead it irresolvable conflicts (as we frequently see on the main IETF list). It is also easy to demand that other areas do things that are beyond the state of the art in that area. I look forward to seeing what can be done to ensure that the cross-functional review doesn't become counter-productive. _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Tue Jan 13 11:41:02 2004 Received: from optimus.ietf.org ([132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA23145 for ; Tue, 13 Jan 2004 11:41:02 -0500 (EST) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AgRaj-00043d-5N for icar-archive@odin.ietf.org; Tue, 13 Jan 2004 11:40:33 -0500 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i0DGeWa2015591 for icar-archive@odin.ietf.org; Tue, 13 Jan 2004 11:40:32 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AgRah-00043O-2K for icar-web-archive@optimus.ietf.org; Tue, 13 Jan 2004 11:40:31 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA23129 for ; Tue, 13 Jan 2004 11:40:28 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1AgRag-0002aE-00 for icar-web-archive@ietf.org; Tue, 13 Jan 2004 11:40:30 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1AgRYs-0002WG-00 for icar-web-archive@ietf.org; Tue, 13 Jan 2004 11:38:39 -0500 Received: from [132.151.1.19] (helo=optimus.ietf.org) by ietf-mx with esmtp (Exim 4.12) id 1AgRXJ-0002S0-00 for icar-web-archive@ietf.org; Tue, 13 Jan 2004 11:37:01 -0500 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AgRXJ-0003ho-1x; Tue, 13 Jan 2004 11:37:01 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AgRWp-0003gK-Bx for icar@optimus.ietf.org; Tue, 13 Jan 2004 11:36:31 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA22817 for ; Tue, 13 Jan 2004 11:36:28 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1AgRWo-0002Nf-00 for icar@ietf.org; Tue, 13 Jan 2004 11:36:30 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1AgRV0-0002Fu-00 for icar@ietf.org; Tue, 13 Jan 2004 11:34:38 -0500 Received: from newdev.eecs.harvard.edu ([140.247.60.212] helo=newdev.harvard.edu) by ietf-mx with esmtp (Exim 4.12) id 1AgRTI-00024c-00 for icar@ietf.org; Tue, 13 Jan 2004 11:32:52 -0500 Received: by newdev.harvard.edu (Postfix, from userid 501) id 8E65CCBF5C; Tue, 13 Jan 2004 11:32:22 -0500 (EST) To: icar@ietf.org Subject: Re: [Icar] ICAR draft charter rev 2 Message-Id: <20040113163222.8E65CCBF5C@newdev.harvard.edu> Date: Tue, 13 Jan 2004 11:32:22 -0500 (EST) From: sob@harvard.edu (Scott Bradner) Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.0 required=5.0 tests=none autolearn=no version=2.60 I like this version of the draft charter - one nit, I'd change the first milestone to: FEB 2004: Publish Internet Drafts on improved community and structured reviews (or maybe march) Scott _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Tue Jan 13 13:09:07 2004 Received: from optimus.ietf.org ([132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA00502 for ; Tue, 13 Jan 2004 13:09:07 -0500 (EST) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AgSy0-0001Ik-GG for icar-archive@odin.ietf.org; Tue, 13 Jan 2004 13:08:40 -0500 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i0DI8evA005003 for icar-archive@odin.ietf.org; Tue, 13 Jan 2004 13:08:40 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AgSy0-0001Ic-Bi for icar-web-archive@optimus.ietf.org; Tue, 13 Jan 2004 13:08:40 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA00466 for ; Tue, 13 Jan 2004 13:08:36 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1AgSxy-00028Z-00 for icar-web-archive@ietf.org; Tue, 13 Jan 2004 13:08:38 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1AgSwL-0001yj-00 for icar-web-archive@ietf.org; Tue, 13 Jan 2004 13:06:58 -0500 Received: from [132.151.1.19] (helo=optimus.ietf.org) by ietf-mx with esmtp (Exim 4.12) id 1AgSuY-0001iG-00 for icar-web-archive@ietf.org; Tue, 13 Jan 2004 13:05:06 -0500 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AgSuV-0000sA-Nt; Tue, 13 Jan 2004 13:05:03 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AgOai-00058j-E1 for icar@optimus.ietf.org; Tue, 13 Jan 2004 08:28:20 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id IAA11573 for ; Tue, 13 Jan 2004 08:28:18 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1AgOah-00065e-00 for icar@ietf.org; Tue, 13 Jan 2004 08:28:19 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1AgOYx-00062X-00 for icar@ietf.org; Tue, 13 Jan 2004 08:26:32 -0500 Received: from tutakai.map-ne.com ([140.239.227.14] helo=Mail.MAP-NE.com) by ietf-mx with esmtp (Exim 4.12) id 1AgOYX-0005yx-00 for icar@ietf.org; Tue, 13 Jan 2004 08:26:05 -0500 Received: by Mail.MAP-NE.com (Postfix, from userid 105) id C01DA3F746; Tue, 13 Jan 2004 08:26:04 -0500 (EST) To: icar@ietf.org In-reply-to: <5.1.0.14.0.20040112175429.01a88040@localhost> (joel@stevecrocker.com) From: "Michael A. Patton" References: <5.1.0.14.0.20040112175429.01a88040@localhost> Message-Id: <20040113132604.C01DA3F746@Mail.MAP-NE.com> Date: Tue, 13 Jan 2004 08:26:04 -0500 (EST) Subject: [Icar] File naming to help trigger review Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.0 required=5.0 tests=none autolearn=no version=2.60 Another one where a comment in another discussion triggered one of my buttons... In other groups I have seen the use of the naming scheme draft-person-icar-xxx used to reflect this stage of the process. As a potential cross-fertilization reviewer, I find the current naming standard for drafts as a bit bothersome in this area. It's often very hard to distinguish between a set of proposals that are related to an existing WG, and the sometimes completely random independent offerings that use the same convention. Sometimes drafts that would benefit most from early review are when there are several proposals, which have not yet been decided between. But, with this convention, these look just like kook submissions and it's hard for a potential reviewer to pick up on this. In fact, when there are several options, it would be nice to see it because review from outside the WG might notice, for example, that one fits really well with some other work, while another would interfere with a protocol nearing WGLC and thus interfere with it later. I agree that the draft names may not be the best place to encode this, but right now they're all we have... I suggest that this is another area of "ID status" that this WG should think about... -MAP _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Tue Jan 13 13:09:07 2004 Received: from optimus.ietf.org ([132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA00515 for ; Tue, 13 Jan 2004 13:09:07 -0500 (EST) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AgSy1-0001JA-8J for icar-archive@odin.ietf.org; Tue, 13 Jan 2004 13:08:41 -0500 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i0DI8fa3005022 for icar-archive@odin.ietf.org; Tue, 13 Jan 2004 13:08:41 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AgSy1-0001Iv-4f for icar-web-archive@optimus.ietf.org; Tue, 13 Jan 2004 13:08:41 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA00470 for ; Tue, 13 Jan 2004 13:08:37 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1AgSxz-00028h-00 for icar-web-archive@ietf.org; Tue, 13 Jan 2004 13:08:39 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1AgSwM-0001ys-00 for icar-web-archive@ietf.org; Tue, 13 Jan 2004 13:06:59 -0500 Received: from [132.151.1.19] (helo=optimus.ietf.org) by ietf-mx with esmtp (Exim 4.12) id 1AgSuZ-0001iH-00 for icar-web-archive@ietf.org; Tue, 13 Jan 2004 13:05:07 -0500 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AgSuV-0000s2-He; Tue, 13 Jan 2004 13:05:03 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AgOQ0-0004ne-NC for icar@optimus.ietf.org; Tue, 13 Jan 2004 08:17:16 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id IAA10725 for ; Tue, 13 Jan 2004 08:17:14 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1AgOPz-0005Ct-00 for icar@ietf.org; Tue, 13 Jan 2004 08:17:15 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1AgOOB-0004zS-00 for icar@ietf.org; Tue, 13 Jan 2004 08:15:24 -0500 Received: from tutakai.map-ne.com ([140.239.227.14] helo=Mail.MAP-NE.com) by ietf-mx with esmtp (Exim 4.12) id 1AgOLh-0004f5-00 for icar@ietf.org; Tue, 13 Jan 2004 08:12:49 -0500 Received: by Mail.MAP-NE.com (Postfix, from userid 105) id 7AD0A3F746; Tue, 13 Jan 2004 08:12:11 -0500 (EST) To: icar@ietf.org In-reply-to: (message from Alex Rousskov on Mon, 12 Jan 2004 15:13:23 -0700 (MST)) From: "Michael A. Patton" References: <5.1.0.14.2.20040109203410.04552a28@ms101.mail1.com> <5.1.0.14.2.20040110075158.0385bbd0@ms101.mail1.com> Message-Id: <20040113131211.7AD0A3F746@Mail.MAP-NE.com> Date: Tue, 13 Jan 2004 08:12:11 -0500 (EST) Subject: [Icar] Reviewers: One single group or per-area groups? Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.0 required=5.0 tests=none autolearn=no version=2.60 Date: Mon, 12 Jan 2004 15:13:23 -0700 (MST) From: Alex Rousskov I see no essential difference between a single board with members that specialize in certain areas and multiple boards with members specializing in board-specific area. I saw this from Alex (and realized a couple others had the same concern, so this isn't really directed at just this remark, but using it as a launching point). Since I'm the one who stood up at the plenary to point out an essential difference, I guess maybe I should try and reinforce it. I, personally, do not consider myself tied to any area. There is no area in the IETF in which more than a few of the WGs are interesting to me and there is no area in the IETF in which I have not been an active participant in at least one WG at some point. There are a few areas that I spend more time with than others, but I think of myself more as a generalist than as any of these. As such, I haven't been tempted to join any of the area-specific oversight groups, but rather have just offerred reviews on an ad-hoc basis... I think one of the most important goals of this group is generalized cross-fertilization, and it's just the sort of generalists who wouldn't be part of an area-specific group, but might be part of a single group, who can, possibly, offer the most in such an effort. So, in that regard, there's probably a fair amount of difference between those two organizational structures. They will encourage different classes of people to participate. -MAP _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Tue Jan 13 13:39:04 2004 Received: from optimus.ietf.org ([132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA02203 for ; Tue, 13 Jan 2004 13:39:04 -0500 (EST) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AgTQy-0003XB-BL for icar-archive@odin.ietf.org; Tue, 13 Jan 2004 13:38:36 -0500 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i0DIcaan013579 for icar-archive@odin.ietf.org; Tue, 13 Jan 2004 13:38:36 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AgTQy-0003Ww-5K for icar-web-archive@optimus.ietf.org; Tue, 13 Jan 2004 13:38:36 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA02124 for ; Tue, 13 Jan 2004 13:38:33 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1AgTQw-0003qw-00 for icar-web-archive@ietf.org; Tue, 13 Jan 2004 13:38:34 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1AgTP4-0003ki-00 for icar-web-archive@ietf.org; Tue, 13 Jan 2004 13:36:39 -0500 Received: from [132.151.1.19] (helo=optimus.ietf.org) by ietf-mx with esmtp (Exim 4.12) id 1AgTNV-0003gZ-00 for icar-web-archive@ietf.org; Tue, 13 Jan 2004 13:35:01 -0500 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AgTNW-0002sQ-NF; Tue, 13 Jan 2004 13:35:02 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AgTMr-0002p7-3N for icar@optimus.ietf.org; Tue, 13 Jan 2004 13:34:21 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA01965 for ; Tue, 13 Jan 2004 13:34:18 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1AgTMo-0003d8-00 for icar@ietf.org; Tue, 13 Jan 2004 13:34:19 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1AgTKv-0003Xu-00 for icar@ietf.org; Tue, 13 Jan 2004 13:32:22 -0500 Received: from adsl-68-76-113-50.dsl.bcvloh.ameritech.net ([68.76.113.50] helo=guns.icir.org) by ietf-mx with esmtp (Exim 4.12) id 1AgTIx-0003Sg-00 for icar@ietf.org; Tue, 13 Jan 2004 13:30:19 -0500 Received: from guns.icir.org (localhost [127.0.0.1]) by guns.icir.org (Postfix) with ESMTP id 1B5BD77A703; Tue, 13 Jan 2004 13:29:48 -0500 (EST) To: Alex Zinin From: Mark Allman Reply-To: mallman@icir.org Cc: icar@ietf.org Subject: Re: [Icar] ICAR draft charter In-Reply-To: <1044648133.20040107122503@psg.com> Organization: ICSI Center for Internet Research (ICIR) Song-of-the-Day: Under the Bridge Date: Tue, 13 Jan 2004 13:29:48 -0500 Message-Id: <20040113182948.1B5BD77A703@guns.icir.org> Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.0 required=5.0 tests=none autolearn=no version=2.60 the latest version of the draft charter looks quite reasonable to me thanks, allman -- Mark Allman -- ICIR -- http://www.icir.org/mallman/ _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Tue Jan 13 13:43:00 2004 Received: from optimus.ietf.org ([132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA02507 for ; Tue, 13 Jan 2004 13:43:00 -0500 (EST) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AgTUl-00045a-Rs for icar-archive@odin.ietf.org; Tue, 13 Jan 2004 13:42:32 -0500 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i0DIgVEg015717 for icar-archive@odin.ietf.org; Tue, 13 Jan 2004 13:42:31 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AgTUl-00045Q-Nm for icar-web-archive@optimus.ietf.org; Tue, 13 Jan 2004 13:42:31 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA02447 for ; Tue, 13 Jan 2004 13:42:28 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1AgTUf-00049W-00 for icar-web-archive@ietf.org; Tue, 13 Jan 2004 13:42:25 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1AgTSt-00042m-00 for icar-web-archive@ietf.org; Tue, 13 Jan 2004 13:40:35 -0500 Received: from [132.151.1.19] (helo=optimus.ietf.org) by ietf-mx with esmtp (Exim 4.12) id 1AgTRL-0003ud-00 for icar-web-archive@ietf.org; Tue, 13 Jan 2004 13:39:00 -0500 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AgTRN-0003Yk-Bw; Tue, 13 Jan 2004 13:39:01 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AgTQv-0003Wr-S4 for icar@optimus.ietf.org; Tue, 13 Jan 2004 13:38:33 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA02121 for ; Tue, 13 Jan 2004 13:38:31 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1AgTQt-0003qb-00 for icar@ietf.org; Tue, 13 Jan 2004 13:38:31 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1AgTP3-0003kZ-00 for icar@ietf.org; Tue, 13 Jan 2004 13:36:38 -0500 Received: from joy.songbird.com ([208.184.79.7]) by ietf-mx with esmtp (Exim 4.12) id 1AgTNQ-0003dN-00 for icar@ietf.org; Tue, 13 Jan 2004 13:34:56 -0500 Received: from bbprime (jay.songbird.com [208.184.79.253]) by joy.songbird.com (8.11.6/8.11.6) with SMTP id i0DIfqc21325; Tue, 13 Jan 2004 10:41:52 -0800 From: Dave Crocker To: , X-Mailer: PocoMail 3.03 (1740) - EVALUATION VERSION X-URL: http://www.pocomail.com/ Date: Tue, 13 Jan 2004 10:34:05 -0800 Message-ID: <200411310345.801759@bbprime> In-Reply-To: <20040113131211.7AD0A3F746@Mail.MAP-NE.com> Subject: Re: [Icar] Reviewers: One single group or per-area groups? Mime-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Transfer-Encoding: 7bit Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.1 required=5.0 tests=AWL autolearn=no version=2.60 Content-Transfer-Encoding: 7bit Content-Transfer-Encoding: 7bit Michael, > I, personally, do not consider myself tied to any area. There is no > area in the IETF in which more than a few of the WGs are interesting > to me and there is no area in the IETF in which I have not been an > active participant in at least one WG at some point. I like this perspective a lot. I've been concerned that reviewers be independent of ADs, but had not thought that an even deeper point is that senior IETF contributors nearly always are cross- (or multi-) area. > I think one of the most important goals of this group is generalized > cross-fertilization, and it's just the sort of generalists who > wouldn't be part of an area-specific group, but might be part of a > single group, who can, possibly, offer the most in such an effort. > So, in that regard, there's probably a fair amount of difference > between those two organizational structures. They will encourage > different classes of people to participate. Nicely said, and many thanks for saying it. d/ -- Dave Crocker Brandenburg InternetWorking _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Tue Jan 13 14:15:46 2004 Received: from optimus.ietf.org ([132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA05211 for ; Tue, 13 Jan 2004 14:15:46 -0500 (EST) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AgU0U-0005Uw-90 for icar-archive@odin.ietf.org; Tue, 13 Jan 2004 14:15:18 -0500 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i0DJFI8p021128 for icar-archive@odin.ietf.org; Tue, 13 Jan 2004 14:15:18 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AgU0U-0005Uh-4m for icar-web-archive@optimus.ietf.org; Tue, 13 Jan 2004 14:15:18 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA05041 for ; Tue, 13 Jan 2004 14:15:15 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1AgU0R-0006pF-00 for icar-web-archive@ietf.org; Tue, 13 Jan 2004 14:15:15 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1AgTyV-0006Z5-00 for icar-web-archive@ietf.org; Tue, 13 Jan 2004 14:13:16 -0500 Received: from [65.246.255.50] (helo=mx2.foretec.com) by ietf-mx with esmtp (Exim 4.12) id 1AgTwW-0006Nw-02 for icar-web-archive@ietf.org; Tue, 13 Jan 2004 14:11:12 -0500 Received: from optimus22.ietf.org ([132.151.6.22] helo=optimus.ietf.org) by mx2.foretec.com with esmtp (Exim 4.24) id 1AgTkm-0006Ba-97 for icar-web-archive@ietf.org; Tue, 13 Jan 2004 13:59:04 -0500 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AgTki-0004ez-U3; Tue, 13 Jan 2004 13:59:00 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AgTki-0004ek-AS for icar@optimus.ietf.org; Tue, 13 Jan 2004 13:59:00 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA03652 for ; Tue, 13 Jan 2004 13:58:57 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1AgTkf-0005Sr-00 for icar@ietf.org; Tue, 13 Jan 2004 13:58:58 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1AgTiZ-0005El-00 for icar@ietf.org; Tue, 13 Jan 2004 13:56:47 -0500 Received: from measurement-factory.com ([206.168.0.5]) by ietf-mx with esmtp (Exim 4.12) id 1AgTh9-00057n-00 for icar@ietf.org; Tue, 13 Jan 2004 13:55:19 -0500 Received: from measurement-factory.com (localhost [127.0.0.1]) by measurement-factory.com (8.12.9/8.12.9) with ESMTP id i0DItDk3074433; Tue, 13 Jan 2004 11:55:13 -0700 (MST) (envelope-from rousskov@measurement-factory.com) Received: (from rousskov@localhost) by measurement-factory.com (8.12.9/8.12.9/Submit) id i0DItDRL074432; Tue, 13 Jan 2004 11:55:13 -0700 (MST) (envelope-from rousskov) Date: Tue, 13 Jan 2004 11:55:13 -0700 (MST) From: Alex Rousskov To: "Michael A. Patton" cc: icar@ietf.org Subject: Re: [Icar] Reviewers: One single group or per-area groups? In-Reply-To: <20040113131211.7AD0A3F746@Mail.MAP-NE.com> Message-ID: References: <5.1.0.14.2.20040109203410.04552a28@ms101.mail1.com> <5.1.0.14.2.20040110075158.0385bbd0@ms101.mail1.com> <20040113131211.7AD0A3F746@Mail.MAP-NE.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.0 required=5.0 tests=AWL autolearn=no version=2.60 Michael, I agree with your observation. My no-difference comment was referring specifically to Margaret's analysis. The per-area boards are better if we want to focus on internal per-area review. A global pool is better if we want to emphasize cross-area and support no-specific-area review. We will indeed attract "different classes of people", especially if we also consider open versus closed pool or AD-ruled versus self-managed board approaches. Alex. On Tue, 13 Jan 2004, Michael A. Patton wrote: > From: Alex Rousskov > > I see no essential difference between a single board with members that > specialize in certain areas and multiple boards with members > specializing in board-specific area. > > I think one of the most important goals of this group is generalized > cross-fertilization, and it's just the sort of generalists who > wouldn't be part of an area-specific group, but might be part of a > single group, who can, possibly, offer the most in such an effort. > So, in that regard, there's probably a fair amount of difference > between those two organizational structures. They will encourage > different classes of people to participate. _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Tue Jan 13 22:40:45 2004 Received: from optimus.ietf.org ([132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id WAA00704 for ; Tue, 13 Jan 2004 22:40:45 -0500 (EST) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AgbtC-0004HE-My for icar-archive@odin.ietf.org; Tue, 13 Jan 2004 22:40:18 -0500 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i0E3eIHR016434 for icar-archive@odin.ietf.org; Tue, 13 Jan 2004 22:40:18 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AgbtC-0004Gz-Ho for icar-web-archive@optimus.ietf.org; Tue, 13 Jan 2004 22:40:18 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id WAA00628 for ; Tue, 13 Jan 2004 22:40:14 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1Agbt8-0007Xd-00 for icar-web-archive@ietf.org; Tue, 13 Jan 2004 22:40:14 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1AgboP-0007Os-00 for icar-web-archive@ietf.org; Tue, 13 Jan 2004 22:35:22 -0500 Received: from [132.151.1.19] (helo=optimus.ietf.org) by ietf-mx with esmtp (Exim 4.12) id 1AgbkV-0007Fk-00 for icar-web-archive@ietf.org; Tue, 13 Jan 2004 22:31:19 -0500 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AgbkD-0003nZ-T0; Tue, 13 Jan 2004 22:31:01 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1Agbjy-0003me-FT for icar@optimus.ietf.org; Tue, 13 Jan 2004 22:30:46 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id WAA00410 for ; Tue, 13 Jan 2004 22:30:37 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1Agbjl-0007EB-00 for icar@ietf.org; Tue, 13 Jan 2004 22:30:33 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1AgbdH-00072z-00 for icar@ietf.org; Tue, 13 Jan 2004 22:23:51 -0500 Received: from psg.com ([147.28.0.62] ident=mailnull) by ietf-mx with esmtp (Exim 4.12) id 1AgbZP-0006ss-00 for icar@ietf.org; Tue, 13 Jan 2004 22:19:51 -0500 Received: from [147.28.0.62] (helo=127.0.0.1) by psg.com with esmtp (Exim 4.24; FreeBSD) id 1AgbZ1-000OsK-54 for icar@ietf.org; Wed, 14 Jan 2004 03:19:27 +0000 Date: Tue, 13 Jan 2004 19:19:13 -0800 From: Alex Zinin X-Mailer: The Bat! (v1.62i) Personal Reply-To: Alex Zinin X-Priority: 3 (Normal) Message-ID: <110547685570.20040113191913@psg.com> To: icar@ietf.org Subject: Re: [Icar] ICAR draft charter rev 2 In-Reply-To: <20040113163222.8E65CCBF5C@newdev.harvard.edu> References: <20040113163222.8E65CCBF5C@newdev.harvard.edu> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Content-Transfer-Encoding: 7bit Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.4 required=5.0 tests=AWL,RCVD_NUMERIC_HELO autolearn=no version=2.60 Content-Transfer-Encoding: 7bit Content-Transfer-Encoding: 7bit Folks- Seems that rev 2 is reasonably good. I will post the final version incorporating several comments to the list tomorrow morning for the final sanity check before forwarding it to Harald. Alex _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Wed Jan 14 13:20:20 2004 Received: from optimus.ietf.org ([132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA01476 for ; Wed, 14 Jan 2004 13:20:20 -0500 (EST) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AgpcQ-0006yK-8V for icar-archive@odin.ietf.org; Wed, 14 Jan 2004 13:19:54 -0500 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i0EIJsMs026796 for icar-archive@odin.ietf.org; Wed, 14 Jan 2004 13:19:54 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AgpcQ-0006y7-4c for icar-web-archive@optimus.ietf.org; Wed, 14 Jan 2004 13:19:54 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA01460 for ; Wed, 14 Jan 2004 13:19:50 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1AgpcO-0000vT-00 for icar-web-archive@ietf.org; Wed, 14 Jan 2004 13:19:52 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1AgpbS-0000tX-00 for icar-web-archive@ietf.org; Wed, 14 Jan 2004 13:18:55 -0500 Received: from [132.151.1.19] (helo=optimus.ietf.org) by ietf-mx with esmtp (Exim 4.12) id 1Agpab-0000s0-00 for icar-web-archive@ietf.org; Wed, 14 Jan 2004 13:18:01 -0500 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1Agpab-0006rV-3K; Wed, 14 Jan 2004 13:18:01 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AgpZa-0006qU-TG for icar@optimus.ietf.org; Wed, 14 Jan 2004 13:17:01 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA01326 for ; Wed, 14 Jan 2004 13:16:55 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1AgpZY-0000pQ-00 for icar@ietf.org; Wed, 14 Jan 2004 13:16:57 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1AgpYc-0000nc-00 for icar@ietf.org; Wed, 14 Jan 2004 13:15:59 -0500 Received: from eikenes.alvestrand.no ([158.38.152.233]) by ietf-mx with esmtp (Exim 4.12) id 1AgpXk-0000ir-00 for icar@ietf.org; Wed, 14 Jan 2004 13:15:04 -0500 Received: from halvestr-w2k1 (localhost.localdomain [127.0.0.1]) by eikenes.alvestrand.no (Postfix) with ESMTP id E99A761C12; Wed, 14 Jan 2004 19:14:31 +0100 (CET) Date: Tue, 13 Jan 2004 22:22:34 -0800 From: Harald Tveit Alvestrand To: "Michael A. Patton" , icar@ietf.org Subject: Re: [Icar] Reviewers: One single group or per-area groups? Message-ID: <144370974.1074032554@localhost> In-Reply-To: <20040113131211.7AD0A3F746@Mail.MAP-NE.com> References: <5.1.0.14.2.20040109203410.04552a28@ms101.mail1.com> <5.1.0.14.2.20040110075158.0385bbd0@ms101.mail1.com> <20040113131211.7AD0A3F746@Mail.MAP-NE.com> X-Mailer: Mulberry/3.1.0 (Win32) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline Content-Transfer-Encoding: 7bit Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.3 required=5.0 tests=AWL,DATE_IN_PAST_06_12 autolearn=no version=2.60 Content-Transfer-Encoding: 7bit Content-Transfer-Encoding: 7bit Michael, let me just drop by to offer a contrarian view..... in the current IESG review structure, it is important for me to know that (for instance) SOMEONE will have looked at the security issues, and that SOMEONE will have tried to compile the MIB if there is one. So the idea behind area-specific reviews is kind of generalizing this concept - that after looking at the list of reviewers and the areas they are reviewing for, we're pretty certain that the important angles have at least been glanced at. This, of course, presupposes that that some kind of (implicit or explicit) "question list" exists for each area review, and that the sum of all areas' questions comes reasonably close to "covering the map" - this is an untested hypothesis for the reviewer case, but hasn't been too far from the truth for the IESG review case..... Harald _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Wed Jan 14 13:59:28 2004 Received: from optimus.ietf.org ([132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA03354 for ; Wed, 14 Jan 2004 13:59:27 -0500 (EST) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AgqEG-0000eb-24 for icar-archive@odin.ietf.org; Wed, 14 Jan 2004 13:59:00 -0500 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i0EIx0Ew002512 for icar-archive@odin.ietf.org; Wed, 14 Jan 2004 13:59:00 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AgqEF-0000eR-UE for icar-web-archive@optimus.ietf.org; Wed, 14 Jan 2004 13:58:59 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA03346 for ; Wed, 14 Jan 2004 13:58:57 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1AgqED-0002kp-00 for icar-web-archive@ietf.org; Wed, 14 Jan 2004 13:58:57 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1AgqDK-0002jL-00 for icar-web-archive@ietf.org; Wed, 14 Jan 2004 13:58:02 -0500 Received: from [132.151.1.19] (helo=optimus.ietf.org) by ietf-mx with esmtp (Exim 4.12) id 1AgqCS-0002h0-00 for icar-web-archive@ietf.org; Wed, 14 Jan 2004 13:57:08 -0500 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AgqCJ-0000aZ-Lx; Wed, 14 Jan 2004 13:56:59 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AgqCG-0000Zd-PO for icar@optimus.ietf.org; Wed, 14 Jan 2004 13:56:56 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA03285 for ; Wed, 14 Jan 2004 13:56:54 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1AgqCE-0002g9-00 for icar@ietf.org; Wed, 14 Jan 2004 13:56:54 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1AgqBc-0002eP-00 for icar@ietf.org; Wed, 14 Jan 2004 13:56:16 -0500 Received: from measurement-factory.com ([206.168.0.5]) by ietf-mx with esmtp (Exim 4.12) id 1AgqAw-0002Yp-00 for icar@ietf.org; Wed, 14 Jan 2004 13:55:34 -0500 Received: from measurement-factory.com (localhost [127.0.0.1]) by measurement-factory.com (8.12.9/8.12.9) with ESMTP id i0EItWk3030665; Wed, 14 Jan 2004 11:55:32 -0700 (MST) (envelope-from rousskov@measurement-factory.com) Received: (from rousskov@localhost) by measurement-factory.com (8.12.9/8.12.9/Submit) id i0EItW3h030664; Wed, 14 Jan 2004 11:55:32 -0700 (MST) (envelope-from rousskov) Date: Wed, 14 Jan 2004 11:55:32 -0700 (MST) From: Alex Rousskov To: Harald Tveit Alvestrand cc: "Michael A. Patton" , icar@ietf.org Subject: Re: [Icar] Reviewers: One single group or per-area groups? In-Reply-To: <144370974.1074032554@localhost> Message-ID: References: <5.1.0.14.2.20040109203410.04552a28@ms101.mail1.com> <5.1.0.14.2.20040110075158.0385bbd0@ms101.mail1.com> <20040113131211.7AD0A3F746@Mail.MAP-NE.com> <144370974.1074032554@localhost> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.0 required=5.0 tests=AWL autolearn=no version=2.60 On Tue, 13 Jan 2004, Harald Tveit Alvestrand wrote: > let me just drop by to offer a contrarian view..... in the current > IESG review structure, it is important for me to know that (for > instance) SOMEONE will have looked at the security issues, and that > SOMEONE will have tried to compile the MIB if there is one. Just for the record: The above is easy to ensure with both single pool of reviewers and a collection of area-specific boards. It is not a distinction point in the "One single group or per-area groups?" debate. In fact, a single pool architecture makes it somewhat easier to support expertise areas that are not accurately covered by any existing IETF area. With a single pool, one can "tag" individual reviewers, and each reviewer may have several tags. The same can be done in isolated boards, but tagging in a single pool is more _natural_ than cross-area tagging in area-specific boards, IMO. Alex. _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Wed Jan 14 14:11:20 2004 Received: from optimus.ietf.org ([132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA03897 for ; Wed, 14 Jan 2004 14:11:20 -0500 (EST) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AgqPk-0001O4-BD for icar-archive@odin.ietf.org; Wed, 14 Jan 2004 14:10:52 -0500 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i0EJAqMi005326 for icar-archive@odin.ietf.org; Wed, 14 Jan 2004 14:10:52 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AgqPk-0001Np-6B for icar-web-archive@optimus.ietf.org; Wed, 14 Jan 2004 14:10:52 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA03875 for ; Wed, 14 Jan 2004 14:10:49 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1AgqPh-0003H0-00 for icar-web-archive@ietf.org; Wed, 14 Jan 2004 14:10:49 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1AgqOo-0003ET-00 for icar-web-archive@ietf.org; Wed, 14 Jan 2004 14:09:55 -0500 Received: from [132.151.1.19] (helo=optimus.ietf.org) by ietf-mx with esmtp (Exim 4.12) id 1AgqNu-00039d-00 for icar-web-archive@ietf.org; Wed, 14 Jan 2004 14:08:58 -0500 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AgqNw-0001Io-Ka; Wed, 14 Jan 2004 14:09:00 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AgqN0-0001Fq-Gr for icar@optimus.ietf.org; Wed, 14 Jan 2004 14:08:02 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA03637 for ; Wed, 14 Jan 2004 14:07:59 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1AgqMx-000344-00 for icar@ietf.org; Wed, 14 Jan 2004 14:08:00 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1AgqM1-00030O-00 for icar@ietf.org; Wed, 14 Jan 2004 14:07:01 -0500 Received: from psg.com ([147.28.0.62] ident=mailnull) by ietf-mx with esmtp (Exim 4.12) id 1AgqLT-0002y0-00 for icar@ietf.org; Wed, 14 Jan 2004 14:06:28 -0500 Received: from [147.28.0.62] (helo=127.0.0.1) by psg.com with esmtp (Exim 4.24; FreeBSD) id 1AgqLT-000Oq9-8X for icar@ietf.org; Wed, 14 Jan 2004 19:06:27 +0000 Date: Wed, 14 Jan 2004 11:04:13 -0800 From: Alex Zinin X-Mailer: The Bat! (v1.62i) Personal Reply-To: Alex Zinin X-Priority: 3 (Normal) Message-ID: <525543380.20040114110413@psg.com> To: icar@ietf.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Content-Transfer-Encoding: 7bit Subject: [Icar] ICAR draft charter rev 3 Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.4 required=5.0 tests=AWL,RCVD_NUMERIC_HELO autolearn=no version=2.60 Content-Transfer-Encoding: 7bit Content-Transfer-Encoding: 7bit Folks, Rev 3, the final version before it goes to Harald, below. html-diff against rev2 is available at http://www.psg.com/~zinin/ietf/icar.charter.v2v3.diff.html Alex R., given a good level of support on the list for the current text of the charter, I decided not to incorporate the substantial changes you suggested, except for a minor clarification of the "earlier" part that Dave also brought up--changing the text substantially would introduce the risk of having to go through the complete rewrite & review cycle again. Thank all for your comments! Alex WG name: improved cross-area review (icar) Chairs: General Area Director(s): Harald Alvestrand General Area Advisor: Harald Alvestrand Mailing list: icar@ietf.org Subscription: icar-request@ietf.org Archives : https://www1.ietf.org/mail-archive/working-groups/icar/current/maillist.html WG Description: The WG will work out mechanisms for improved cross-functional review within the IETF, addressing the issues identified by the PROBLEM WG. This includes a better community review, as well as more structured pre-IESG review that may be used to improve scalability of the IESG review function. Definitions: o Cross-functional review: document review covering different aspects of Internet technology, including those represented by WGs within different IETF areas. o Community review: document review performed by individual IETF participants and not caused by their responsibilities within the IETF management structure. o Structured review: more formal and role-based document review performed by individuals assuming certain responsibilities (such as by WG chairs, directorate and IESG members.) It is an explicit goal of the WG to come up with mechanisms encouraging early review of the documents while ideas are still in the formation stage. An early review is best for catching architectural problems while they're still relatively easy to solve. In particular, many cross-functional interactions can be spotted and dealt with, thus avoiding many "late surprises". A final review (currently done by the IESG) can catch remaining cross-functional interactions, as well as deal with overall quality issues. The WG will cooperate with others in starting and evaluating experiments with early community and structured reviews. The evaluation of such experiments may be published as Informational RFCs if the group so desires. The WG will also coordinate with other WGs involved in the IETF reform process on proposed changes to the IETF Working Group operations and Standards process if those are considered necessary. WG milestones: FEB 2004: Publish Internet Drafts on improved community and structured reviews SEP 2004: Submit draft on improved community review to the IESG for publication as BCP SEP 2004: Submit draft on improved structured community review to the IESG for publication as BCP JAN 2005: Evaluate WG progress and potential; close or recharter _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Wed Jan 14 21:27:40 2004 Received: from optimus.ietf.org ([132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id VAA26846 for ; Wed, 14 Jan 2004 21:27:40 -0500 (EST) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AgxE1-0008JJ-8k for icar-archive@odin.ietf.org; Wed, 14 Jan 2004 21:27:13 -0500 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i0F2RD7E031941 for icar-archive@odin.ietf.org; Wed, 14 Jan 2004 21:27:13 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AgxE1-0008J6-53 for icar-web-archive@optimus.ietf.org; Wed, 14 Jan 2004 21:27:13 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id VAA26805 for ; Wed, 14 Jan 2004 21:27:10 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1AgxDy-0007hp-00 for icar-web-archive@ietf.org; Wed, 14 Jan 2004 21:27:10 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1AgxD5-0007g2-00 for icar-web-archive@ietf.org; Wed, 14 Jan 2004 21:26:16 -0500 Received: from [132.151.1.19] (helo=optimus.ietf.org) by ietf-mx with esmtp (Exim 4.12) id 1AgxCo-0007dZ-00 for icar-web-archive@ietf.org; Wed, 14 Jan 2004 21:25:58 -0500 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AgxCq-0007vg-4G; Wed, 14 Jan 2004 21:26:00 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AgxC1-0007in-JN for icar@optimus.ietf.org; Wed, 14 Jan 2004 21:25:09 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id VAA26736 for ; Wed, 14 Jan 2004 21:25:06 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1AgxBy-0007cP-00 for icar@ietf.org; Wed, 14 Jan 2004 21:25:07 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1AgxB3-0007am-00 for icar@ietf.org; Wed, 14 Jan 2004 21:24:10 -0500 Received: from rwcrmhc13.comcast.net ([204.127.198.39]) by ietf-mx with esmtp (Exim 4.12) id 1AgxAC-0007XV-00 for icar@ietf.org; Wed, 14 Jan 2004 21:23:16 -0500 Received: from dfnjgl21 (c-24-1-97-129.client.comcast.net[24.1.97.129]) by comcast.net (rwcrmhc13) with SMTP id <2004011502224601500e2vr9e> (Authid: sdawkins@comcast.net); Thu, 15 Jan 2004 02:22:46 +0000 Message-ID: <003801c3db0e$7789a3c0$0400a8c0@DFNJGL21> Reply-To: "Spencer Dawkins" From: "Spencer Dawkins" To: References: <196108581021.20040108171716@psg.com> Subject: Re: [Icar] ICAR draft charter rev 2 Date: Wed, 14 Jan 2004 20:22:46 -0600 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2800.1158 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1165 Content-Transfer-Encoding: 7bit Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.0 required=5.0 tests=none autolearn=no version=2.60 Content-Transfer-Encoding: 7bit Content-Transfer-Encoding: 7bit Yeah, it's getting there - short note below. > > It is an explicit goal of the WG to come up with mechanisms > encouraging earlier review of the documents. An early review is I would like an explicit discussion of timing scope. Maybe words like "It is an explicit goal of the WG to come up with mechanisms encouraging review of the documents at every stage of development - from drafts under consideration for WG adoption (do I have this end right?) to drafts being forwarded for publication"? > best for catching architectural problems while they're still > relatively easy to solve. In particular, many cross-functional > interactions can be spotted and dealt with, thus avoiding many > "late surprises". A final review (currently done by the IESG) can > catch remaining cross-functional interactions, as well as deal > with overall quality issues. _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Thu Jan 15 15:09:39 2004 Received: from optimus.ietf.org ([132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA25202 for ; Thu, 15 Jan 2004 15:09:39 -0500 (EST) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AhDnk-0007G0-3y for icar-archive@odin.ietf.org; Thu, 15 Jan 2004 15:09:12 -0500 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i0FK9C6N027890 for icar-archive@odin.ietf.org; Thu, 15 Jan 2004 15:09:12 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AhDnj-0007Fl-V4 for icar-web-archive@optimus.ietf.org; Thu, 15 Jan 2004 15:09:12 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA25105 for ; Thu, 15 Jan 2004 15:09:08 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1AhDng-00076v-00 for icar-web-archive@ietf.org; Thu, 15 Jan 2004 15:09:08 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1AhDmb-0006yt-00 for icar-web-archive@ietf.org; Thu, 15 Jan 2004 15:08:02 -0500 Received: from [132.151.1.19] (helo=optimus.ietf.org) by ietf-mx with esmtp (Exim 4.12) id 1AhDlf-0006tt-00 for icar-web-archive@ietf.org; Thu, 15 Jan 2004 15:07:03 -0500 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AhDlg-0006tj-ST; Thu, 15 Jan 2004 15:07:04 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AhDlc-0006qv-0c for icar@optimus.ietf.org; Thu, 15 Jan 2004 15:07:00 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA24823 for ; Thu, 15 Jan 2004 15:06:56 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1AhDlY-0006sY-00 for icar@ietf.org; Thu, 15 Jan 2004 15:06:57 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1AhDks-0006oI-00 for icar@ietf.org; Thu, 15 Jan 2004 15:06:15 -0500 Received: from psg.com ([147.28.0.62] ident=mailnull) by ietf-mx with esmtp (Exim 4.12) id 1AhDk9-0006i3-00 for icar@ietf.org; Thu, 15 Jan 2004 15:05:29 -0500 Received: from [147.28.0.62] (helo=127.0.0.1) by psg.com with esmtp (Exim 4.24; FreeBSD) id 1AhDk9-000Nau-4H; Thu, 15 Jan 2004 20:05:29 +0000 Date: Thu, 15 Jan 2004 12:05:20 -0800 From: Alex Zinin X-Mailer: The Bat! (v1.62i) Personal Reply-To: Alex Zinin X-Priority: 3 (Normal) Message-ID: <6595610370.20040115120520@psg.com> To: "Spencer Dawkins" CC: icar@ietf.org Subject: Re: [Icar] ICAR draft charter rev 2 In-Reply-To: <003801c3db0e$7789a3c0$0400a8c0@DFNJGL21> References: <196108581021.20040108171716@psg.com> <003801c3db0e$7789a3c0$0400a8c0@DFNJGL21> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Content-Transfer-Encoding: 7bit Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.4 required=5.0 tests=AWL,RCVD_NUMERIC_HELO autolearn=no version=2.60 Content-Transfer-Encoding: 7bit Content-Transfer-Encoding: 7bit Spencer, Rev 3 I posted last night expands a bit more on this: It is an explicit goal of the WG to come up with mechanisms encouraging early review of the documents while ideas are still in the formation stage... I know it's not exactly what you suggest, but it does answer the question of "how early" that was asked several times. -- Alex http://www.psg.com/~zinin/ Wednesday, January 14, 2004, 6:22:46 PM, Spencer Dawkins wrote: > Yeah, it's getting there - short note below. >> >> It is an explicit goal of the WG to come up with mechanisms >> encouraging earlier review of the documents. An early review is > I would like an explicit discussion of timing scope. Maybe words like > "It is an > explicit goal of the WG to come up with mechanisms encouraging review > of the documents at every stage of development - from drafts under > consideration for WG adoption (do I have this end right?) to drafts > being forwarded for publication"? >> best for catching architectural problems while they're still >> relatively easy to solve. In particular, many cross-functional >> interactions can be spotted and dealt with, thus avoiding many >> "late surprises". A final review (currently done by the IESG) > can >> catch remaining cross-functional interactions, as well as deal >> with overall quality issues. > _______________________________________________ > Icar mailing list > Icar@ietf.org > https://www1.ietf.org/mailman/listinfo/icar _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Mon Jan 19 15:52:40 2004 Received: from optimus.ietf.org ([132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA08318 for ; Mon, 19 Jan 2004 15:52:40 -0500 (EST) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AigNW-00016G-Ri for icar-archive@odin.ietf.org; Mon, 19 Jan 2004 15:52:10 -0500 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i0JKqA3n004227 for icar-archive@odin.ietf.org; Mon, 19 Jan 2004 15:52:10 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AigNW-000166-NX for icar-web-archive@optimus.ietf.org; Mon, 19 Jan 2004 15:52:10 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA08312 for ; Mon, 19 Jan 2004 15:52:08 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1AigNQ-0004Zv-00 for icar-web-archive@ietf.org; Mon, 19 Jan 2004 15:52:04 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1AigM8-0004XL-00 for icar-web-archive@ietf.org; Mon, 19 Jan 2004 15:50:45 -0500 Received: from [132.151.1.19] (helo=optimus.ietf.org) by ietf-mx with esmtp (Exim 4.12) id 1AigLS-0004V3-00 for icar-web-archive@ietf.org; Mon, 19 Jan 2004 15:50:02 -0500 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AigLS-00012u-Km; Mon, 19 Jan 2004 15:50:02 -0500 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AigLC-00012E-Bw for icar@optimus.ietf.org; Mon, 19 Jan 2004 15:49:46 -0500 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA08278 for ; Mon, 19 Jan 2004 15:49:43 -0500 (EST) Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1AigLA-0004Tu-00 for icar@ietf.org; Mon, 19 Jan 2004 15:49:45 -0500 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1AigKM-0004RM-00 for icar@ietf.org; Mon, 19 Jan 2004 15:48:55 -0500 Received: from smtp.exodus.net ([66.35.230.236]) by ietf-mx with esmtp (Exim 4.12) id 1AigJO-0004Km-00 for icar@ietf.org; Mon, 19 Jan 2004 15:47:54 -0500 Received: from ms101.mail1.com (ms101.mail1.com [209.1.5.174]) by smtp.exodus.net (8.12.8/8.12.8) with ESMTP id i0JMQtw3029020 for ; Mon, 19 Jan 2004 14:26:57 -0800 Received: from ala-mrwtemp.thingmagic.com (unverified [24.61.30.237]) by accounting.espmail.com (Rockliffe SMTPRA 5.2.5) with ESMTP id ; Mon, 19 Jan 2004 12:47:20 -0800 Message-Id: <5.1.0.14.2.20040119153928.03719df0@ms101.mail1.com> X-Sender: margaret@thingmagic.com@ms101.mail1.com X-Mailer: QUALCOMM Windows Eudora Version 5.1 Date: Mon, 19 Jan 2004 15:45:43 -0500 To: Alex Zinin From: Margaret Wasserman Subject: Re: [Icar] ICAR draft charter rev 3 Cc: icar@ietf.org In-Reply-To: <525543380.20040114110413@psg.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.0 required=5.0 tests=AWL autolearn=no version=2.60 Hi Alex, Just a couple of minor questions/comments: > The WG will cooperate with others in starting and evaluating > experiments with early community and structured reviews. Are you intending that the word "early" is applied to both community and structured reviews, or only to community reviews in this sentence? In other words, do you mean that this group will work on both structured and unstructured early review, or is it also intended to work on changes to the final/later review process? > FEB 2004: Publish Internet Drafts on improved community and > structured reviews I'd rather see separate sets of drafts (I assume that these are two separate sets of drafts?) have separate entries. Also, is this intended to be a deadline for the community to submit ideas that will be considered in the final proposals? Or is this the date for publication of the first WG I-Ds? > SEP 2004: Submit draft on improved community review to > the IESG for publication as BCP > SEP 2004: Submit draft on improved structured community review to > the IESG for publication as BCP Based on the definitions that you included earlier in the charter, I think that "structured community review" is an oxymoron. Margaret _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar