From exim@www1.ietf.org Sun May 2 04:31:20 2004 Received: from optimus.ietf.org (iesg.org [132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id EAA03938 for ; Sun, 2 May 2004 04:31:20 -0400 (EDT) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BKCL3-00046K-Bt for icar-archive@odin.ietf.org; Sun, 02 May 2004 04:28:41 -0400 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i428Sf3H015765 for icar-archive@odin.ietf.org; Sun, 2 May 2004 04:28:41 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BKCIH-000350-Pa for icar-web-archive@optimus.ietf.org; Sun, 02 May 2004 04:25:49 -0400 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id EAA03857 for ; Sun, 2 May 2004 04:25:47 -0400 (EDT) Received: from ietf-mx.ietf.org ([132.151.6.1] helo=ietf-mx) by ietf-mx with esmtp (Exim 4.32) id 1BKCIE-0002Tp-VX for icar-web-archive@ietf.org; Sun, 02 May 2004 04:25:46 -0400 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1BKCHN-0002Fu-00 for icar-web-archive@ietf.org; Sun, 02 May 2004 04:24:54 -0400 Received: from optimus.ietf.org ([132.151.1.19]) by ietf-mx with esmtp (Exim 4.12) id 1BKCGR-00021Z-00 for icar-web-archive@ietf.org; Sun, 02 May 2004 04:23:55 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BKCEd-0008PC-EE; Sun, 02 May 2004 04:22:03 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BKCCM-0007TQ-Q8 for icar@optimus.ietf.org; Sun, 02 May 2004 04:19:42 -0400 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id EAA03684 for ; Sun, 2 May 2004 04:19:40 -0400 (EDT) Received: from ietf-mx.ietf.org ([132.151.6.1] helo=ietf-mx) by ietf-mx with esmtp (Exim 4.32) id 1BKCCJ-000180-Ve for icar@ietf.org; Sun, 02 May 2004 04:19:40 -0400 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1BKCBN-0000v3-00 for icar@ietf.org; Sun, 02 May 2004 04:18:42 -0400 Received: from mtagate5.de.ibm.com ([195.212.29.154]) by ietf-mx with esmtp (Exim 4.12) id 1BKCB7-0000i7-00 for icar@ietf.org; Sun, 02 May 2004 04:18:25 -0400 Received: from d12nrmr1507.megacenter.de.ibm.com (d12nrmr1507.megacenter.de.ibm.com [9.149.167.1]) by mtagate5.de.ibm.com (8.12.10/8.12.10) with ESMTP id i428HmYO135090; Sun, 2 May 2004 08:17:48 GMT Received: from sihl.zurich.ibm.com (sihl.zurich.ibm.com [9.4.16.232]) by d12nrmr1507.megacenter.de.ibm.com (8.12.10/NCO/VER6.6) with ESMTP id i428HlCE135618; Sun, 2 May 2004 10:17:48 +0200 Received: from zurich.ibm.com (gsine09.us.sine.ibm.com [9.14.6.39]) by sihl.zurich.ibm.com (AIX4.3/8.9.3p2/8.9.3) with SMTP id KAA62264; Sun, 2 May 2004 10:17:45 +0200 Message-ID: <4094AEB2.4050606@zurich.ibm.com> Date: Sun, 02 May 2004 10:17:54 +0200 From: Brian E Carpenter Organization: IBM User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.6) Gecko/20040113 X-Accept-Language: en, fr, de MIME-Version: 1.0 To: Harald Tveit Alvestrand CC: icar@ietf.org Subject: Re: [Icar] Re: Comments on draft-alvestrand-icar-xarea-00.txt References: <697DAA22C5004B4596E033803A7CEF44024DC130@daebe007.americas.noki a.com> <159749067.1083333395@localhost> In-Reply-To: <159749067.1083333395@localhost> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Content-Transfer-Encoding: 7bit Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.0 required=5.0 tests=AWL autolearn=no version=2.60 Content-Transfer-Encoding: 7bit Content-Transfer-Encoding: 7bit Harald Tveit Alvestrand wrote: > > > --On 30. april 2004 13:49 -0500 Basavaraj.Patil@nokia.com wrote: > >> ... >> 2. The I-D is unclear about when the review team is actually involved >> in doing the document reviews. It says late review. But just for >> clarification does it imply post WG last call or IETF last call? > > > I was thinking post IETF Last Call, because Last Call comments should > form part of the input to the review team, just as for the IESG today. > Other models are possible. Good point! And this is a problem in the split model we seem to be evolving, with some mid-review done by WG-based teams and this very late review done by cross-area teams. IMHO cross-area review should be a continuous process, starting at the draft-authorname-...-00.txt version and continuing until the draft-ietf-...-99.txt version. The arguments for that opinion are in the SIRs draft. We still need to get rid of late surprises. Brian > >> >> 3. If there is consensus by the review team on a document, the IESG >> role is quite limited isnt it? Does the IESG have to review the >> document even after review team approval? Or does the IESG simply >> review the "review team's" notes? From a process efficiency >> perspective at least, there is no reason for the IESG to redo the >> work done by the review team. > > > As I said in the Korea ICAR meeting, I would like to try to first create > the function, then move the formal responsibility. In this case, it's > likely (I hope) that IESG review could be limited to "writeup looks > reasonable, we believe them". > > Harald > > > > _______________________________________________ > Icar mailing list > Icar@ietf.org > https://www1.ietf.org/mailman/listinfo/icar > _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Sun May 2 15:50:08 2004 Received: from optimus.ietf.org (optimus.ietf.org [132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA00329 for ; Sun, 2 May 2004 15:50:07 -0400 (EDT) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BKMlT-00021W-5W for icar-archive@odin.ietf.org; Sun, 02 May 2004 15:36:39 -0400 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i42JadIL007774 for icar-archive@odin.ietf.org; Sun, 2 May 2004 15:36:39 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BKMk8-0000XH-2E for icar-web-archive@optimus.ietf.org; Sun, 02 May 2004 15:35:16 -0400 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA25896 for ; Sun, 2 May 2004 14:48:07 -0400 (EDT) Received: from ietf-mx.ietf.org ([132.151.6.1] helo=ietf-mx) by ietf-mx with esmtp (Exim 4.32) id 1BKM0W-00036f-50 for icar-web-archive@ietf.org; Sun, 02 May 2004 14:48:08 -0400 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1BKLzX-0002sP-00 for icar-web-archive@ietf.org; Sun, 02 May 2004 14:47:08 -0400 Received: from optimus.ietf.org ([132.151.1.19]) by ietf-mx with esmtp (Exim 4.12) id 1BKLyr-0002e8-00 for icar-web-archive@ietf.org; Sun, 02 May 2004 14:46:25 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BKLva-0000AI-SF; Sun, 02 May 2004 14:43:02 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BKLsn-00073k-Iw for icar@optimus.ietf.org; Sun, 02 May 2004 14:40:09 -0400 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA25576 for ; Sun, 2 May 2004 14:40:06 -0400 (EDT) Received: from ietf-mx.ietf.org ([132.151.6.1] helo=ietf-mx) by ietf-mx with esmtp (Exim 4.32) id 1BKLsk-0001CI-R0 for icar@ietf.org; Sun, 02 May 2004 14:40:06 -0400 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1BKLru-0000xy-00 for icar@ietf.org; Sun, 02 May 2004 14:39:14 -0400 Received: from eikenes.alvestrand.no ([158.38.152.233]) by ietf-mx with esmtp (Exim 4.12) id 1BKLqx-0000Vh-00 for icar@ietf.org; Sun, 02 May 2004 14:38:15 -0400 Received: from localhost (localhost.localdomain [127.0.0.1]) by eikenes.alvestrand.no (Postfix) with ESMTP id 462C761C05; Sun, 2 May 2004 20:37:45 +0200 (CEST) Received: from eikenes.alvestrand.no ([127.0.0.1]) by localhost (eikenes.alvestrand.no [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 24199-02; Sun, 2 May 2004 20:37:44 +0200 (CEST) Received: from halvestr-w2k02.emea.cisco.com (localhost.localdomain [127.0.0.1]) by eikenes.alvestrand.no (Postfix) with ESMTP id E195361B7D; Sun, 2 May 2004 20:37:42 +0200 (CEST) Date: Sun, 02 May 2004 09:30:53 -0700 From: Harald Tveit Alvestrand To: Brian E Carpenter Cc: icar@ietf.org Subject: Re: [Icar] Re: Comments on draft-alvestrand-icar-xarea-00.txt Message-ID: <316605053.1083490253@localhost> In-Reply-To: <4094AEB2.4050606@zurich.ibm.com> References: <697DAA22C5004B4596E033803A7CEF44024DC130@daebe007.americas.noki a.com> <159749067.1083333395@localhost> <4094AEB2.4050606@zurich.ibm.com> X-Mailer: Mulberry/3.1.0 (Win32) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline X-Virus-Scanned: by amavisd-new at alvestrand.no Content-Transfer-Encoding: 7bit Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.0 required=5.0 tests=AWL autolearn=no version=2.60 Content-Transfer-Encoding: 7bit Content-Transfer-Encoding: 7bit --On 2. mai 2004 10:17 +0200 Brian E Carpenter wrote: > Harald Tveit Alvestrand wrote: >> >> >> --On 30. april 2004 13:49 -0500 Basavaraj.Patil@nokia.com wrote: >> >>> > ... >>> 2. The I-D is unclear about when the review team is actually involved >>> in doing the document reviews. It says late review. But just for >>> clarification does it imply post WG last call or IETF last call? >> >> >> I was thinking post IETF Last Call, because Last Call comments should >> form part of the input to the review team, just as for the IESG today. >> Other models are possible. Good point! > > And this is a problem in the split model we seem to be evolving, > with some mid-review done by WG-based teams and this very late review > done by cross-area teams. > > IMHO cross-area review should be a continuous process, starting at the > draft-authorname-...-00.txt version and continuing until the > draft-ietf-...-99.txt version. The arguments for that opinion > are in the SIRs draft. trying to depart from the box I put on the table.... would it make sense to move the start of review-team review to (say) "publication requested", and only say that the *end* of review is constrained to be after Last Call? Would also give reviewers more time..... _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Mon May 3 05:27:23 2004 Received: from optimus.ietf.org (optimus.ietf.org [132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id FAA16644 for ; Mon, 3 May 2004 05:27:23 -0400 (EDT) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BKZeS-00073u-DP for icar-archive@odin.ietf.org; Mon, 03 May 2004 05:22:16 -0400 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i439MGpw027147 for icar-archive@odin.ietf.org; Mon, 3 May 2004 05:22:16 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BKZYo-0004mm-Nw for icar-web-archive@optimus.ietf.org; Mon, 03 May 2004 05:16:26 -0400 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id FAA16029 for ; Mon, 3 May 2004 05:16:23 -0400 (EDT) Received: from ietf-mx.ietf.org ([132.151.6.1] helo=ietf-mx) by ietf-mx with esmtp (Exim 4.32) id 1BKZYl-0002hV-HH for icar-web-archive@ietf.org; Mon, 03 May 2004 05:16:23 -0400 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1BKZVi-00029K-00 for icar-web-archive@ietf.org; Mon, 03 May 2004 05:13:16 -0400 Received: from optimus.ietf.org ([132.151.1.19]) by ietf-mx with esmtp (Exim 4.12) id 1BKZRK-0001Qi-00 for icar-web-archive@ietf.org; Mon, 03 May 2004 05:08:42 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BKZPi-0001lH-6p; Mon, 03 May 2004 05:07:02 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BKZIh-0007qu-HL for icar@optimus.ietf.org; Mon, 03 May 2004 04:59:47 -0400 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id EAA15399 for ; Mon, 3 May 2004 04:59:43 -0400 (EDT) Received: from ietf-mx.ietf.org ([132.151.6.1] helo=ietf-mx) by ietf-mx with esmtp (Exim 4.32) id 1BKZId-0007mH-Hm for icar@ietf.org; Mon, 03 May 2004 04:59:43 -0400 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1BKZFH-0007BX-00 for icar@ietf.org; Mon, 03 May 2004 04:56:16 -0400 Received: from mtagate3.de.ibm.com ([195.212.29.152]) by ietf-mx with esmtp (Exim 4.12) id 1BKZBf-0006QO-00 for icar@ietf.org; Mon, 03 May 2004 04:52:31 -0400 Received: from d12nrmr1607.megacenter.de.ibm.com (d12nrmr1607.megacenter.de.ibm.com [9.149.167.49]) by mtagate3.de.ibm.com (8.12.10/8.12.10) with ESMTP id i438pTn5031332; Mon, 3 May 2004 08:51:32 GMT Received: from sihl.zurich.ibm.com (sihl.zurich.ibm.com [9.4.16.232]) by d12nrmr1607.megacenter.de.ibm.com (8.12.10/NCO/VER6.6) with ESMTP id i438pFk6220768; Mon, 3 May 2004 10:51:16 +0200 Received: from zurich.ibm.com (sig-9-145-137-58.de.ibm.com [9.145.137.58]) by sihl.zurich.ibm.com (AIX4.3/8.9.3p2/8.9.3) with ESMTP id KAA59368; Mon, 3 May 2004 10:51:15 +0200 Message-ID: <4096080E.7020601@zurich.ibm.com> Date: Mon, 03 May 2004 10:51:26 +0200 From: Brian E Carpenter Organization: IBM User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.6) Gecko/20040113 X-Accept-Language: en, fr, de MIME-Version: 1.0 To: Harald Tveit Alvestrand CC: icar@ietf.org Subject: Re: [Icar] Re: Comments on draft-alvestrand-icar-xarea-00.txt References: <697DAA22C5004B4596E033803A7CEF44024DC130@daebe007.americas.noki a.com> <159749067.1083333395@localhost> <4094AEB2.4050606@zurich.ibm.com> <316605053.1083490253@localhost> In-Reply-To: <316605053.1083490253@localhost> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Content-Transfer-Encoding: 7bit Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.0 required=5.0 tests=AWL autolearn=no version=2.60 Content-Transfer-Encoding: 7bit Content-Transfer-Encoding: 7bit Harald Tveit Alvestrand wrote: > > > --On 2. mai 2004 10:17 +0200 Brian E Carpenter wrote: > >> Harald Tveit Alvestrand wrote: >> >>> >>> >>> --On 30. april 2004 13:49 -0500 Basavaraj.Patil@nokia.com wrote: >>> >>>> >> ... >> >>>> 2. The I-D is unclear about when the review team is actually involved >>>> in doing the document reviews. It says late review. But just for >>>> clarification does it imply post WG last call or IETF last call? >>> >>> >>> >>> I was thinking post IETF Last Call, because Last Call comments should >>> form part of the input to the review team, just as for the IESG today. >>> Other models are possible. Good point! >> >> >> And this is a problem in the split model we seem to be evolving, >> with some mid-review done by WG-based teams and this very late review >> done by cross-area teams. >> >> IMHO cross-area review should be a continuous process, starting at the >> draft-authorname-...-00.txt version and continuing until the >> draft-ietf-...-99.txt version. The arguments for that opinion >> are in the SIRs draft. > > > trying to depart from the box I put on the table.... > would it make sense to move the start of review-team review to (say) > "publication requested", and only say that the *end* of review is > constrained to be after Last Call? > Would also give reviewers more time..... Yes, makes sense. Of course the I-D version number may increase during that time. Brian _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Sat May 8 12:13:43 2004 Received: from optimus.ietf.org (www.iesg.org [132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA10673 for ; Sat, 8 May 2004 12:13:43 -0400 (EDT) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BMUGf-00016s-72 for icar-archive@odin.ietf.org; Sat, 08 May 2004 12:01:37 -0400 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i48G1bWT004266 for icar-archive@odin.ietf.org; Sat, 8 May 2004 12:01:37 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BMUD2-0000ea-Sc for icar-web-archive@optimus.ietf.org; Sat, 08 May 2004 11:57:52 -0400 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA10239 for ; Sat, 8 May 2004 11:57:49 -0400 (EDT) Received: from ietf-mx.ietf.org ([132.151.6.1] helo=ietf-mx) by ietf-mx with esmtp (Exim 4.32) id 1BMUD1-00011B-NN for icar-web-archive@ietf.org; Sat, 08 May 2004 11:57:51 -0400 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1BMUC6-0000dy-00 for icar-web-archive@ietf.org; Sat, 08 May 2004 11:56:55 -0400 Received: from optimus.ietf.org ([132.151.1.19]) by ietf-mx with esmtp (Exim 4.12) id 1BMUBl-0000Gm-00 for icar-web-archive@ietf.org; Sat, 08 May 2004 11:56:33 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BMU9L-0000Eq-9U; Sat, 08 May 2004 11:54:03 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BMU6o-0008Jn-Nm for icar@optimus.ietf.org; Sat, 08 May 2004 11:51:27 -0400 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA10041 for ; Sat, 8 May 2004 11:51:23 -0400 (EDT) Received: from ietf-mx.ietf.org ([132.151.6.1] helo=ietf-mx) by ietf-mx with esmtp (Exim 4.32) id 1BMU6n-00065w-JU for icar@ietf.org; Sat, 08 May 2004 11:51:25 -0400 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1BMU5x-0005ea-00 for icar@ietf.org; Sat, 08 May 2004 11:50:34 -0400 Received: from joy.songbird.com ([208.184.79.7]) by ietf-mx with esmtp (Exim 4.12) id 1BMU4w-00058T-00 for icar@ietf.org; Sat, 08 May 2004 11:49:30 -0400 Received: from bbprime (jay.songbird.com [208.184.79.253]) by joy.songbird.com (8.11.6/8.11.6) with ESMTP id i48Fmrd20311; Sat, 8 May 2004 08:48:53 -0700 Date: Sat, 8 May 2004 08:48:48 -0700 From: Dave Crocker Reply-To: Dave Crocker Organization: Brandenburg InternetWorking X-Priority: 3 (Normal) Message-ID: <156105829.20040508084848@brandenburg.com> To: Brian E Carpenter CC: icar@ietf.org Subject: Re: [Icar] Re: Comments on draft-alvestrand-icar-xarea-00.txt In-Reply-To: <4094AEB2.4050606@zurich.ibm.com> References: <697DAA22C5004B4596E033803A7CEF44024DC130@daebe007.americas.noki a.com> <159749067.1083333395@localhost> <4094AEB2.4050606@zurich.ibm.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Content-Transfer-Encoding: 7bit Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.7 required=5.0 tests=AWL,PRIORITY_NO_NAME autolearn=no version=2.60 Content-Transfer-Encoding: 7bit Content-Transfer-Encoding: 7bit Brian, BEC> And this is a problem in the split model we seem to be evolving, BEC> with some mid-review done by WG-based teams and this very late review BEC> done by cross-area teams. WG-based teams can, and will, get reviews from folks with diverse expertise. Reviewers working for area directors do not have a corner on the supply of cross-area reviewers. In other words, you are correct that there needs to be a continuing process of reviewing from different perspectives. d/ -- Dave Crocker Brandenburg InternetWorking Sunnyvale, CA USA , _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Tue May 18 14:50:04 2004 Received: from optimus.ietf.org (www.iesg.org [132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA05150 for ; Tue, 18 May 2004 14:50:04 -0400 (EDT) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BQ9Xb-00035H-0W for icar-archive@odin.ietf.org; Tue, 18 May 2004 14:42:15 -0400 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i4IIgEHI011851 for icar-archive@odin.ietf.org; Tue, 18 May 2004 14:42:14 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BQ9O0-0001YC-H3 for icar-web-archive@optimus.ietf.org; Tue, 18 May 2004 14:32:20 -0400 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA04043 for ; Tue, 18 May 2004 14:32:17 -0400 (EDT) Received: from ietf-mx.ietf.org ([132.151.6.1] helo=ietf-mx) by ietf-mx with esmtp (Exim 4.32) id 1BQ9Nx-0005h4-Vz for icar-web-archive@ietf.org; Tue, 18 May 2004 14:32:18 -0400 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1BQ9N7-0005bL-00 for icar-web-archive@ietf.org; Tue, 18 May 2004 14:31:26 -0400 Received: from optimus.ietf.org ([132.151.1.19]) by ietf-mx with esmtp (Exim 4.12) id 1BQ9MM-0005WI-00 for icar-web-archive@ietf.org; Tue, 18 May 2004 14:30:38 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BQ93Y-0003zg-AZ; Tue, 18 May 2004 14:11:12 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BQ8wv-0002Ga-8a for icar@optimus.ietf.org; Tue, 18 May 2004 14:04:21 -0400 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA01988 for ; Tue, 18 May 2004 14:04:18 -0400 (EDT) Received: from ietf-mx.ietf.org ([132.151.6.1] helo=ietf-mx) by ietf-mx with esmtp (Exim 4.32) id 1BQ8ws-0003hu-Ra for icar@ietf.org; Tue, 18 May 2004 14:04:18 -0400 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1BQ8w1-0003eN-00 for icar@ietf.org; Tue, 18 May 2004 14:03:26 -0400 Received: from wyvern.icir.org ([192.150.187.14]) by ietf-mx with esmtp (Exim 4.12) id 1BQ8vE-0003ag-00 for icar@ietf.org; Tue, 18 May 2004 14:02:36 -0400 Received: from guns.icir.org (adsl-68-76-113-50.dsl.bcvloh.ameritech.net [68.76.113.50]) by wyvern.icir.org (8.12.9p1/8.12.8) with ESMTP id i4II2Zim038674 for ; Tue, 18 May 2004 11:02:35 -0700 (PDT) (envelope-from mallman@guns.icir.org) Received: from guns.icir.org (localhost [127.0.0.1]) by guns.icir.org (Postfix) with ESMTP id 313BA77AB51 for ; Tue, 18 May 2004 14:02:34 -0400 (EDT) To: icar@ietf.org From: Mark Allman Reply-To: mallman@icir.org Organization: ICSI Center for Internet Research (ICIR) Song-of-the-Day: Finishing Touches MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha1; protocol="application/pgp-signature" Date: Tue, 18 May 2004 14:02:34 -0400 Message-Id: <20040518180234.313BA77AB51@guns.icir.org> Subject: [Icar] an early review experiment Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.0 required=5.0 tests=AWL autolearn=no version=2.60 --=-=-= Content-Type: text/plain; charset=us-ascii Folks- Joel and I chatted with some folks from the change process coordination team this afternoon. We would really like to get some discussion going on the mailing list with regards to moving ahead on *an* early review experiment. Our sense is that even though there are several documents in the early review space that there is an intersection among them that people would like to see, but that has not yet emerged in a concrete way. In an effort to push along the conversation we are going to ask some broad sorts of question below to try to synthesize thoughts into an experimental design. If this doesn't work (for whatever reason) our fallback is to try to take each concrete proposal and beat it into shape and try experiments with all that we can beat into enough shape to do so. In that context, the broad questions that we'd like to discuss are: + At what point should early review happen? Pre-WG document? WG -00 document? Pre-WGLC? All of the above? Other? + Who is responsible for finding reviewers for a document? + From what pool are the reviewers drawn? + Precisely, what is produced? + What does a reviewer do with a review? + What other things need nailed down before we can think about an experiment? (And, note, some of these are thrown out for completeness and we believe there is decent consensus on the answer - e.g., the "what to do with a review?" question.) Discussion would be appreciated! allman -- Mark Allman -- ICIR -- http://www.icir.org/mallman/ --=-=-= Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.3 (FreeBSD) iD8DBQFAqk+6WyrrWs4yIs4RAgKLAKCOMfMUnp7ZJ7ccXQOLS7KqNU9lrACfbf0b hyI2tpYAKqBE7iKcYo+Fpog= =5Nuw -----END PGP SIGNATURE----- --=-=-=-- _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Tue May 18 17:17:09 2004 Received: from optimus.ietf.org (www.iesg.org [132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA19499 for ; Tue, 18 May 2004 17:17:09 -0400 (EDT) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BQBq8-0001Kw-Nq for icar-archive@odin.ietf.org; Tue, 18 May 2004 17:09:33 -0400 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i4IL9W1d005134 for icar-archive@odin.ietf.org; Tue, 18 May 2004 17:09:32 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BQBbe-00055B-Su for icar-web-archive@optimus.ietf.org; Tue, 18 May 2004 16:54:34 -0400 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA17709 for ; Tue, 18 May 2004 16:54:32 -0400 (EDT) Received: from ietf-mx.ietf.org ([132.151.6.1] helo=ietf-mx) by ietf-mx with esmtp (Exim 4.32) id 1BQBbc-0003E0-Rh for icar-web-archive@ietf.org; Tue, 18 May 2004 16:54:32 -0400 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1BQBap-000365-00 for icar-web-archive@ietf.org; Tue, 18 May 2004 16:53:43 -0400 Received: from optimus.ietf.org ([132.151.1.19]) by ietf-mx with esmtp (Exim 4.12) id 1BQBZd-0002um-00 for icar-web-archive@ietf.org; Tue, 18 May 2004 16:52:29 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BQBOc-0007gp-OC; Tue, 18 May 2004 16:41:06 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BQAWP-0006zi-3y for icar@optimus.ietf.org; Tue, 18 May 2004 15:45:05 -0400 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA10020 for ; Tue, 18 May 2004 15:45:03 -0400 (EDT) Received: from ietf-mx.ietf.org ([132.151.6.1] helo=ietf-mx) by ietf-mx with esmtp (Exim 4.32) id 1BQAWN-0002CA-Ld for icar@ietf.org; Tue, 18 May 2004 15:45:03 -0400 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1BQAVQ-00027y-00 for icar@ietf.org; Tue, 18 May 2004 15:44:05 -0400 Received: from hoemail2.lucent.com ([192.11.226.163] helo=hoemail2.firewall.lucent.com) by ietf-mx with esmtp (Exim 4.12) id 1BQAUR-00021G-00 for icar@ietf.org; Tue, 18 May 2004 15:43:03 -0400 Received: from nl0006exch001h.wins.lucent.com (h135-85-76-62.lucent.com [135.85.76.62]) by hoemail2.firewall.lucent.com (Switch-2.2.8/Switch-2.2.8) with ESMTP id i4IJgRc03303 for ; Tue, 18 May 2004 14:42:27 -0500 (CDT) Received: by nl0006exch001h.nl.lucent.com with Internet Mail Service (5.5.2657.72) id ; Tue, 18 May 2004 21:42:29 +0200 Message-ID: <7D5D48D2CAA3D84C813F5B154F43B1550455911D@nl0006exch001u.nl.lucent.com> From: "Wijnen, Bert (Bert)" To: mallman@icir.org, icar@ietf.org Subject: RE: [Icar] an early review experiment Date: Tue, 18 May 2004 21:42:28 +0200 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2657.72) Content-Type: text/plain Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.0 required=5.0 tests=AWL autolearn=no version=2.60 I think that instead of "early review" you mean "early cross area review" (i.e. icar) in the below. Do I have that correct? Thanks, Bert > -----Original Message----- > From: Mark Allman [mailto:mallman@icir.org] > Sent: dinsdag 18 mei 2004 20:03 > To: icar@ietf.org > Subject: [Icar] an early review experiment > > > > Folks- > > Joel and I chatted with some folks from the change process > coordination > team this afternoon. We would really like to get some > discussion going > on the mailing list with regards to moving ahead on *an* early review > experiment. Our sense is that even though there are several documents > in the early review space that there is an intersection among > them that > people would like to see, but that has not yet emerged in a concrete > way. In an effort to push along the conversation we are going to ask > some broad sorts of question below to try to synthesize > thoughts into an > experimental design. If this doesn't work (for whatever reason) our > fallback is to try to take each concrete proposal and beat it > into shape > and try experiments with all that we can beat into enough shape to do > so. > > In that context, the broad questions that we'd like to discuss are: > > + At what point should early review happen? Pre-WG > document? WG -00 > document? Pre-WGLC? All of the above? Other? > > + Who is responsible for finding reviewers for a document? > > + From what pool are the reviewers drawn? > > + Precisely, what is produced? > > + What does a reviewer do with a review? > > + What other things need nailed down before we can think about an > experiment? > > (And, note, some of these are thrown out for completeness and > we believe > there is decent consensus on the answer - e.g., the "what to do with a > review?" question.) > > Discussion would be appreciated! > > allman > > > -- > Mark Allman -- ICIR -- http://www.icir.org/mallman/ > > > > _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Tue May 18 17:17:27 2004 Received: from optimus.ietf.org (www.iesg.org [132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA19555 for ; Tue, 18 May 2004 17:17:27 -0400 (EDT) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BQBq8-0001L9-Qc for icar-archive@odin.ietf.org; Tue, 18 May 2004 17:09:33 -0400 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i4IL9Wbd005147 for icar-archive@odin.ietf.org; Tue, 18 May 2004 17:09:32 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BQBbn-00057Y-Tk for icar-web-archive@optimus.ietf.org; Tue, 18 May 2004 16:54:43 -0400 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA17720 for ; Tue, 18 May 2004 16:54:41 -0400 (EDT) Received: from ietf-mx.ietf.org ([132.151.6.1] helo=ietf-mx) by ietf-mx with esmtp (Exim 4.32) id 1BQBbl-0003El-T9 for icar-web-archive@ietf.org; Tue, 18 May 2004 16:54:42 -0400 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1BQBb1-00037u-00 for icar-web-archive@ietf.org; Tue, 18 May 2004 16:53:55 -0400 Received: from optimus.ietf.org ([132.151.1.19]) by ietf-mx with esmtp (Exim 4.12) id 1BQBZv-0002xL-00 for icar-web-archive@ietf.org; Tue, 18 May 2004 16:52:47 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BQBOd-0007h3-2E; Tue, 18 May 2004 16:41:07 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BQAWa-0006zm-Ob for icar@optimus.ietf.org; Tue, 18 May 2004 15:45:16 -0400 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA10028 for ; Tue, 18 May 2004 15:45:14 -0400 (EDT) Received: from ietf-mx.ietf.org ([132.151.6.1] helo=ietf-mx) by ietf-mx with esmtp (Exim 4.32) id 1BQAWZ-0002Dm-9c for icar@ietf.org; Tue, 18 May 2004 15:45:15 -0400 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1BQAVd-00029c-00 for icar@ietf.org; Tue, 18 May 2004 15:44:18 -0400 Received: from wyvern.icir.org ([192.150.187.14]) by ietf-mx with esmtp (Exim 4.12) id 1BQAUl-00025A-00 for icar@ietf.org; Tue, 18 May 2004 15:43:23 -0400 Received: from guns.icir.org (adsl-68-76-113-50.dsl.bcvloh.ameritech.net [68.76.113.50]) by wyvern.icir.org (8.12.9p1/8.12.8) with ESMTP id i4IJhNim040033; Tue, 18 May 2004 12:43:23 -0700 (PDT) (envelope-from mallman@guns.icir.org) Received: from guns.icir.org (localhost [127.0.0.1]) by guns.icir.org (Postfix) with ESMTP id 8C0DC77AB51; Tue, 18 May 2004 15:43:21 -0400 (EDT) To: "Wijnen, Bert (Bert)" From: Mark Allman Reply-To: mallman@icir.org Cc: icar@ietf.org Subject: Re: [Icar] an early review experiment In-Reply-To: <7D5D48D2CAA3D84C813F5B154F43B1550455911D@nl0006exch001u.nl.lucent.com> Organization: ICSI Center for Internet Research (ICIR) Song-of-the-Day: Finishing Touches MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha1; protocol="application/pgp-signature" Date: Tue, 18 May 2004 15:43:21 -0400 Message-Id: <20040518194321.8C0DC77AB51@guns.icir.org> Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.0 required=5.0 tests=AWL autolearn=no version=2.60 --=-=-= Content-Type: text/plain; charset=us-ascii > I think that instead of "early review" you mean "early cross area > review" (i.e. icar) in the below. Do I have that correct? Indeed that is what I meant. I was typing too rapidly. Thanks for clarifying. allman --=-=-= Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.3 (FreeBSD) iD8DBQFAqmdZWyrrWs4yIs4RAitXAKCKoRIw7A0tJS9ifbu/QJfY/4vaGwCfe/rF nWtJE+thah6YzVSShbjtoX8= =Yw1a -----END PGP SIGNATURE----- --=-=-=-- _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Tue May 18 17:20:08 2004 Received: from optimus.ietf.org (www.iesg.org [132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA19789 for ; Tue, 18 May 2004 17:20:08 -0400 (EDT) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BQBuv-0002op-Cz for icar-archive@odin.ietf.org; Tue, 18 May 2004 17:14:29 -0400 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i4ILETXi010836 for icar-archive@odin.ietf.org; Tue, 18 May 2004 17:14:29 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BQBkM-0007L0-5y for icar-web-archive@optimus.ietf.org; Tue, 18 May 2004 17:03:34 -0400 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA18237 for ; Tue, 18 May 2004 17:03:31 -0400 (EDT) Received: from ietf-mx.ietf.org ([132.151.6.1] helo=ietf-mx) by ietf-mx with esmtp (Exim 4.32) id 1BQBkK-0003qh-6G for icar-web-archive@ietf.org; Tue, 18 May 2004 17:03:32 -0400 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1BQBjG-0003m0-00 for icar-web-archive@ietf.org; Tue, 18 May 2004 17:02:27 -0400 Received: from optimus.ietf.org ([132.151.1.19]) by ietf-mx with esmtp (Exim 4.12) id 1BQBil-0003hV-00 for icar-web-archive@ietf.org; Tue, 18 May 2004 17:01:55 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BQBVV-0001lD-6x; Tue, 18 May 2004 16:48:13 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BQAxJ-0007Fh-Fn for icar@optimus.ietf.org; Tue, 18 May 2004 16:12:53 -0400 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA12085 for ; Tue, 18 May 2004 16:12:51 -0400 (EDT) Received: from ietf-mx.ietf.org ([132.151.6.1] helo=ietf-mx) by ietf-mx with esmtp (Exim 4.32) id 1BQAxH-0004rj-P2 for icar@ietf.org; Tue, 18 May 2004 16:12:51 -0400 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1BQAw7-0004gb-00 for icar@ietf.org; Tue, 18 May 2004 16:11:40 -0400 Received: from netcore.fi ([193.94.160.1]) by ietf-mx with esmtp (Exim 4.12) id 1BQAvC-0004V0-00 for icar@ietf.org; Tue, 18 May 2004 16:10:42 -0400 Received: from localhost (pekkas@localhost) by netcore.fi (8.11.6/8.11.6) with ESMTP id i4IKA2W31666; Tue, 18 May 2004 23:10:02 +0300 Date: Tue, 18 May 2004 23:10:02 +0300 (EEST) From: Pekka Savola To: Mark Allman cc: icar@ietf.org Subject: Re: [Icar] an early review experiment In-Reply-To: <20040518180234.313BA77AB51@guns.icir.org> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.0 required=5.0 tests=AWL autolearn=no version=2.60 On Tue, 18 May 2004, Mark Allman wrote: > In that context, the broad questions that we'd like to discuss are: > > + At what point should early review happen? Pre-WG document? WG -00 > document? Pre-WGLC? All of the above? Other? Because it seems *very* difficult to kill off WG work items after they have been accepted, some review should happen just before accepting a document as a WG work item. This should be done to kill any obvious bogosities and alert of cross-area problems. A second set of early reviews should happen at some point after -00 WG document and between/around WGLC. Whenever the document is deemed suitably stable for consumption.. > + Who is responsible for finding reviewers for a document? > > + From what pool are the reviewers drawn? These two are IMHO the most difficult questions from the practical point of view. For example, as WG chair, I've been instructed to get review from e.g., apps area for a document. Despite begging from the apps ADs, mailing numerous times on appsarea mailing list, getting a person signed up for review (who ultimately did nothing), I have to say it must be *very* clear who has the "responsibility" for producing reviewers/reviews for the document. This cannot just be the WG chair or the shepherding AD in question. Maybe if there was an area-specific early review board, the chair(s) of that board would be responsible for getting the document reviewed. As to the pool where the reviewers should come from.. that's maybe slightly easier. This stuff would just need to be organized sufficiently well, with sufficient buy-in from the IESG, and appreciation by the peers. The criteria like used for SIRS or to get ARTS might work here, as well as volunteers. But the most important thing is that those people do actually review documents (at least 1 per month), and there is a system to make sure the review will end up getting done. > + Precisely, what is produced? A SIRS-like decree like "ship it, needs polish, needs a lot of work, kill this madness" would probably be useful, but this should be coupled with technical comments as well. > + What does a reviewer do with a review? Did you mean 'what does the reviewee (whose document one reviewed) do with the review?' as I don't understand why the above question is relevant. Assuming so, this is again a good question. The review should be sent tot he document editor(s), WG chairs, and possibly the associated AD -- and one of which should file it in an issue tracker if used. The reviews should also be publicly stored in a web location. The comments should be treated with equal weight as IESG Discuss comments. > + What other things need nailed down before we can think about an > experiment? It's already implicit from the above, but it would probably make sense to consider at more length when this kind of review should be initiated and by whom. I.e., whose responsibility is it to start the review process, and ensure that sufficient review gets done before going to the next step in a draft's lifetime (e.g., adopting it as WG item, etc.). -- Pekka Savola "You each name yourselves king, yet the Netcore Oy kingdom bleeds." Systems. Networks. Security. -- George R.R. Martin: A Clash of Kings _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Wed May 19 06:02:54 2004 Received: from optimus.ietf.org (optimus.ietf.org [132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id GAA26874 for ; Wed, 19 May 2004 06:02:53 -0400 (EDT) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BQNk5-0000IH-0E for icar-archive@odin.ietf.org; Wed, 19 May 2004 05:52:05 -0400 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i4J9q47J001124 for icar-archive@odin.ietf.org; Wed, 19 May 2004 05:52:04 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BQNYL-0005SE-5I for icar-web-archive@optimus.ietf.org; Wed, 19 May 2004 05:39:57 -0400 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id FAA24830 for ; Wed, 19 May 2004 05:39:53 -0400 (EDT) Received: from ietf-mx.ietf.org ([132.151.6.1] helo=ietf-mx) by ietf-mx with esmtp (Exim 4.32) id 1BQNYH-0004h5-NX for icar-web-archive@ietf.org; Wed, 19 May 2004 05:39:53 -0400 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1BQNXF-0004SK-00 for icar-web-archive@ietf.org; Wed, 19 May 2004 05:38:50 -0400 Received: from optimus.ietf.org ([132.151.1.19]) by ietf-mx with esmtp (Exim 4.12) id 1BQNWQ-0004F4-00 for icar-web-archive@ietf.org; Wed, 19 May 2004 05:37:58 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BQN6T-0007fF-Lg; Wed, 19 May 2004 05:11:09 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BQM8o-0008Kd-TN for icar@optimus.ietf.org; Wed, 19 May 2004 04:09:31 -0400 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id EAA20505 for ; Wed, 19 May 2004 04:09:28 -0400 (EDT) Received: from ietf-mx.ietf.org ([132.151.6.1] helo=ietf-mx) by ietf-mx with esmtp (Exim 4.32) id 1BQM8m-0006mF-8p for icar@ietf.org; Wed, 19 May 2004 04:09:28 -0400 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1BQM7n-0006e9-00 for icar@ietf.org; Wed, 19 May 2004 04:08:28 -0400 Received: from mtagate2.de.ibm.com ([195.212.29.151]) by ietf-mx with esmtp (Exim 4.12) id 1BQM6s-0006QW-00 for icar@ietf.org; Wed, 19 May 2004 04:07:30 -0400 Received: from d12nrmr1507.megacenter.de.ibm.com (d12nrmr1507.megacenter.de.ibm.com [9.149.167.1]) by mtagate2.de.ibm.com (8.12.10/8.12.10) with ESMTP id i4J86tlg095742; Wed, 19 May 2004 08:06:55 GMT Received: from sihl.zurich.ibm.com (sihl.zurich.ibm.com [9.4.16.232]) by d12nrmr1507.megacenter.de.ibm.com (8.12.10/NCO/VER6.6) with ESMTP id i4J86sYB246708; Wed, 19 May 2004 10:06:54 +0200 Received: from zurich.ibm.com (sig-9-145-226-16.de.ibm.com [9.145.226.16]) by sihl.zurich.ibm.com (AIX4.3/8.9.3p2/8.9.3) with ESMTP id KAA26480; Wed, 19 May 2004 10:06:53 +0200 Message-ID: <40AB15A0.5090804@zurich.ibm.com> Date: Wed, 19 May 2004 10:06:56 +0200 From: Brian E Carpenter Organization: IBM User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.6) Gecko/20040113 X-Accept-Language: en, fr, de MIME-Version: 1.0 To: mallman@icir.org CC: icar@ietf.org Subject: Re: [Icar] an early review experiment References: <20040518180234.313BA77AB51@guns.icir.org> In-Reply-To: <20040518180234.313BA77AB51@guns.icir.org> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Content-Transfer-Encoding: 7bit Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.0 required=5.0 tests=AWL autolearn=no version=2.60 Content-Transfer-Encoding: 7bit Content-Transfer-Encoding: 7bit These questions are discussed in detail in draft-carpenter-solution-sirs-01.txt but I have done some cut and paste below. Mark Allman wrote: > > + At what point should early review happen? Pre-WG document? WG -00 > document? Pre-WGLC? All of the above? Other? It is not expected that every version of an Internet Draft should be submitted for SIR review. However, it is advisable to request reviews at the very beginning (to check for fundamental issues), as major technical issues are resolved, and again just before the document is submitted for IESG approval (or to the RFC Editor). Thus three SIR review cycles per document may be considered the minimum. The document author should ensure that the SIRs reviewing a document understand what stage in its life cycle it has reached, so that they can review it appropriately. Both Working Groups and individual submitters should realise that carding should start early (to detect and hopefully fix fundamental problems) and be repeated as often as needed (to avoid submitting inadequate documents to the IESG). > > + Who is responsible for finding reviewers for a document? 2.2.1. Working Group documents For documents being processed by a Working Group (WG), the WG solicits the assistance of SIRs. [which to my mind means the WG Chairs or Secretary] ... 2.2.2. Individual submissions For individual submissions, the document author(s) should solicit SIR reviews, according to the same principles applied to Working Group documents, > > + From what pool are the reviewers drawn? An initial target of 100 people is suggested. It is recognized that this may be insufficient, so the number should be revised in the light of experience. Q. Is this anything like the right number? A. Assume we need 9 reviews per year (3 review cycles, 3 reviewers each time) for each of 200 RFCs. That gives each SIR one review every 20 days, approximately. This seems high, and suggests that the final target should be 200 people. The goal is to identify a team of people with adequate experience contributing to IETF technical work, who are likely to be trusted as a group to be both technically expert and unbiased. > > + Precisely, what is produced? There is a whole section (3.3) about this in the SIRs draft, from which this is a very small extract: SIRs should review for all kinds of problems, from basic architectural or security issues, Internet-wide impact, technical nits, problems of form and format (such as IANA Considerations or incorrect references), and editorial issues. As a draft progresses from its initial, "-00" version towards one that is ready for submission, successive SIR reviews should progress from the general architectural level to the editorial level. > > + What does a reviewer do with a review? 3.1. Reviewing in Public ... All reviews will be published on the web site, with adequate tooling (linked to the ID Tracker) so that SIRs can publish reviews without adminstrative overhead, every review can be readily found, and all reviews will be automatically archived. In fact, a WG or document author in need of reviews should be able to request them through the web site > > + What other things need nailed down before we can think about an > experiment? From the SIRs experience, do not even think about starting until you have a good web site up & running where (a) all reviewers can be registered (b) authors and WG Chairs can request reviews and (c) reviewers can post, and the public can read, their reviews. Brian _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Wed May 19 09:08:20 2004 Received: from optimus.ietf.org (www.iesg.org [132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id JAA08811 for ; Wed, 19 May 2004 09:08:20 -0400 (EDT) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BQQlW-0002Yu-5F for icar-archive@odin.ietf.org; Wed, 19 May 2004 09:05:46 -0400 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i4JD5kas009849 for icar-archive@odin.ietf.org; Wed, 19 May 2004 09:05:46 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BQQa9-00076r-RO for icar-web-archive@optimus.ietf.org; Wed, 19 May 2004 08:54:01 -0400 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id IAA06806 for ; Wed, 19 May 2004 08:53:59 -0400 (EDT) Received: from ietf-mx.ietf.org ([132.151.6.1] helo=ietf-mx) by ietf-mx with esmtp (Exim 4.32) id 1BQQa8-0007R3-ET for icar-web-archive@ietf.org; Wed, 19 May 2004 08:54:00 -0400 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1BQQZG-0007Ed-00 for icar-web-archive@ietf.org; Wed, 19 May 2004 08:53:07 -0400 Received: from optimus.ietf.org ([132.151.1.19]) by ietf-mx with esmtp (Exim 4.12) id 1BQQYE-00072h-00 for icar-web-archive@ietf.org; Wed, 19 May 2004 08:52:02 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BQQL4-0004CY-1o; Wed, 19 May 2004 08:38:26 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BQPbB-0005a1-RK for icar@optimus.ietf.org; Wed, 19 May 2004 07:51:01 -0400 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id HAA03492 for ; Wed, 19 May 2004 07:51:00 -0400 (EDT) Received: from ietf-mx.ietf.org ([132.151.6.1] helo=ietf-mx) by ietf-mx with esmtp (Exim 4.32) id 1BQPbA-0003pT-UR for icar@ietf.org; Wed, 19 May 2004 07:51:01 -0400 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1BQPZi-0003TT-00 for icar@ietf.org; Wed, 19 May 2004 07:49:31 -0400 Received: from rwcrmhc11.comcast.net ([204.127.198.35]) by ietf-mx with esmtp (Exim 4.12) id 1BQPY8-00032K-00 for icar@ietf.org; Wed, 19 May 2004 07:47:52 -0400 Received: from dfnjgl21 (c-24-1-99-5.client.comcast.net[24.1.99.5]) by comcast.net (rwcrmhc11) with SMTP id <20040519114716013003rqh8e> (Authid: sdawkins@comcast.net); Wed, 19 May 2004 11:47:16 +0000 Message-ID: <068e01c43d97$24266ab0$0200a8c0@DFNJGL21> Reply-To: "Spencer Dawkins" From: "Spencer Dawkins" To: References: <20040518180234.313BA77AB51@guns.icir.org> <40AB15A0.5090804@zurich.ibm.com> Subject: Re: [Icar] an early review experiment Date: Wed, 19 May 2004 06:48:02 -0500 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2800.1409 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1409 Content-Transfer-Encoding: 7bit Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.0 required=5.0 tests=AWL autolearn=no version=2.60 Content-Transfer-Encoding: 7bit Content-Transfer-Encoding: 7bit I'm registered as a SIR, so I'm obviously pretty comfortable with the view of SIRs in this draft, but it seems to me like one tipping point needs to be pointed out. From: "Brian E Carpenter" Sent: Wednesday, May 19, 2004 3:06 AM > These questions are discussed in detail in > draft-carpenter-solution-sirs-01.txt > but I have done some cut and paste below. > > Mark Allman wrote: > > > > + Precisely, what is produced? > > There is a whole section (3.3) about this in the SIRs draft, > from which this is a very small extract: > > SIRs should review for all kinds of problems, from > basic architectural or security issues, Internet-wide > impact, technical nits, problems of form and format > (such as IANA Considerations or incorrect references), > and editorial issues. As a draft progresses from its > initial, "-00" version towards one that is ready for > submission, successive SIR reviews should progress from > the general architectural level to the editorial level. It seems to me that I'm seeing two types of reviewer pools proposed - a reviewer pool that ties to a particular Area of expertise (so the transport ADs provide transport reviewers to the other Area Review Teams, and these reviewers look for transport sanity), or a reviewer pool that reviews as generalists, as do SIRs (and as do ADs, during AD review). There are reasons for both, even for early cross-area review. It Would Be Nice if we didn't try to choose between them. > > + What other things need nailed down before we can think about an > > experiment? > > From the SIRs experience, do not even think about starting until > you have a good web site up & running where (a) all reviewers can be > registered (b) authors and WG Chairs can request reviews and > (c) reviewers can post, and the public can read, their reviews. Amen. _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Wed May 19 10:45:33 2004 Received: from optimus.ietf.org (www.iesg.org [132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id KAA16582 for ; Wed, 19 May 2004 10:45:32 -0400 (EDT) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BQSAD-00053x-3p for icar-archive@odin.ietf.org; Wed, 19 May 2004 10:35:22 -0400 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i4JEZK9P019454 for icar-archive@odin.ietf.org; Wed, 19 May 2004 10:35:20 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BQRyM-0001jz-HR for icar-web-archive@optimus.ietf.org; Wed, 19 May 2004 10:23:06 -0400 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id KAA14531 for ; Wed, 19 May 2004 10:23:03 -0400 (EDT) Received: from ietf-mx.ietf.org ([132.151.6.1] helo=ietf-mx) by ietf-mx with esmtp (Exim 4.32) id 1BQRyK-0000Tm-BJ for icar-web-archive@ietf.org; Wed, 19 May 2004 10:23:04 -0400 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1BQRxK-0000Lm-00 for icar-web-archive@ietf.org; Wed, 19 May 2004 10:22:03 -0400 Received: from optimus.ietf.org ([132.151.1.19]) by ietf-mx with esmtp (Exim 4.12) id 1BQRwI-0000E0-00 for icar-web-archive@ietf.org; Wed, 19 May 2004 10:20:58 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BQRfF-0002Rh-ML; Wed, 19 May 2004 10:03:21 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BQQpE-0003mM-Ag for icar@optimus.ietf.org; Wed, 19 May 2004 09:09:36 -0400 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id JAA09008 for ; Wed, 19 May 2004 09:09:33 -0400 (EDT) Received: from ietf-mx.ietf.org ([132.151.6.1] helo=ietf-mx) by ietf-mx with esmtp (Exim 4.32) id 1BQQpC-0003K0-QQ for icar@ietf.org; Wed, 19 May 2004 09:09:34 -0400 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1BQQoD-0003Ag-00 for icar@ietf.org; Wed, 19 May 2004 09:08:33 -0400 Received: from wyvern.icir.org ([192.150.187.14]) by ietf-mx with esmtp (Exim 4.12) id 1BQQmx-0002yC-00 for icar@ietf.org; Wed, 19 May 2004 09:07:15 -0400 Received: from guns.icir.org (adsl-68-76-113-50.dsl.bcvloh.ameritech.net [68.76.113.50]) by wyvern.icir.org (8.12.9p1/8.12.8) with ESMTP id i4JD7Bim054313; Wed, 19 May 2004 06:07:12 -0700 (PDT) (envelope-from mallman@guns.icir.org) Received: from guns.icir.org (localhost [127.0.0.1]) by guns.icir.org (Postfix) with ESMTP id F21BA77AA5C; Wed, 19 May 2004 09:07:10 -0400 (EDT) To: Pekka Savola From: Mark Allman Reply-To: mallman@icir.org Cc: icar@ietf.org Subject: Re: [Icar] an early review experiment In-Reply-To: Organization: ICSI Center for Internet Research (ICIR) Song-of-the-Day: Panama MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha1; protocol="application/pgp-signature" Date: Wed, 19 May 2004 09:07:10 -0400 Message-Id: <20040519130710.F21BA77AA5C@guns.icir.org> Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.0 required=5.0 tests=AWL autolearn=no version=2.60 --=-=-= Content-Type: text/plain; charset=us-ascii > > + What does a reviewer do with a review? > > Did you mean 'what does the reviewee (whose document one reviewed) do > with the review?' as I don't understand why the above question is > relevant. No. I was asking where reviews go, how the information generated by the reviewer is fed back to the WG / authors. > The comments should be treated with equal weight as IESG Discuss > comments. This brings up a good point that should have been included on my list of questions. + How binding are early, cross area review comments on authors / editors / WGs? I'd like to hear additional opinions on this point. (And, Pekka's assertion above leads to many questions. I'd rather not rat-hole on those until we figure out what sort of high-level feelings folks have on this question.) Thanks! allman --=-=-= Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.3 (FreeBSD) iD8DBQFAq1v+WyrrWs4yIs4RAmd/AJ9LRgtsy5nZRKfmTCiNSxoyzPpb+ACZAdv3 l8kqRUwO65EghRsIRms8ZPQ= =1jNT -----END PGP SIGNATURE----- --=-=-=-- _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Wed May 19 15:39:58 2004 Received: from optimus.ietf.org (iesg.org [132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA06765 for ; Wed, 19 May 2004 15:39:58 -0400 (EDT) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BQWky-0007ZN-5Y for icar-archive@odin.ietf.org; Wed, 19 May 2004 15:29:36 -0400 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i4JJTaWB029098 for icar-archive@odin.ietf.org; Wed, 19 May 2004 15:29:36 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BQW52-0001vy-C4 for icar-web-archive@optimus.ietf.org; Wed, 19 May 2004 14:46:16 -0400 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA29862 for ; Wed, 19 May 2004 14:46:11 -0400 (EDT) Received: from ietf-mx.ietf.org ([132.151.6.1] helo=ietf-mx) by ietf-mx with esmtp (Exim 4.32) id 1BQW4x-0007kG-Vb for icar-web-archive@ietf.org; Wed, 19 May 2004 14:46:12 -0400 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1BQW42-0007ca-00 for icar-web-archive@ietf.org; Wed, 19 May 2004 14:45:15 -0400 Received: from optimus.ietf.org ([132.151.1.19]) by ietf-mx with esmtp (Exim 4.12) id 1BQW3b-0007Vl-00 for icar-web-archive@ietf.org; Wed, 19 May 2004 14:44:47 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BQVmT-0006CF-WD; Wed, 19 May 2004 14:27:05 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BQSEU-0006p6-5h for icar@optimus.ietf.org; Wed, 19 May 2004 10:39:46 -0400 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id KAA16071 for ; Wed, 19 May 2004 10:39:42 -0400 (EDT) Received: from ietf-mx.ietf.org ([132.151.6.1] helo=ietf-mx) by ietf-mx with esmtp (Exim 4.32) id 1BQSER-000246-LH for icar@ietf.org; Wed, 19 May 2004 10:39:43 -0400 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1BQSDV-0001zI-00 for icar@ietf.org; Wed, 19 May 2004 10:38:46 -0400 Received: from sj-iport-3-in.cisco.com ([171.71.176.72] helo=sj-iport-3.cisco.com) by ietf-mx with esmtp (Exim 4.12) id 1BQSCX-0001rY-00 for icar@ietf.org; Wed, 19 May 2004 10:37:45 -0400 Received: from sj-core-2.cisco.com (171.71.177.254) by sj-iport-3.cisco.com with ESMTP; 19 May 2004 06:44:42 +0000 Received: from mira-sjc5-c.cisco.com (IDENT:mirapoint@mira-sjc5-c.cisco.com [171.71.163.17]) by sj-core-2.cisco.com (8.12.10/8.12.6) with ESMTP id i4JEbDSu011212; Wed, 19 May 2004 07:37:13 -0700 (PDT) Received: from cisco.com ([10.25.65.178]) by mira-sjc5-c.cisco.com (MOS 3.4.5-GR) with SMTP id AUG77350; Wed, 19 May 2004 07:37:12 -0700 (PDT) Date: Wed, 19 May 2004 10:37:07 -0400 Subject: Re: [Icar] an early review experiment Content-Type: text/plain; charset=US-ASCII; format=flowed Mime-Version: 1.0 (Apple Message framework v553) Cc: icar@ietf.org To: mallman@icir.org From: Melinda Shore In-Reply-To: <20040519130710.F21BA77AA5C@guns.icir.org> Message-Id: <00FC40D4-A9A2-11D8-8468-000A95E35274@cisco.com> Content-Transfer-Encoding: 7bit X-Mailer: Apple Mail (2.553) Content-Transfer-Encoding: 7bit Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.0 required=5.0 tests=AWL autolearn=no version=2.60 Content-Transfer-Encoding: 7bit Content-Transfer-Encoding: 7bit On Wednesday, May 19, 2004, at 09:07 AM, Mark Allman wrote: > No. I was asking where reviews go, how the information generated by > the > reviewer is fed back to the WG / authors. I prefer that it go to the wg mailing list - the wg needs to know what's going on, and the process needs to remain as transparent as possible. I think a bigger question is how the completed reviews are managed - whether they're maintained in a database (the draft tracker?), whether or not they should be bundled with the draft when its submitted to the IESG, etc. > + How binding are early, cross area review comments on authors / > editors / WGs? I'm not sure they're *binding*, although they're surely more weighty. Worst case scenario: wg wants to do something inane, early review catches it, wg says "nope, we really intend to do this," inanity gets through wg last call and is submitted to the IESG, the IESG says "this is inane, we're not going to publish it." The wg has had plenty of warning that they're off track, presumably the responsible AD has had the same notice, and it's completely on the shoulders of the wg and the wg chair for having produced inane work. That strikes me as a healthy process. Having reviewers tell the working group what they can and cannot do seems potentially stifling to me. Melinda _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Wed May 19 15:40:27 2004 Received: from optimus.ietf.org (iesg.org [132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA06901 for ; Wed, 19 May 2004 15:40:27 -0400 (EDT) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BQWl7-0007dV-6U for icar-archive@odin.ietf.org; Wed, 19 May 2004 15:29:45 -0400 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i4JJTj6q029342 for icar-archive@odin.ietf.org; Wed, 19 May 2004 15:29:45 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BQW7T-0002kP-Cl for icar-web-archive@optimus.ietf.org; Wed, 19 May 2004 14:48:47 -0400 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA00036 for ; Wed, 19 May 2004 14:48:44 -0400 (EDT) Received: from ietf-mx.ietf.org ([132.151.6.1] helo=ietf-mx) by ietf-mx with esmtp (Exim 4.32) id 1BQW7Q-0000J7-Iu for icar-web-archive@ietf.org; Wed, 19 May 2004 14:48:44 -0400 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1BQW6C-00008F-00 for icar-web-archive@ietf.org; Wed, 19 May 2004 14:47:29 -0400 Received: from optimus.ietf.org ([132.151.1.19]) by ietf-mx with esmtp (Exim 4.12) id 1BQW5W-0007mA-00 for icar-web-archive@ietf.org; Wed, 19 May 2004 14:46:46 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BQVmc-0006EK-Sz; Wed, 19 May 2004 14:27:14 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BQSQA-0000bR-Ia for icar@optimus.ietf.org; Wed, 19 May 2004 10:51:50 -0400 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id KAA16906 for ; Wed, 19 May 2004 10:51:46 -0400 (EDT) Received: from ietf-mx.ietf.org ([132.151.6.1] helo=ietf-mx) by ietf-mx with esmtp (Exim 4.32) id 1BQSQ8-0002wb-2G for icar@ietf.org; Wed, 19 May 2004 10:51:48 -0400 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1BQSPB-0002tS-00 for icar@ietf.org; Wed, 19 May 2004 10:50:49 -0400 Received: from wyvern.icir.org ([192.150.187.14]) by ietf-mx with esmtp (Exim 4.12) id 1BQSOM-0002pg-00 for icar@ietf.org; Wed, 19 May 2004 10:49:58 -0400 Received: from guns.icir.org (adsl-68-76-113-50.dsl.bcvloh.ameritech.net [68.76.113.50]) by wyvern.icir.org (8.12.9p1/8.12.8) with ESMTP id i4JEnpim055572; Wed, 19 May 2004 07:49:52 -0700 (PDT) (envelope-from mallman@guns.icir.org) Received: from guns.icir.org (localhost [127.0.0.1]) by guns.icir.org (Postfix) with ESMTP id EB9D977AA5C; Wed, 19 May 2004 10:49:50 -0400 (EDT) To: Melinda Shore From: Mark Allman Reply-To: mallman@icir.org Cc: icar@ietf.org Subject: Re: [Icar] an early review experiment In-Reply-To: <00FC40D4-A9A2-11D8-8468-000A95E35274@cisco.com> Organization: ICSI Center for Internet Research (ICIR) Song-of-the-Day: Panama MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha1; protocol="application/pgp-signature" Date: Wed, 19 May 2004 10:49:50 -0400 Message-Id: <20040519144950.EB9D977AA5C@guns.icir.org> Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.0 required=5.0 tests=AWL autolearn=no version=2.60 --=-=-= Content-Type: text/plain; charset=us-ascii > > + How binding are early, cross area review comments on authors / > > editors / WGs? > > I'm not sure they're *binding*, although they're surely more weighty. "More weighty" than ... what? If I were to sketch the "weighyness" space of early, cross area reviews, it'd be something like this: + one end of the spectrum would be that ICAR-type comments must be addressed before the document moves forward + the other end of the space seems to me to be that the ICAR-type reviews are just like any other review that comes in from a community member (i.e., no special status) + there may be middle ground It'd be nice to hear thoughts on where folks are in this space. (If I were to interpret what has been said thus far, I'd say Pekka's note falls under the first bullet and Melinda's note under the second. Roughly. If that is way off, of course you should yell because I don't want to put words in anyone's mouth.) Thanks! allman --=-=-= Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.3 (FreeBSD) iD8DBQFAq3QOWyrrWs4yIs4RAhGrAKCIdZ8jjGjGUuooyRHGkWIruMdvRQCfd0jE fmNgDs9EbFt6vcimOnF35jI= =SiY5 -----END PGP SIGNATURE----- --=-=-=-- _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Wed May 19 15:40:28 2004 Received: from optimus.ietf.org (iesg.org [132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA06916 for ; Wed, 19 May 2004 15:40:28 -0400 (EDT) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BQWlH-0007kD-MK for icar-archive@odin.ietf.org; Wed, 19 May 2004 15:29:55 -0400 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i4JJTtIk029763 for icar-archive@odin.ietf.org; Wed, 19 May 2004 15:29:55 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BQW9f-0003Mo-0l for icar-web-archive@optimus.ietf.org; Wed, 19 May 2004 14:51:03 -0400 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA00333 for ; Wed, 19 May 2004 14:50:59 -0400 (EDT) Received: from ietf-mx.ietf.org ([132.151.6.1] helo=ietf-mx) by ietf-mx with esmtp (Exim 4.32) id 1BQW9c-0000hE-6d for icar-web-archive@ietf.org; Wed, 19 May 2004 14:51:00 -0400 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1BQW8p-0000aF-00 for icar-web-archive@ietf.org; Wed, 19 May 2004 14:50:12 -0400 Received: from optimus.ietf.org ([132.151.1.19]) by ietf-mx with esmtp (Exim 4.12) id 1BQW7s-0000QS-00 for icar-web-archive@ietf.org; Wed, 19 May 2004 14:49:12 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BQVn6-0006Oi-3M; Wed, 19 May 2004 14:27:44 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BQSec-0002hf-LM for icar@optimus.ietf.org; Wed, 19 May 2004 11:06:46 -0400 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA18120 for ; Wed, 19 May 2004 11:06:42 -0400 (EDT) Received: from ietf-mx.ietf.org ([132.151.6.1] helo=ietf-mx) by ietf-mx with esmtp (Exim 4.32) id 1BQSeZ-0004Nu-Ty for icar@ietf.org; Wed, 19 May 2004 11:06:44 -0400 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1BQSdd-0004IX-00 for icar@ietf.org; Wed, 19 May 2004 11:05:46 -0400 Received: from sj-iport-3-in.cisco.com ([171.71.176.72] helo=sj-iport-3.cisco.com) by ietf-mx with esmtp (Exim 4.12) id 1BQScg-0004Ai-00 for icar@ietf.org; Wed, 19 May 2004 11:04:46 -0400 Received: from sj-core-5.cisco.com (171.71.177.238) by sj-iport-3.cisco.com with ESMTP; 19 May 2004 07:11:42 +0000 Received: from mira-sjc5-c.cisco.com (IDENT:mirapoint@mira-sjc5-c.cisco.com [171.71.163.17]) by sj-core-5.cisco.com (8.12.10/8.12.6) with ESMTP id i4JF4DaP026406; Wed, 19 May 2004 08:04:14 -0700 (PDT) Received: from cisco.com ([10.25.65.178]) by mira-sjc5-c.cisco.com (MOS 3.4.5-GR) with SMTP id AUG79675; Wed, 19 May 2004 08:04:12 -0700 (PDT) Date: Wed, 19 May 2004 11:04:07 -0400 Subject: Re: [Icar] an early review experiment Content-Type: text/plain; charset=US-ASCII; format=flowed Mime-Version: 1.0 (Apple Message framework v553) Cc: icar@ietf.org To: mallman@icir.org From: Melinda Shore In-Reply-To: <20040519144950.EB9D977AA5C@guns.icir.org> Message-Id: Content-Transfer-Encoding: 7bit X-Mailer: Apple Mail (2.553) Content-Transfer-Encoding: 7bit Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.0 required=5.0 tests=AWL autolearn=no version=2.60 Content-Transfer-Encoding: 7bit Content-Transfer-Encoding: 7bit On Wednesday, May 19, 2004, at 10:49 AM, Mark Allman wrote: > + one end of the spectrum would be that ICAR-type comments must be > addressed before the document moves forward > > + the other end of the space seems to me to be that the ICAR-type > reviews are just like any other review that comes in from a > community member (i.e., no special status) > > + there may be middle ground I'm in the middle ground. It seems to me that what ICAR-type reviews bring to the wg is external context and persective that isn't available within the working group. I wouldn't like to see working group responsibility for the quality of their own product be shifted to non- participants. What I think we need is more checks as documents progress, and while a working group needs to take review comments very seriously, sometimes a reviewer is full of crap, too. I don't think it's the role of this working group (icar) to make fundamental changes to the standardization process and responsibilities. More validity checks on working group documents earlier in the process: great. Changing the process so that reviewers become responsible for working group documents: not so great. And giving reviewers what's essentially veto power over documents gives them responsibility for those documents. Melinda _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Wed May 19 17:32:09 2004 Received: from optimus.ietf.org (iesg.org [132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA19180 for ; Wed, 19 May 2004 17:32:09 -0400 (EDT) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BQYQF-0002fR-SN for icar-archive@odin.ietf.org; Wed, 19 May 2004 17:16:19 -0400 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i4JLGJPD010249 for icar-archive@odin.ietf.org; Wed, 19 May 2004 17:16:19 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BQXVK-0006QT-9m for icar-web-archive@optimus.ietf.org; Wed, 19 May 2004 16:17:30 -0400 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA11238 for ; Wed, 19 May 2004 16:17:27 -0400 (EDT) Received: from ietf-mx.ietf.org ([132.151.6.1] helo=ietf-mx) by ietf-mx with esmtp (Exim 4.32) id 1BQXVI-0006HV-HQ for icar-web-archive@ietf.org; Wed, 19 May 2004 16:17:28 -0400 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1BQXTs-00060f-00 for icar-web-archive@ietf.org; Wed, 19 May 2004 16:16:01 -0400 Received: from optimus.ietf.org ([132.151.1.19]) by ietf-mx with esmtp (Exim 4.12) id 1BQXSm-0005jF-00 for icar-web-archive@ietf.org; Wed, 19 May 2004 16:14:53 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BQXCT-0007hw-Ea; Wed, 19 May 2004 15:58:01 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BQWqA-0000ay-DF for icar@optimus.ietf.org; Wed, 19 May 2004 15:34:58 -0400 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA06074 for ; Wed, 19 May 2004 15:34:56 -0400 (EDT) Received: from ietf-mx.ietf.org ([132.151.6.1] helo=ietf-mx) by ietf-mx with esmtp (Exim 4.32) id 1BQWq8-0007FT-S8 for icar@ietf.org; Wed, 19 May 2004 15:34:56 -0400 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1BQWp9-00079s-00 for icar@ietf.org; Wed, 19 May 2004 15:33:56 -0400 Received: from measurement-factory.com ([206.168.0.5]) by ietf-mx with esmtp (Exim 4.12) id 1BQWoP-00075B-00 for icar@ietf.org; Wed, 19 May 2004 15:33:09 -0400 Received: from measurement-factory.com (localhost [127.0.0.1]) by measurement-factory.com (8.12.9/8.12.9) with ESMTP id i4JJX5p6094086; Wed, 19 May 2004 13:33:05 -0600 (MDT) (envelope-from rousskov@measurement-factory.com) Received: (from rousskov@localhost) by measurement-factory.com (8.12.9/8.12.9/Submit) id i4JJX5UL094085; Wed, 19 May 2004 13:33:05 -0600 (MDT) (envelope-from rousskov) Date: Wed, 19 May 2004 13:33:05 -0600 (MDT) From: Alex Rousskov To: Mark Allman cc: icar@ietf.org Subject: Re: [Icar] an early review experiment In-Reply-To: <20040519144950.EB9D977AA5C@guns.icir.org> Message-ID: References: <20040519144950.EB9D977AA5C@guns.icir.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.0 required=5.0 tests=AWL autolearn=no version=2.60 On Wed, 19 May 2004, Mark Allman wrote: > If I were to sketch the "weighyness" space of early, cross area > reviews, it'd be something like this: > > + one end of the spectrum would be that ICAR-type comments must be > addressed before the document moves forward > > + the other end of the space seems to me to be that the ICAR-type > reviews are just like any other review that comes in from a > community member (i.e., no special status) > > + there may be middle ground > > It'd be nice to hear thoughts on where folks are in this space. IMO, WG MUST publicly address all reviewer concerns, but can do so any way it wants. WG MAY reject to act on any comment, for any reason, but MUST publicly document that the comment is rejected and SHOULD document rejection rationale. IESG SHOULD NOT review documents with unaddressed concerns. For these rules to be meaningful/effective, there has to be a mechanism/tool to register specific concerns and WG responses to them (e.g., ID Tracker DISCUSS items on steroids). Just exchanging freeform e-mails is not good enough. It seems to me that same rules should apply to any review, regardless of its source. There may be periods when WG does not accept reviews for specific documents though (because they are not ready for any kind of review or because the WG has finished working on them and wants IESG to take over). One simple way to manage review timings is to have both First Call and Last Call for each draft (at least). These calls will make it clear when reviews are welcome (and when reviewer comments are virtually guaranteed to be addressed). I do not know whether the above fits any of the three bullets you proposed. It seems lile the second bullet because all reviews have the same status (but I would call that status "special" compared to the current "no special status" situation), but it also matches the intent of the first bullet as far as final IESG action is concerned. HTH, Alex. _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Wed May 19 17:58:11 2004 Received: from optimus.ietf.org (iesg.org [132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA22399 for ; Wed, 19 May 2004 17:58:11 -0400 (EDT) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BQZ1n-0001f4-OH for icar-archive@odin.ietf.org; Wed, 19 May 2004 17:55:09 -0400 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i4JLt70f006384 for icar-archive@odin.ietf.org; Wed, 19 May 2004 17:55:07 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BQYiw-0003r9-25 for icar-web-archive@optimus.ietf.org; Wed, 19 May 2004 17:35:38 -0400 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA19561 for ; Wed, 19 May 2004 17:35:34 -0400 (EDT) Received: from ietf-mx.ietf.org ([132.151.6.1] helo=ietf-mx) by ietf-mx with esmtp (Exim 4.32) id 1BQYit-0003xu-IP for icar-web-archive@ietf.org; Wed, 19 May 2004 17:35:35 -0400 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1BQYeD-0003I6-00 for icar-web-archive@ietf.org; Wed, 19 May 2004 17:30:47 -0400 Received: from optimus.ietf.org ([132.151.1.19]) by ietf-mx with esmtp (Exim 4.12) id 1BQYc4-0002oO-00 for icar-web-archive@ietf.org; Wed, 19 May 2004 17:28:32 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BQY5e-00033S-Ey; Wed, 19 May 2004 16:55:02 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BQXKL-0001vG-3O for icar@optimus.ietf.org; Wed, 19 May 2004 16:06:09 -0400 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA10213 for ; Wed, 19 May 2004 16:06:06 -0400 (EDT) Received: from ietf-mx.ietf.org ([132.151.6.1] helo=ietf-mx) by ietf-mx with esmtp (Exim 4.32) id 1BQXKJ-0004Tq-9x for icar@ietf.org; Wed, 19 May 2004 16:06:07 -0400 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1BQXJW-0004NA-00 for icar@ietf.org; Wed, 19 May 2004 16:05:18 -0400 Received: from eikenes.alvestrand.no ([158.38.152.233]) by ietf-mx with esmtp (Exim 4.12) id 1BQXId-00049A-00 for icar@ietf.org; Wed, 19 May 2004 16:04:23 -0400 Received: from localhost (localhost.localdomain [127.0.0.1]) by eikenes.alvestrand.no (Postfix) with ESMTP id A538F622A6; Wed, 19 May 2004 22:03:52 +0200 (CEST) Received: from eikenes.alvestrand.no ([127.0.0.1]) by localhost (eikenes.alvestrand.no [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 01019-06; Wed, 19 May 2004 22:03:52 +0200 (CEST) Received: from halvestr-w2k02.emea.cisco.com (localhost.localdomain [127.0.0.1]) by eikenes.alvestrand.no (Postfix) with ESMTP id 95614621CB; Wed, 19 May 2004 22:03:49 +0200 (CEST) Date: Wed, 19 May 2004 12:39:27 -0700 From: Harald Tveit Alvestrand To: mallman@icir.org, Pekka Savola Cc: icar@ietf.org Subject: Re: [Icar] an early review experiment Message-ID: <183112141.1084970367@localhost> In-Reply-To: <20040519130710.F21BA77AA5C@guns.icir.org> References: <20040519130710.F21BA77AA5C@guns.icir.org> X-Mailer: Mulberry/3.1.0 (Win32) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline X-Virus-Scanned: by amavisd-new at alvestrand.no Content-Transfer-Encoding: 7bit Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.0 required=5.0 tests=AWL autolearn=no version=2.60 Content-Transfer-Encoding: 7bit Content-Transfer-Encoding: 7bit --On 19. mai 2004 09:07 -0400 Mark Allman wrote: > + How binding are early, cross area review comments on authors / > editors / WGs? > > I'd like to hear additional opinions on this point. having been through some 70 reviews in the Gen-ART group, I know very well that some review comments are based on misunderstandings, and some only point out that the point needs to be better explained... I think that it's vital that review comments be *addressed*, but frankly, sometimes the WG or the author will be *right*! (I sometimes miss more intensive arguments from a WG against an AD's opinions... I should be careful what I wish for :-) _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Wed May 19 17:58:42 2004 Received: from optimus.ietf.org (iesg.org [132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA22447 for ; Wed, 19 May 2004 17:58:42 -0400 (EDT) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BQZ2C-0001ql-Tm for icar-archive@odin.ietf.org; Wed, 19 May 2004 17:55:32 -0400 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i4JLtWTi007107 for icar-archive@odin.ietf.org; Wed, 19 May 2004 17:55:32 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BQYmm-00052p-PR for icar-web-archive@optimus.ietf.org; Wed, 19 May 2004 17:39:36 -0400 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA20301 for ; Wed, 19 May 2004 17:39:33 -0400 (EDT) Received: from ietf-mx.ietf.org ([132.151.6.1] helo=ietf-mx) by ietf-mx with esmtp (Exim 4.32) id 1BQYmk-0004kP-1v for icar-web-archive@ietf.org; Wed, 19 May 2004 17:39:34 -0400 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1BQYjT-00043j-00 for icar-web-archive@ietf.org; Wed, 19 May 2004 17:36:12 -0400 Received: from optimus.ietf.org ([132.151.1.19]) by ietf-mx with esmtp (Exim 4.12) id 1BQYed-0003N6-00 for icar-web-archive@ietf.org; Wed, 19 May 2004 17:31:12 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BQYJR-0000kA-20; Wed, 19 May 2004 17:09:17 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BQXOK-0003Ji-AM for icar@optimus.ietf.org; Wed, 19 May 2004 16:10:16 -0400 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA10508 for ; Wed, 19 May 2004 16:10:13 -0400 (EDT) Received: from ietf-mx.ietf.org ([132.151.6.1] helo=ietf-mx) by ietf-mx with esmtp (Exim 4.32) id 1BQXOI-00052a-Cf for icar@ietf.org; Wed, 19 May 2004 16:10:14 -0400 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1BQXNV-0004tq-00 for icar@ietf.org; Wed, 19 May 2004 16:09:25 -0400 Received: from wyvern.icir.org ([192.150.187.14]) by ietf-mx with esmtp (Exim 4.12) id 1BQXMO-0004jf-00 for icar@ietf.org; Wed, 19 May 2004 16:08:17 -0400 Received: from guns.icir.org (adsl-68-76-113-50.dsl.bcvloh.ameritech.net [68.76.113.50]) by wyvern.icir.org (8.12.9p1/8.12.8) with ESMTP id i4JK8Eim061539 for ; Wed, 19 May 2004 13:08:14 -0700 (PDT) (envelope-from mallman@guns.icir.org) Received: from guns.icir.org (localhost [127.0.0.1]) by guns.icir.org (Postfix) with ESMTP id 9C98277AA5C for ; Wed, 19 May 2004 16:08:12 -0400 (EDT) To: icar@ietf.org From: Mark Allman Reply-To: mallman@icir.org Subject: Re: [Icar] an early review experiment Organization: ICSI Center for Internet Research (ICIR) Song-of-the-Day: Panama MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha1; protocol="application/pgp-signature" Date: Wed, 19 May 2004 16:08:12 -0400 Message-Id: <20040519200812.9C98277AA5C@guns.icir.org> Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.0 required=5.0 tests=AWL autolearn=no version=2.60 --=-=-= Content-Type: text/plain; charset=us-ascii (I should not be allowed to write email.) > If I were to sketch the "weighyness" space of early, cross area reviews, > it'd be something like this: > > + one end of the spectrum would be that ICAR-type comments must be > addressed before the document moves forward > > + the other end of the space seems to me to be that the ICAR-type > reviews are just like any other review that comes in from a > community member (i.e., no special status) > > + there may be middle ground > > It'd be nice to hear thoughts on where folks are in this space. Let me revise the first bullet above ... + one end of the spectrum would be that ICAR-type comments must be followed before the document moves forward (i.e., these comments are binding on the WG and the WG must fix the document accordingly) allman --=-=-= Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.3 (FreeBSD) iD8DBQFAq76sWyrrWs4yIs4RAnNWAJ9YIVqz/qQcuuphhNT5D4lULj4suQCgibcl EvSTxrhGzv30jhIC3rvO76A= =1WS2 -----END PGP SIGNATURE----- --=-=-=-- _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Wed May 19 17:59:08 2004 Received: from optimus.ietf.org (iesg.org [132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA22505 for ; Wed, 19 May 2004 17:59:08 -0400 (EDT) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BQZ2K-0001zV-Hn for icar-archive@odin.ietf.org; Wed, 19 May 2004 17:55:40 -0400 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i4JLteFF007648 for icar-archive@odin.ietf.org; Wed, 19 May 2004 17:55:40 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BQYnG-00056u-DM for icar-web-archive@optimus.ietf.org; Wed, 19 May 2004 17:40:06 -0400 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA20470 for ; Wed, 19 May 2004 17:40:02 -0400 (EDT) Received: from ietf-mx.ietf.org ([132.151.6.1] helo=ietf-mx) by ietf-mx with esmtp (Exim 4.32) id 1BQYnD-0004qg-Hs for icar-web-archive@ietf.org; Wed, 19 May 2004 17:40:03 -0400 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1BQYkQ-0004Da-00 for icar-web-archive@ietf.org; Wed, 19 May 2004 17:37:11 -0400 Received: from optimus.ietf.org ([132.151.1.19]) by ietf-mx with esmtp (Exim 4.12) id 1BQYfL-0003TK-00 for icar-web-archive@ietf.org; Wed, 19 May 2004 17:31:55 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BQYRJ-0002yf-9C; Wed, 19 May 2004 17:17:25 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BQXXe-0006jr-A5 for icar@optimus.ietf.org; Wed, 19 May 2004 16:19:54 -0400 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA11614 for ; Wed, 19 May 2004 16:19:51 -0400 (EDT) Received: from ietf-mx.ietf.org ([132.151.6.1] helo=ietf-mx) by ietf-mx with esmtp (Exim 4.32) id 1BQXXc-0006jy-G4 for icar@ietf.org; Wed, 19 May 2004 16:19:52 -0400 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1BQXWd-0006Wh-00 for icar@ietf.org; Wed, 19 May 2004 16:18:51 -0400 Received: from wyvern.icir.org ([192.150.187.14]) by ietf-mx with esmtp (Exim 4.12) id 1BQXVB-0006Fy-00 for icar@ietf.org; Wed, 19 May 2004 16:17:21 -0400 Received: from guns.icir.org (adsl-68-76-113-50.dsl.bcvloh.ameritech.net [68.76.113.50]) by wyvern.icir.org (8.12.9p1/8.12.8) with ESMTP id i4JKHJim061673; Wed, 19 May 2004 13:17:19 -0700 (PDT) (envelope-from mallman@guns.icir.org) Received: from guns.icir.org (localhost [127.0.0.1]) by guns.icir.org (Postfix) with ESMTP id D137477AA5C; Wed, 19 May 2004 16:17:17 -0400 (EDT) To: Alex Rousskov From: Mark Allman Reply-To: mallman@icir.org Cc: icar@ietf.org Subject: Re: [Icar] an early review experiment In-Reply-To: Organization: ICSI Center for Internet Research (ICIR) Song-of-the-Day: Panama MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha1; protocol="application/pgp-signature" Date: Wed, 19 May 2004 16:17:17 -0400 Message-Id: <20040519201717.D137477AA5C@guns.icir.org> Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.0 required=5.0 tests=AWL autolearn=no version=2.60 --=-=-= Content-Type: text/plain; charset=us-ascii I want to take a tangent for a moment... > IMO, WG MUST publicly address all reviewer concerns, but can do so any > way it wants. WG MAY reject to act on any comment, for any reason, but > MUST publicly document that the comment is rejected and SHOULD > document rejection rationale. I'd like to hear whether folk's thoughts WGs completely ignoring comments. I understand that it *could* be a problem and that it could be aggivated by a new early review scheme. But, I am wondering whether folks think WGs currently ignore comments they do not like without comment. This is different than having a discussion and then not following the given comments. I am talking about comments that are sent and then completely ignored -- no discussion, no response, nothing .... And, this is also different from any particular machinery (e.g., filing the comments in a proper issue tracker, etc.). Thanks! allman -- Mark Allman -- ICIR -- http://www.icir.org/mallman/ --=-=-= Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.3 (FreeBSD) iD8DBQFAq8DNWyrrWs4yIs4RAszBAJ9cRFyWpOlXzMmtcLSYO0i29yjYFACeJyxh SG3wTZMdK/uwSS5wLJzCpwY= =pnU/ -----END PGP SIGNATURE----- --=-=-=-- _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Wed May 19 17:59:39 2004 Received: from optimus.ietf.org (iesg.org [132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA22592 for ; Wed, 19 May 2004 17:59:38 -0400 (EDT) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BQZ2L-00020x-UE for icar-archive@odin.ietf.org; Wed, 19 May 2004 17:55:41 -0400 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i4JLtfUR007739 for icar-archive@odin.ietf.org; Wed, 19 May 2004 17:55:41 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BQYnS-00058H-Dn for icar-web-archive@optimus.ietf.org; Wed, 19 May 2004 17:40:18 -0400 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA20531 for ; Wed, 19 May 2004 17:40:14 -0400 (EDT) Received: from ietf-mx.ietf.org ([132.151.6.1] helo=ietf-mx) by ietf-mx with esmtp (Exim 4.32) id 1BQYnP-0004tA-TT for icar-web-archive@ietf.org; Wed, 19 May 2004 17:40:16 -0400 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1BQYks-0004Gh-00 for icar-web-archive@ietf.org; Wed, 19 May 2004 17:37:38 -0400 Received: from optimus.ietf.org ([132.151.1.19]) by ietf-mx with esmtp (Exim 4.12) id 1BQYfs-0003Zl-00 for icar-web-archive@ietf.org; Wed, 19 May 2004 17:32:28 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BQYWi-00053x-TM; Wed, 19 May 2004 17:23:00 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BQXbi-0007yF-S2 for icar@optimus.ietf.org; Wed, 19 May 2004 16:24:06 -0400 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA11852 for ; Wed, 19 May 2004 16:24:04 -0400 (EDT) Received: from ietf-mx.ietf.org ([132.151.6.1] helo=ietf-mx) by ietf-mx with esmtp (Exim 4.32) id 1BQXbg-0007Fg-Ui for icar@ietf.org; Wed, 19 May 2004 16:24:05 -0400 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1BQXak-00079W-00 for icar@ietf.org; Wed, 19 May 2004 16:23:06 -0400 Received: from netcore.fi ([193.94.160.1]) by ietf-mx with esmtp (Exim 4.12) id 1BQXaN-00073i-00 for icar@ietf.org; Wed, 19 May 2004 16:22:44 -0400 Received: from localhost (pekkas@localhost) by netcore.fi (8.11.6/8.11.6) with ESMTP id i4JKLwP23621; Wed, 19 May 2004 23:21:58 +0300 Date: Wed, 19 May 2004 23:21:58 +0300 (EEST) From: Pekka Savola To: Harald Tveit Alvestrand cc: mallman@icir.org, Subject: Re: [Icar] an early review experiment In-Reply-To: <183112141.1084970367@localhost> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.0 required=5.0 tests=AWL autolearn=no version=2.60 On Wed, 19 May 2004, Harald Tveit Alvestrand wrote: > --On 19. mai 2004 09:07 -0400 Mark Allman wrote: > > + How binding are early, cross area review comments on authors / > > editors / WGs? > > > > I'd like to hear additional opinions on this point. > > having been through some 70 reviews in the Gen-ART group, I know very well > that some review comments are based on misunderstandings, and some only > point out that the point needs to be better explained... I think that it's > vital that review comments be *addressed*, but frankly, sometimes the WG or > the author will be *right*! What Harald said. IMO it's a requreirement that the comments have to be addressed. If you think the comment is off base, send a mail to _discuss_ why you think so. If the person agrees, the comment is addressed, no need to change anything. Or add something else wherever to clarify. If you can't reach consensus (e.g., you think the suggestion is bad one, but the review will not relent), bigger guns must come to the play to judge the situation, e.g., the appropriate ADs, or whatever. The whole point of early cross-area review would be useless if the comments could be ignored on whim. And on the other hand, if the commenters are not sufficient good judges of what's a critical issue and what's not, we'd have an equally big problem. -- Pekka Savola "You each name yourselves king, yet the Netcore Oy kingdom bleeds." Systems. Networks. Security. -- George R.R. Martin: A Clash of Kings _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Wed May 19 17:59:42 2004 Received: from optimus.ietf.org (iesg.org [132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA22610 for ; Wed, 19 May 2004 17:59:42 -0400 (EDT) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BQZ2M-00021w-NL for icar-archive@odin.ietf.org; Wed, 19 May 2004 17:55:42 -0400 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i4JLtgFd007799 for icar-archive@odin.ietf.org; Wed, 19 May 2004 17:55:42 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BQYnb-00059j-3J for icar-web-archive@optimus.ietf.org; Wed, 19 May 2004 17:40:27 -0400 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA20563 for ; Wed, 19 May 2004 17:40:23 -0400 (EDT) Received: from ietf-mx.ietf.org ([132.151.6.1] helo=ietf-mx) by ietf-mx with esmtp (Exim 4.32) id 1BQYnY-0004uZ-MF for icar-web-archive@ietf.org; Wed, 19 May 2004 17:40:24 -0400 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1BQYlK-0004Oa-00 for icar-web-archive@ietf.org; Wed, 19 May 2004 17:38:07 -0400 Received: from optimus.ietf.org ([132.151.1.19]) by ietf-mx with esmtp (Exim 4.12) id 1BQYge-0003e2-00 for icar-web-archive@ietf.org; Wed, 19 May 2004 17:33:16 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BQYaa-0006UW-Sy; Wed, 19 May 2004 17:27:00 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BQXce-0008Ml-0W for icar@optimus.ietf.org; Wed, 19 May 2004 16:25:04 -0400 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA11868 for ; Wed, 19 May 2004 16:24:55 -0400 (EDT) Received: from ietf-mx.ietf.org ([132.151.6.1] helo=ietf-mx) by ietf-mx with esmtp (Exim 4.32) id 1BQXcW-0007Lc-AL for icar@ietf.org; Wed, 19 May 2004 16:24:56 -0400 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1BQXbW-0007EX-00 for icar@ietf.org; Wed, 19 May 2004 16:23:55 -0400 Received: from ns.execdsl.net ([208.184.15.238] helo=EXECDSL.COM) by ietf-mx with esmtp (Exim 4.12) id 1BQXaX-000743-00 for icar@ietf.org; Wed, 19 May 2004 16:22:54 -0400 Received: from [64.254.114.114] (HELO JLaptop.stevecrocker.com) by EXECDSL.COM (CommuniGate Pro SMTP 3.3) with ESMTP id 6969256; Wed, 19 May 2004 16:22:20 -0400 Message-Id: <5.1.0.14.0.20040519161941.0241a008@localhost> X-Sender: joel@stevecrocker.com@localhost X-Mailer: QUALCOMM Windows Eudora Version 5.1 Date: Wed, 19 May 2004 16:22:02 -0400 To: icar@ietf.org From: "Joel M. Halpern" Subject: Re: [Icar] an early review experiment Cc: Alex Rousskov In-Reply-To: References: <20040519144950.EB9D977AA5C@guns.icir.org> <20040519144950.EB9D977AA5C@guns.icir.org> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.0 required=5.0 tests=AWL autolearn=no version=2.60 Most of these comments, while fair concerns, seem to be primarily addressed to the general question of how working groups manage comments. General working group process is probably outside of the scope of this working group. Yours, Joel M. Halpern PS: To be specific, I do not think that the ICAR working group can direct that working groups should (much less must) have problem trackers, and should (or must) address all comments received, regardless of source. Even though those may be good ideas. At 01:33 PM 5/19/2004 -0600, Alex Rousskov wrote: >On Wed, 19 May 2004, Mark Allman wrote: > > > If I were to sketch the "weighyness" space of early, cross area > > reviews, it'd be something like this: > > > > + one end of the spectrum would be that ICAR-type comments must be > > addressed before the document moves forward > > > > + the other end of the space seems to me to be that the ICAR-type > > reviews are just like any other review that comes in from a > > community member (i.e., no special status) > > > > + there may be middle ground > > > > It'd be nice to hear thoughts on where folks are in this space. > >IMO, WG MUST publicly address all reviewer concerns, but can do so any >way it wants. WG MAY reject to act on any comment, for any reason, but >MUST publicly document that the comment is rejected and SHOULD >document rejection rationale. IESG SHOULD NOT review documents with >unaddressed concerns. > >For these rules to be meaningful/effective, there has to be a >mechanism/tool to register specific concerns and WG responses to them >(e.g., ID Tracker DISCUSS items on steroids). Just exchanging freeform >e-mails is not good enough. > >It seems to me that same rules should apply to any review, regardless >of its source. There may be periods when WG does not accept reviews >for specific documents though (because they are not ready for any kind >of review or because the WG has finished working on them and wants >IESG to take over). > >One simple way to manage review timings is to have both First Call and >Last Call for each draft (at least). These calls will make it clear >when reviews are welcome (and when reviewer comments are virtually >guaranteed to be addressed). > >I do not know whether the above fits any of the three bullets you >proposed. It seems lile the second bullet because all reviews have the >same status (but I would call that status "special" compared to the >current "no special status" situation), but it also matches the intent >of the first bullet as far as final IESG action is concerned. > >HTH, > >Alex. > > >_______________________________________________ >Icar mailing list >Icar@ietf.org >https://www1.ietf.org/mailman/listinfo/icar _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Wed May 19 18:00:07 2004 Received: from optimus.ietf.org (iesg.org [132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id SAA22697 for ; Wed, 19 May 2004 18:00:07 -0400 (EDT) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BQZ2N-00022t-GI for icar-archive@odin.ietf.org; Wed, 19 May 2004 17:55:43 -0400 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i4JLthPr007858 for icar-archive@odin.ietf.org; Wed, 19 May 2004 17:55:43 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BQYnq-0005Gw-DZ for icar-web-archive@optimus.ietf.org; Wed, 19 May 2004 17:40:42 -0400 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA20605 for ; Wed, 19 May 2004 17:40:38 -0400 (EDT) Received: from ietf-mx.ietf.org ([132.151.6.1] helo=ietf-mx) by ietf-mx with esmtp (Exim 4.32) id 1BQYnn-0004wW-Tp for icar-web-archive@ietf.org; Wed, 19 May 2004 17:40:40 -0400 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1BQYlh-0004TE-00 for icar-web-archive@ietf.org; Wed, 19 May 2004 17:38:30 -0400 Received: from optimus.ietf.org ([132.151.1.19]) by ietf-mx with esmtp (Exim 4.12) id 1BQYhc-0003il-00 for icar-web-archive@ietf.org; Wed, 19 May 2004 17:34:16 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BQYbd-0007Zu-1l; Wed, 19 May 2004 17:28:05 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BQXje-0001su-Ee for icar@optimus.ietf.org; Wed, 19 May 2004 16:32:18 -0400 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA12400 for ; Wed, 19 May 2004 16:32:15 -0400 (EDT) Received: from ietf-mx.ietf.org ([132.151.6.1] helo=ietf-mx) by ietf-mx with esmtp (Exim 4.32) id 1BQXjc-0000Tx-Bw for icar@ietf.org; Wed, 19 May 2004 16:32:16 -0400 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1BQXgC-00005i-00 for icar@ietf.org; Wed, 19 May 2004 16:28:45 -0400 Received: from psg.com ([147.28.0.62] ident=mailnull) by ietf-mx with esmtp (Exim 4.12) id 1BQXer-0007gP-00 for icar@ietf.org; Wed, 19 May 2004 16:27:21 -0400 Received: from localhost ([127.0.0.1] helo=psg.com) by psg.com with esmtp (Exim 4.30; FreeBSD) id 1BQXep-000N45-U4; Wed, 19 May 2004 20:27:19 +0000 To: Mark Allman cc: icar@ietf.org Subject: Re: [Icar] an early review experiment In-Reply-To: Message from Mark Allman of "Wed, 19 May 2004 09:07:10 EDT." <20040519130710.F21BA77AA5C@guns.icir.org> Date: Wed, 19 May 2004 13:27:19 -0700 From: Allison Mankin Message-Id: Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.0 required=5.0 tests=AWL autolearn=no version=2.60 Hi, Mark, Views on both of these: > > + What does a reviewer do with a review? I think reviews which are part of the ICAR process need to be public. I agree with Melinda's reasoning. In addition, the working group can get better at cross-area thinking, which can happen best if the review's in the open. > > + How binding are early, cross area review comments on authors / > editors / WGs? > > + one end of the spectrum would be that ICAR-type comments must be > addressed before the document moves forward > > + the other end of the space seems to me to be that the ICAR-type > reviews are just like any other review that comes in from a > community member (i.e., no special status) I'm in the middle, for now. You started out asking for this to be a discussion of an experiment. After we experiment, my position could change. I'd like to see all reviews result in some kind of response about how comments are addressed. ICAR-type comments should require* that this response be prepared and sent to the mailing list. But if the document does not change at all in response to the review, that is OK: the document should not block (that is, the Chair's, WG's and AD's discretion with respect to the document should not change). The ICAR review still has clout: from the enforced response, the Chair should lead a discussion, and the result should be a serious working group consideration of the review. In my experience of quite a few AD reviewss (and those of AD's directorates), there's a steep learning curve for cross-area review, and also a difficult line between when something is actually a working group consensus matter rather than a cross-area matter. My experience is that the IESG learns how best to cross-area review from reviewing each others' reviews and from reviewing hundreds of documents in all the areas. There are certainlly other ways to get there, but we have no handle on this now. Bottom line: I think it's important to see how well the IETF can produce consistent cross-area reviewers, which includes developing the shared and concrete view of what the cross-area field is. For this experiment, I'm against blocking documents Allison. _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Wed May 19 18:00:13 2004 Received: from optimus.ietf.org (iesg.org [132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id SAA22753 for ; Wed, 19 May 2004 18:00:12 -0400 (EDT) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BQZ2T-00027K-PH for icar-archive@odin.ietf.org; Wed, 19 May 2004 17:55:49 -0400 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i4JLtnpZ008130 for icar-archive@odin.ietf.org; Wed, 19 May 2004 17:55:49 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BQYpB-00067J-AS for icar-web-archive@optimus.ietf.org; Wed, 19 May 2004 17:42:05 -0400 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA20764 for ; Wed, 19 May 2004 17:42:01 -0400 (EDT) Received: from ietf-mx.ietf.org ([132.151.6.1] helo=ietf-mx) by ietf-mx with esmtp (Exim 4.32) id 1BQYp8-0005Ii-HQ for icar-web-archive@ietf.org; Wed, 19 May 2004 17:42:02 -0400 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1BQYo7-00053R-00 for icar-web-archive@ietf.org; Wed, 19 May 2004 17:40:59 -0400 Received: from optimus.ietf.org ([132.151.1.19]) by ietf-mx with esmtp (Exim 4.12) id 1BQYm8-0004ZM-00 for icar-web-archive@ietf.org; Wed, 19 May 2004 17:38:56 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BQYcC-0008GZ-TQ; Wed, 19 May 2004 17:28:40 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BQXpt-0005SA-TU for icar@optimus.ietf.org; Wed, 19 May 2004 16:38:45 -0400 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA13354 for ; Wed, 19 May 2004 16:38:41 -0400 (EDT) Received: from ietf-mx.ietf.org ([132.151.6.1] helo=ietf-mx) by ietf-mx with esmtp (Exim 4.32) id 1BQXpq-0001pH-IY for icar@ietf.org; Wed, 19 May 2004 16:38:42 -0400 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1BQXoI-0001Vn-00 for icar@ietf.org; Wed, 19 May 2004 16:37:08 -0400 Received: from measurement-factory.com ([206.168.0.5]) by ietf-mx with esmtp (Exim 4.12) id 1BQXmk-000190-00 for icar@ietf.org; Wed, 19 May 2004 16:35:31 -0400 Received: from measurement-factory.com (localhost [127.0.0.1]) by measurement-factory.com (8.12.9/8.12.9) with ESMTP id i4JKZTp6097305; Wed, 19 May 2004 14:35:29 -0600 (MDT) (envelope-from rousskov@measurement-factory.com) Received: (from rousskov@localhost) by measurement-factory.com (8.12.9/8.12.9/Submit) id i4JKZT0C097304; Wed, 19 May 2004 14:35:29 -0600 (MDT) (envelope-from rousskov) Date: Wed, 19 May 2004 14:35:28 -0600 (MDT) From: Alex Rousskov To: Mark Allman cc: icar@ietf.org Subject: Re: [Icar] an early review experiment In-Reply-To: <20040519201717.D137477AA5C@guns.icir.org> Message-ID: References: <20040519201717.D137477AA5C@guns.icir.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.0 required=5.0 tests=AWL autolearn=no version=2.60 On Wed, 19 May 2004, Mark Allman wrote: > I am wondering whether folks think WGs currently ignore comments > they do not like without comment. This is different than having a > discussion and then not following the given comments. In my limited experience, I have seen enough cases where my comments were not addressed. Thus, I suspect that there are lots of cases like that, especially in "controversial" areas. Please note that I personally do not think there is an important difference between "discussing and then ignoring" a comment and "ignoring without discussion" (as far as the scope of your questions goes). The result is the same -- the comment is not addressed. Without a concise statement of action, informal and fuzzy mailing list discussions mean little if anything. > And, this is also different from any particular machinery (e.g., > filing the comments in a proper issue tracker, etc.). While there is certainly a theoretical difference, in practice, it makes little sense to ask whether citizens are usually law-obeying if the country does not keep a record of law violations or, worse, does not define what a violation is. You will only hear about high-profile murders and neighbor disputes which are unlikely to give enough quality information about the general state of affairs. Alex. _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Wed May 19 18:00:42 2004 Received: from optimus.ietf.org (iesg.org [132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id SAA22825 for ; Wed, 19 May 2004 18:00:42 -0400 (EDT) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BQZ2h-0002EE-5e for icar-archive@odin.ietf.org; Wed, 19 May 2004 17:56:03 -0400 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i4JLu33c008560 for icar-archive@odin.ietf.org; Wed, 19 May 2004 17:56:03 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BQYsm-0007Lg-7s for icar-web-archive@optimus.ietf.org; Wed, 19 May 2004 17:45:48 -0400 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA21421 for ; Wed, 19 May 2004 17:45:44 -0400 (EDT) Received: from ietf-mx.ietf.org ([132.151.6.1] helo=ietf-mx) by ietf-mx with esmtp (Exim 4.32) id 1BQYsj-0005vD-L9 for icar-web-archive@ietf.org; Wed, 19 May 2004 17:45:45 -0400 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1BQYre-0005kg-00 for icar-web-archive@ietf.org; Wed, 19 May 2004 17:44:38 -0400 Received: from optimus.ietf.org ([132.151.1.19]) by ietf-mx with esmtp (Exim 4.12) id 1BQYqa-0005Y1-00 for icar-web-archive@ietf.org; Wed, 19 May 2004 17:43:32 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BQYeW-0001bA-5G; Wed, 19 May 2004 17:31:04 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BQXvY-0007oz-NZ for icar@optimus.ietf.org; Wed, 19 May 2004 16:44:36 -0400 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA13997 for ; Wed, 19 May 2004 16:44:33 -0400 (EDT) Received: from ietf-mx.ietf.org ([132.151.6.1] helo=ietf-mx) by ietf-mx with esmtp (Exim 4.32) id 1BQXvW-0002rl-K6 for icar@ietf.org; Wed, 19 May 2004 16:44:34 -0400 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1BQXuZ-0002hg-00 for icar@ietf.org; Wed, 19 May 2004 16:43:35 -0400 Received: from measurement-factory.com ([206.168.0.5]) by ietf-mx with esmtp (Exim 4.12) id 1BQXtm-0002XM-00 for icar@ietf.org; Wed, 19 May 2004 16:42:46 -0400 Received: from measurement-factory.com (localhost [127.0.0.1]) by measurement-factory.com (8.12.9/8.12.9) with ESMTP id i4JKgkp6097838; Wed, 19 May 2004 14:42:46 -0600 (MDT) (envelope-from rousskov@measurement-factory.com) Received: (from rousskov@localhost) by measurement-factory.com (8.12.9/8.12.9/Submit) id i4JKgkjS097837; Wed, 19 May 2004 14:42:46 -0600 (MDT) (envelope-from rousskov) Date: Wed, 19 May 2004 14:42:46 -0600 (MDT) From: Alex Rousskov To: "Joel M. Halpern" cc: icar@ietf.org In-Reply-To: <5.1.0.14.0.20040519161941.0241a008@localhost> Message-ID: References: <20040519144950.EB9D977AA5C@guns.icir.org> <20040519144950.EB9D977AA5C@guns.icir.org> <5.1.0.14.0.20040519161941.0241a008@localhost> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Subject: [Icar] ICAR charter scope and review management Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.0 required=5.0 tests=AWL autolearn=no version=2.60 On Wed, 19 May 2004, Joel M. Halpern wrote: > Most of these comments, while fair concerns, seem to be primarily > addressed to the general question of how working groups manage > comments. General working group process is probably outside of the > scope of this working group. The charter seems to place any mechanism that encourages review of documents in scope. There is probably nothing more discouraging than seeing your comments ignored! > PS: To be specific, I do not think that the ICAR working group can > direct that working groups should (much less must) have problem > trackers, and should (or must) address all comments received, > regardless of source. I do not see why not. The charter does not seem to place such essential mechanisms out of scope. Am I missing something? Thanks, Alex. _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Wed May 19 18:28:14 2004 Received: from optimus.ietf.org (iesg.org [132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id SAA25128 for ; Wed, 19 May 2004 18:28:13 -0400 (EDT) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BQZQ7-0007uY-71 for icar-archive@odin.ietf.org; Wed, 19 May 2004 18:20:15 -0400 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i4JMKFpb030392 for icar-archive@odin.ietf.org; Wed, 19 May 2004 18:20:15 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BQZEV-0000is-Bi for icar-web-archive@optimus.ietf.org; Wed, 19 May 2004 18:08:15 -0400 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id SAA23576 for ; Wed, 19 May 2004 18:08:11 -0400 (EDT) Received: from ietf-mx.ietf.org ([132.151.6.1] helo=ietf-mx) by ietf-mx with esmtp (Exim 4.32) id 1BQZES-0001CR-He for icar-web-archive@ietf.org; Wed, 19 May 2004 18:08:12 -0400 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1BQZDg-00015d-00 for icar-web-archive@ietf.org; Wed, 19 May 2004 18:07:24 -0400 Received: from optimus.ietf.org ([132.151.1.19]) by ietf-mx with esmtp (Exim 4.12) id 1BQZCx-0000yV-00 for icar-web-archive@ietf.org; Wed, 19 May 2004 18:06:39 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BQZ3e-0002lU-Ul; Wed, 19 May 2004 17:57:02 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BQYt0-0007Ne-Gd for icar@optimus.ietf.org; Wed, 19 May 2004 17:46:02 -0400 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA21431 for ; Wed, 19 May 2004 17:45:58 -0400 (EDT) Received: from ietf-mx.ietf.org ([132.151.6.1] helo=ietf-mx) by ietf-mx with esmtp (Exim 4.32) id 1BQYsx-00060U-Mt for icar@ietf.org; Wed, 19 May 2004 17:45:59 -0400 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1BQYry-0005ng-00 for icar@ietf.org; Wed, 19 May 2004 17:44:59 -0400 Received: from ns.execdsl.net ([208.184.15.238] helo=EXECDSL.COM) by ietf-mx with esmtp (Exim 4.12) id 1BQYqs-0005aP-00 for icar@ietf.org; Wed, 19 May 2004 17:43:50 -0400 Received: from [64.254.114.114] (HELO JLaptop.stevecrocker.com) by EXECDSL.COM (CommuniGate Pro SMTP 3.3) with ESMTP id 6969559 for icar@ietf.org; Wed, 19 May 2004 17:43:50 -0400 Message-Id: <5.1.0.14.0.20040519173932.0224dec0@localhost> X-Sender: joel@stevecrocker.com@localhost X-Mailer: QUALCOMM Windows Eudora Version 5.1 Date: Wed, 19 May 2004 17:43:31 -0400 To: icar@ietf.org From: "Joel M. Halpern" In-Reply-To: References: <5.1.0.14.0.20040519161941.0241a008@localhost> <20040519144950.EB9D977AA5C@guns.icir.org> <20040519144950.EB9D977AA5C@guns.icir.org> <5.1.0.14.0.20040519161941.0241a008@localhost> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed Subject: [Icar] Re: ICAR charter scope and review management Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.0 required=5.0 tests=AWL autolearn=no version=2.60 It is possible that you did not intend as large a scope as I read your comments as covering. If so, I apologize. Mandating that issues raised in ICAR describe reviews must be responded to in some fashion by the working group is something within charter for ICAR to say. Mandating that a WG must respond formally and on record to all comments it receives at any point in its work may or may not be a good idea, but is a working group process question, not a cross-area review question. I had read your (elided for brevity) comments as calling for the later. I apologize if the error was in my reading. Yours, Joel M. Halpern At 02:42 PM 5/19/2004 -0600, Alex Rousskov wrote: >On Wed, 19 May 2004, Joel M. Halpern wrote: > > > Most of these comments, while fair concerns, seem to be primarily > > addressed to the general question of how working groups manage > > comments. General working group process is probably outside of the > > scope of this working group. > >The charter seems to place any mechanism that encourages review of >documents in scope. There is probably nothing more discouraging than >seeing your comments ignored! > > > PS: To be specific, I do not think that the ICAR working group can > > direct that working groups should (much less must) have problem > > trackers, and should (or must) address all comments received, > > regardless of source. > >I do not see why not. The charter does not seem to place such >essential mechanisms out of scope. Am I missing something? > >Thanks, > >Alex. _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Wed May 19 18:28:26 2004 Received: from optimus.ietf.org (iesg.org [132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id SAA25147 for ; Wed, 19 May 2004 18:28:26 -0400 (EDT) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BQZQB-0007wH-97 for icar-archive@odin.ietf.org; Wed, 19 May 2004 18:20:19 -0400 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i4JMKJI9030512 for icar-archive@odin.ietf.org; Wed, 19 May 2004 18:20:19 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BQZFO-0001Wz-1U for icar-web-archive@optimus.ietf.org; Wed, 19 May 2004 18:09:10 -0400 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id SAA23712 for ; Wed, 19 May 2004 18:09:05 -0400 (EDT) Received: from ietf-mx.ietf.org ([132.151.6.1] helo=ietf-mx) by ietf-mx with esmtp (Exim 4.32) id 1BQZFL-0001Im-7W for icar-web-archive@ietf.org; Wed, 19 May 2004 18:09:07 -0400 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1BQZEP-0001C2-00 for icar-web-archive@ietf.org; Wed, 19 May 2004 18:08:09 -0400 Received: from optimus.ietf.org ([132.151.1.19]) by ietf-mx with esmtp (Exim 4.12) id 1BQZDd-000158-00 for icar-web-archive@ietf.org; Wed, 19 May 2004 18:07:21 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BQZ3f-0002lz-EV; Wed, 19 May 2004 17:57:03 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BQYtP-0007R2-Jq for icar@optimus.ietf.org; Wed, 19 May 2004 17:46:27 -0400 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA21502 for ; Wed, 19 May 2004 17:46:23 -0400 (EDT) Received: from ietf-mx.ietf.org ([132.151.6.1] helo=ietf-mx) by ietf-mx with esmtp (Exim 4.32) id 1BQYtM-00063x-UZ for icar@ietf.org; Wed, 19 May 2004 17:46:25 -0400 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1BQYse-0005uX-00 for icar@ietf.org; Wed, 19 May 2004 17:45:41 -0400 Received: from sj-iport-2-in.cisco.com ([171.71.176.71] helo=sj-iport-2.cisco.com) by ietf-mx with esmtp (Exim 4.12) id 1BQYrY-0005fE-00 for icar@ietf.org; Wed, 19 May 2004 17:44:32 -0400 Received: from sj-core-1.cisco.com (171.71.177.237) by sj-iport-2.cisco.com with ESMTP; 19 May 2004 13:49:53 +0000 Received: from mira-sjc5-c.cisco.com (IDENT:mirapoint@mira-sjc5-c.cisco.com [171.71.163.17]) by sj-core-1.cisco.com (8.12.10/8.12.6) with ESMTP id i4JLi0C1010713; Wed, 19 May 2004 14:44:00 -0700 (PDT) Received: from cisco.com ([10.25.65.178]) by mira-sjc5-c.cisco.com (MOS 3.4.5-GR) with SMTP id AUH33683; Wed, 19 May 2004 14:43:58 -0700 (PDT) Date: Wed, 19 May 2004 17:43:53 -0400 Subject: Re: [Icar] an early review experiment Content-Type: text/plain; charset=US-ASCII; format=flowed Mime-Version: 1.0 (Apple Message framework v553) Cc: Alex Rousskov , icar@ietf.org To: mallman@icir.org From: Melinda Shore In-Reply-To: <20040519201717.D137477AA5C@guns.icir.org> Message-Id: <9FA9743C-A9DD-11D8-8468-000A95E35274@cisco.com> Content-Transfer-Encoding: 7bit X-Mailer: Apple Mail (2.553) Content-Transfer-Encoding: 7bit Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.0 required=5.0 tests=AWL autolearn=no version=2.60 Content-Transfer-Encoding: 7bit Content-Transfer-Encoding: 7bit On Wednesday, May 19, 2004, at 04:17 PM, Mark Allman wrote: > I'd like to hear whether folk's thoughts WGs completely ignoring > comments. I've seen working groups fail to adequately address comments and then have it come back to bite them in later review or even when the IESG is reviewing a document. I've also seen them completely ignore comments, but haven't seen one ignore comments from someone with a formal role. The thing is that the reasons for them doing so vary, from the comments being muddled and irrelevant to the working group being insular and overly committed to the wrong path. I would tend to look for a chairing failure when it happens. Melinda _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Wed May 19 18:50:25 2004 Received: from optimus.ietf.org (iesg.org [132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id SAA26484 for ; Wed, 19 May 2004 18:50:25 -0400 (EDT) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BQZnr-0007Mt-Lw for icar-archive@odin.ietf.org; Wed, 19 May 2004 18:44:47 -0400 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i4JMil7T028321 for icar-archive@odin.ietf.org; Wed, 19 May 2004 18:44:47 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BQZfW-0004fm-Lj for icar-web-archive@optimus.ietf.org; Wed, 19 May 2004 18:36:10 -0400 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id SAA25578 for ; Wed, 19 May 2004 18:36:06 -0400 (EDT) Received: from ietf-mx.ietf.org ([132.151.6.1] helo=ietf-mx) by ietf-mx with esmtp (Exim 4.32) id 1BQZfT-0004Ek-J8 for icar-web-archive@ietf.org; Wed, 19 May 2004 18:36:07 -0400 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1BQZeZ-000491-00 for icar-web-archive@ietf.org; Wed, 19 May 2004 18:35:12 -0400 Received: from optimus.ietf.org ([132.151.1.19]) by ietf-mx with esmtp (Exim 4.12) id 1BQZe7-000433-00 for icar-web-archive@ietf.org; Wed, 19 May 2004 18:34:43 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BQZXl-0002RQ-MH; Wed, 19 May 2004 18:28:09 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BQZO2-0006Zc-M7 for icar@optimus.ietf.org; Wed, 19 May 2004 18:18:06 -0400 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id SAA24682 for ; Wed, 19 May 2004 18:18:02 -0400 (EDT) Received: from ietf-mx.ietf.org ([132.151.6.1] helo=ietf-mx) by ietf-mx with esmtp (Exim 4.32) id 1BQZNz-0002Hb-KH for icar@ietf.org; Wed, 19 May 2004 18:18:03 -0400 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1BQZN1-0002C1-00 for icar@ietf.org; Wed, 19 May 2004 18:17:04 -0400 Received: from measurement-factory.com ([206.168.0.5]) by ietf-mx with esmtp (Exim 4.12) id 1BQZMX-000270-00 for icar@ietf.org; Wed, 19 May 2004 18:16:33 -0400 Received: from measurement-factory.com (localhost [127.0.0.1]) by measurement-factory.com (8.12.9/8.12.9) with ESMTP id i4JMGSp6003139; Wed, 19 May 2004 16:16:28 -0600 (MDT) (envelope-from rousskov@measurement-factory.com) Received: (from rousskov@localhost) by measurement-factory.com (8.12.9/8.12.9/Submit) id i4JMGShb003138; Wed, 19 May 2004 16:16:28 -0600 (MDT) (envelope-from rousskov) Date: Wed, 19 May 2004 16:16:28 -0600 (MDT) From: Alex Rousskov To: "Joel M. Halpern" cc: icar@ietf.org Subject: Re: [Icar] Re: ICAR charter scope and review management In-Reply-To: <5.1.0.14.0.20040519173932.0224dec0@localhost> Message-ID: References: <5.1.0.14.0.20040519161941.0241a008@localhost> <20040519144950.EB9D977AA5C@guns.icir.org> <20040519144950.EB9D977AA5C@guns.icir.org> <5.1.0.14.0.20040519161941.0241a008@localhost> <5.1.0.14.0.20040519173932.0224dec0@localhost> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.0 required=5.0 tests=AWL autolearn=no version=2.60 On Wed, 19 May 2004, Joel M. Halpern wrote: > Mandating that a WG must respond formally and on record to all > comments it receives at any point in its work may or may not be a > good idea, but is a working group process question, not a cross-area > review question. I had read your (elided for brevity) comments as > calling for the later. I was calling for both, I guess. Even if ICAR scope prevents us from developing a general mechanism for all reviews, it does not prevent us from developing an in-scope mechanism that can be used equally well for out-of-ICAR-scope reviews. For example, we can say that WG MUST address all First Call and Last Call cross-area review comments. We can they try to apply the same principle to all First Call and Last Call comments, not just cross-area ones. Alex. _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Wed May 19 23:13:37 2004 Received: from optimus.ietf.org (www.iesg.org [132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id XAA08685 for ; Wed, 19 May 2004 23:13:37 -0400 (EDT) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BQdyb-0003ev-Sw for icar-archive@odin.ietf.org; Wed, 19 May 2004 23:12:10 -0400 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i4K3C9Hr014065 for icar-archive@odin.ietf.org; Wed, 19 May 2004 23:12:09 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BQdxO-0003N2-Nw for icar-web-archive@optimus.ietf.org; Wed, 19 May 2004 23:10:54 -0400 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id XAA08625 for ; Wed, 19 May 2004 23:10:50 -0400 (EDT) Received: from ietf-mx.ietf.org ([132.151.6.1] helo=ietf-mx) by ietf-mx with esmtp (Exim 4.32) id 1BQdxK-0001Jk-Jx for icar-web-archive@ietf.org; Wed, 19 May 2004 23:10:50 -0400 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1BQduz-00010m-00 for icar-web-archive@ietf.org; Wed, 19 May 2004 23:08:26 -0400 Received: from optimus.ietf.org ([132.151.1.19]) by ietf-mx with esmtp (Exim 4.12) id 1BQdsb-0000hC-00 for icar-web-archive@ietf.org; Wed, 19 May 2004 23:05:57 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BQdnx-0000rI-SL; Wed, 19 May 2004 23:01:09 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BQddR-0007GB-55 for icar@optimus.ietf.org; Wed, 19 May 2004 22:50:17 -0400 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id WAA07746 for ; Wed, 19 May 2004 22:50:13 -0400 (EDT) Received: from ietf-mx.ietf.org ([132.151.6.1] helo=ietf-mx) by ietf-mx with esmtp (Exim 4.32) id 1BQddN-0006JU-Jl for icar@ietf.org; Wed, 19 May 2004 22:50:13 -0400 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1BQdcM-00068X-00 for icar@ietf.org; Wed, 19 May 2004 22:49:11 -0400 Received: from wyvern.icir.org ([192.150.187.14]) by ietf-mx with esmtp (Exim 4.12) id 1BQdbR-0005wB-00 for icar@ietf.org; Wed, 19 May 2004 22:48:13 -0400 Received: from guns.icir.org (adsl-68-76-113-50.dsl.bcvloh.ameritech.net [68.76.113.50]) by wyvern.icir.org (8.12.9p1/8.12.8) with ESMTP id i4K2mCim067181; Wed, 19 May 2004 19:48:13 -0700 (PDT) (envelope-from mallman@icir.org) Received: from lawyers.icir.org (adsl-68-76-113-50.dsl.bcvloh.ameritech.net [68.76.113.50]) by guns.icir.org (Postfix) with ESMTP id AFC6877AA5C; Wed, 19 May 2004 22:48:10 -0400 (EDT) Received: from lawyers.icir.org (localhost [127.0.0.1]) by lawyers.icir.org (Postfix) with ESMTP id 8E28814FE9D; Wed, 19 May 2004 22:48:11 -0400 (EDT) To: Allison Mankin From: Mark Allman Reply-To: mallman@icir.org Cc: icar@ietf.org Subject: Re: [Icar] an early review experiment In-Reply-To: <200405192027.i4JKRKim061810@wyvern.icir.org> Organization: ICSI Center for Internet Research (ICIR) Song-of-the-Day: Panama MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha1; protocol="application/pgp-signature" Date: Wed, 19 May 2004 22:48:11 -0400 Message-Id: <20040520024811.8E28814FE9D@lawyers.icir.org> Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.0 required=5.0 tests=AWL autolearn=no version=2.60 --=-=-= Content-Type: text/plain Allison- Thanks for the note. > In my experience of quite a few AD reviewss (and those of AD's > directorates), there's a steep learning curve for cross-area review, > and also a difficult line between when something is actually a working > group consensus matter rather than a cross-area matter. My experience > is that the IESG learns how best to cross-area review from reviewing > each others' reviews and from reviewing hundreds of documents in all > the areas. There are certainlly other ways to get there, but we have > no handle on this now. Ah, that's a nice bit of experience. Thanks for sharing. So, it seems that there could (at some point; not immediately) there could be some interaction with the EDU team to see if there are ways to bootstrap folks (although, I suspect that experience in doing such reviews is going to be the most important aspect here). Thanks! allman --=-=-= Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.2 (Darwin) iD8DBQFArBxrWyrrWs4yIs4RAqHOAJ0ZymiSM1Gar1CSoIZLsKgDKeLXIgCfTUbM AxoZU5M0MSnPmU9xa/F/Auo= =BP8p -----END PGP SIGNATURE----- --=-=-=-- _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Wed May 19 23:14:08 2004 Received: from optimus.ietf.org (www.iesg.org [132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id XAA08708 for ; Wed, 19 May 2004 23:14:08 -0400 (EDT) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BQdyq-0003g3-VG for icar-archive@odin.ietf.org; Wed, 19 May 2004 23:12:25 -0400 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i4K3COSs014131 for icar-archive@odin.ietf.org; Wed, 19 May 2004 23:12:24 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BQdxt-0003SD-TM for icar-web-archive@optimus.ietf.org; Wed, 19 May 2004 23:11:25 -0400 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id XAA08647 for ; Wed, 19 May 2004 23:11:21 -0400 (EDT) Received: from ietf-mx.ietf.org ([132.151.6.1] helo=ietf-mx) by ietf-mx with esmtp (Exim 4.32) id 1BQdxp-0001M7-VQ for icar-web-archive@ietf.org; Wed, 19 May 2004 23:11:22 -0400 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1BQdvm-00018k-00 for icar-web-archive@ietf.org; Wed, 19 May 2004 23:09:18 -0400 Received: from optimus.ietf.org ([132.151.1.19]) by ietf-mx with esmtp (Exim 4.12) id 1BQdtN-0000pb-00 for icar-web-archive@ietf.org; Wed, 19 May 2004 23:06:45 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BQdpk-00014s-FN; Wed, 19 May 2004 23:03:00 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BQdl8-0008SZ-4t for icar@optimus.ietf.org; Wed, 19 May 2004 22:58:14 -0400 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id WAA08026 for ; Wed, 19 May 2004 22:58:10 -0400 (EDT) Received: from ietf-mx.ietf.org ([132.151.6.1] helo=ietf-mx) by ietf-mx with esmtp (Exim 4.32) id 1BQdl4-0007S2-Hx for icar@ietf.org; Wed, 19 May 2004 22:58:10 -0400 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1BQdk9-0007KC-00 for icar@ietf.org; Wed, 19 May 2004 22:57:14 -0400 Received: from wyvern.icir.org ([192.150.187.14]) by ietf-mx with esmtp (Exim 4.12) id 1BQdjE-0007BT-00 for icar@ietf.org; Wed, 19 May 2004 22:56:16 -0400 Received: from guns.icir.org (adsl-68-76-113-50.dsl.bcvloh.ameritech.net [68.76.113.50]) by wyvern.icir.org (8.12.9p1/8.12.8) with ESMTP id i4K2uGim067278; Wed, 19 May 2004 19:56:16 -0700 (PDT) (envelope-from mallman@icir.org) Received: from lawyers.icir.org (adsl-68-76-113-50.dsl.bcvloh.ameritech.net [68.76.113.50]) by guns.icir.org (Postfix) with ESMTP id 02DC877AA5C; Wed, 19 May 2004 22:56:14 -0400 (EDT) Received: from lawyers.icir.org (localhost [127.0.0.1]) by lawyers.icir.org (Postfix) with ESMTP id 9C9BD14FED8; Wed, 19 May 2004 22:56:14 -0400 (EDT) To: "Joel M. Halpern" From: Mark Allman Reply-To: mallman@icir.org Cc: icar@ietf.org, Alex Rousskov Subject: Re: [Icar] an early review experiment In-Reply-To: <5.1.0.14.0.20040519161941.0241a008@localhost> Organization: ICSI Center for Internet Research (ICIR) Song-of-the-Day: Panama MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha1; protocol="application/pgp-signature" Date: Wed, 19 May 2004 22:56:14 -0400 Message-Id: <20040520025614.9C9BD14FED8@lawyers.icir.org> Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.0 required=5.0 tests=AWL autolearn=no version=2.60 --=-=-= Content-Type: text/plain Joel- WG chair sparring for fun & profit. :-) > PS: To be specific, I do not think that the ICAR working group can > direct that working groups should (much less must) have problem > trackers, and should (or must) address all comments received, > regardless of source. Even though those may be good ideas. While I agree that general changes are out-of-scope for this WG, I think that something like requiring an issue tracker for some given ICAR scheme / experiment is well within bounds. And, if said issue tracker is more generally useful than just for ICAR things then so much the better. It would seem as if we would just hamstring an ICAR mechanism if we were to declare such things as out-of-scope. allman -- Mark Allman -- ICIR -- http://www.icir.org/mallman/ --=-=-= Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.2 (Darwin) iD8DBQFArB5OWyrrWs4yIs4RArsBAJsFDEtEcs9WMFcZMR9ezfp9C+URYACcDnz5 IzsHBsNNHR+99oj+NAhs948= =zDLJ -----END PGP SIGNATURE----- --=-=-=-- _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Thu May 20 00:11:55 2004 Received: from optimus.ietf.org (optimus.ietf.org [132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id AAA11257 for ; Thu, 20 May 2004 00:11:55 -0400 (EDT) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BQenx-0006YO-LX for icar-archive@odin.ietf.org; Thu, 20 May 2004 00:05:13 -0400 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i4K45Dh9025193 for icar-archive@odin.ietf.org; Thu, 20 May 2004 00:05:13 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BQefO-0004ZV-B9 for icar-web-archive@optimus.ietf.org; Wed, 19 May 2004 23:56:22 -0400 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id XAA10655 for ; Wed, 19 May 2004 23:56:19 -0400 (EDT) Received: from ietf-mx.ietf.org ([132.151.6.1] helo=ietf-mx) by ietf-mx with esmtp (Exim 4.32) id 1BQefM-0000Ba-46 for icar-web-archive@ietf.org; Wed, 19 May 2004 23:56:20 -0400 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1BQeeO-00002g-00 for icar-web-archive@ietf.org; Wed, 19 May 2004 23:55:21 -0400 Received: from optimus.ietf.org ([132.151.1.19]) by ietf-mx with esmtp (Exim 4.12) id 1BQedW-0007iB-00 for icar-web-archive@ietf.org; Wed, 19 May 2004 23:54:26 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BQec9-0003Do-T0; Wed, 19 May 2004 23:53:01 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BQeLA-0008Og-ES for icar@optimus.ietf.org; Wed, 19 May 2004 23:35:28 -0400 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id XAA10152 for ; Wed, 19 May 2004 23:35:25 -0400 (EDT) Received: from ietf-mx.ietf.org ([132.151.6.1] helo=ietf-mx) by ietf-mx with esmtp (Exim 4.32) id 1BQeL8-0005DU-DZ for icar@ietf.org; Wed, 19 May 2004 23:35:26 -0400 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1BQeKG-00054l-00 for icar@ietf.org; Wed, 19 May 2004 23:34:33 -0400 Received: from rwcrmhc12.comcast.net ([216.148.227.85]) by ietf-mx with esmtp (Exim 4.12) id 1BQeJZ-0004sq-00 for icar@ietf.org; Wed, 19 May 2004 23:33:49 -0400 Received: from dfnjgl21 (c-24-1-99-5.client.comcast.net[24.1.99.5]) by comcast.net (rwcrmhc12) with SMTP id <2004052003331401400jbjr1e> (Authid: sdawkins@comcast.net); Thu, 20 May 2004 03:33:14 +0000 Message-ID: <07f601c43e1b$4ce1ebd0$0200a8c0@DFNJGL21> Reply-To: "Spencer Dawkins" From: "Spencer Dawkins" To: References: <20040518180234.313BA77AB51@guns.icir.org> Subject: Re: [Icar] an early review experiment Date: Wed, 19 May 2004 22:34:04 -0500 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2800.1409 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1409 Content-Transfer-Encoding: 7bit Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.0 required=5.0 tests=AWL autolearn=no version=2.60 Content-Transfer-Encoding: 7bit Content-Transfer-Encoding: 7bit I may be going through a phase, but I would prefer to see ICAR reviews with no special status. My dream (and perhaps the dream of others) is that ICAR reviews are accessible from the document tracker. If this happens, it seems trivial enough for the IESG, or whomever we finally decide actually approves publication, to e-mail the ICAR reviewers and ask if there are still outstanding issues in the reviewer's opinion. If "no", the working group resolved the issues. If "yes", the IESG knows which end of the haystack to start looking at. If the review comment seems important enough to "discuss" (in the IESG sense of the word), it will be resolved, later and more painfully than was necessary. My experience has been that reviewees were either thrilled to have someone carefully reading their document, or were so detached that (I am not making this up) a document that hit the IESG telechat agenda within the last two months referred to keywords ala RFC 2119 AS A DRAFT: [KEYWORDS] Bradner, Scott, "Key Words for use in RFCs to Indicate Requirement Levels", draft-bradner-key-words-03.txt, January, 1997. - so no one had looked for nits in the document in the last seven years. Either way, I don't think making a document editor agree with me is necessary or desirable. YMMV, of course. Spencer > > + How binding are early, cross area review comments on authors / > editors / WGs? > > + one end of the spectrum would be that ICAR-type comments must be > addressed before the document moves forward > > + the other end of the space seems to me to be that the ICAR-type > reviews are just like any other review that comes in from a > community member (i.e., no special status) _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Thu May 20 00:40:05 2004 Received: from optimus.ietf.org (www.iesg.org [132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id AAA12723 for ; Thu, 20 May 2004 00:40:05 -0400 (EDT) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BQfGP-0006I1-GF for icar-archive@odin.ietf.org; Thu, 20 May 2004 00:34:37 -0400 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i4K4YbKu024172 for icar-archive@odin.ietf.org; Thu, 20 May 2004 00:34:37 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BQfDS-0005Xv-90 for icar-web-archive@optimus.ietf.org; Thu, 20 May 2004 00:31:34 -0400 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id AAA12375 for ; Thu, 20 May 2004 00:31:30 -0400 (EDT) Received: from ietf-mx.ietf.org ([132.151.6.1] helo=ietf-mx) by ietf-mx with esmtp (Exim 4.32) id 1BQfDP-0005bn-Ns for icar-web-archive@ietf.org; Thu, 20 May 2004 00:31:31 -0400 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1BQfCT-0005SR-00 for icar-web-archive@ietf.org; Thu, 20 May 2004 00:30:33 -0400 Received: from optimus.ietf.org ([132.151.1.19]) by ietf-mx with esmtp (Exim 4.12) id 1BQfC4-0005IF-00 for icar-web-archive@ietf.org; Thu, 20 May 2004 00:30:08 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BQf6L-0003e9-Os; Thu, 20 May 2004 00:24:13 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BQf4R-0002tv-Ja for icar@optimus.ietf.org; Thu, 20 May 2004 00:22:15 -0400 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id AAA11785 for ; Thu, 20 May 2004 00:22:12 -0400 (EDT) Received: from ietf-mx.ietf.org ([132.151.6.1] helo=ietf-mx) by ietf-mx with esmtp (Exim 4.32) id 1BQf4P-00048J-58 for icar@ietf.org; Thu, 20 May 2004 00:22:13 -0400 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1BQf3S-000400-00 for icar@ietf.org; Thu, 20 May 2004 00:21:15 -0400 Received: from measurement-factory.com ([206.168.0.5]) by ietf-mx with esmtp (Exim 4.12) id 1BQf2V-0003pb-00 for icar@ietf.org; Thu, 20 May 2004 00:20:16 -0400 Received: from measurement-factory.com (localhost [127.0.0.1]) by measurement-factory.com (8.12.9/8.12.9) with ESMTP id i4K4KEp6023609; Wed, 19 May 2004 22:20:14 -0600 (MDT) (envelope-from rousskov@measurement-factory.com) Received: (from rousskov@localhost) by measurement-factory.com (8.12.9/8.12.9/Submit) id i4K4KE02023608; Wed, 19 May 2004 22:20:14 -0600 (MDT) (envelope-from rousskov) Date: Wed, 19 May 2004 22:20:14 -0600 (MDT) From: Alex Rousskov To: Spencer Dawkins cc: icar@ietf.org Subject: Re: [Icar] an early review experiment In-Reply-To: <07f601c43e1b$4ce1ebd0$0200a8c0@DFNJGL21> Message-ID: References: <20040518180234.313BA77AB51@guns.icir.org> <07f601c43e1b$4ce1ebd0$0200a8c0@DFNJGL21> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.0 required=5.0 tests=AWL autolearn=no version=2.60 On Wed, 19 May 2004, Spencer Dawkins wrote: > My dream (and perhaps the dream of others) is that ICAR reviews are > accessible from the document tracker. If this happens, it seems > trivial enough for the IESG, or whomever we finally decide actually > approves publication, to e-mail the ICAR reviewers and ask if there > are still outstanding issues in the reviewer's opinion. IMO, the above would give us 80% of the benefit with 20% of the effort, which is a wonderful ratio for any pilot experiment. There are two tricky points here, I think: 1) The comments must be individually identifiable (like DISCUSS items). A long blob of text without specific items for WG to address is not good enough. 2) ID Tracker would have to authenticate reviewers one way or another to prevent unauthorized access and reviewer identity theft. If the experiment is limited to few super-qualified SIRs, then this step is easy. If the experiment is democratic, then it is not. > Either way, I don't think making a document editor agree with me is > necessary or desirable. Agreed. Alex. _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Thu May 20 12:05:48 2004 Received: from optimus.ietf.org (www.iesg.org [132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA27847 for ; Thu, 20 May 2004 12:05:47 -0400 (EDT) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BQplO-0001Lq-Ty for icar-archive@odin.ietf.org; Thu, 20 May 2004 11:47:19 -0400 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i4KFlIBF005194 for icar-archive@odin.ietf.org; Thu, 20 May 2004 11:47:18 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BQpLn-0008B9-R2 for icar-web-archive@optimus.ietf.org; Thu, 20 May 2004 11:20:51 -0400 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA23017 for ; Thu, 20 May 2004 11:20:49 -0400 (EDT) Received: from ietf-mx.ietf.org ([132.151.6.1] helo=ietf-mx) by ietf-mx with esmtp (Exim 4.32) id 1BQpLm-0002fn-Rf for icar-web-archive@ietf.org; Thu, 20 May 2004 11:20:50 -0400 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1BQpKx-0002P2-00 for icar-web-archive@ietf.org; Thu, 20 May 2004 11:20:00 -0400 Received: from optimus.ietf.org ([132.151.1.19]) by ietf-mx with esmtp (Exim 4.12) id 1BQpJf-0001uK-00 for icar-web-archive@ietf.org; Thu, 20 May 2004 11:18:39 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BQp62-0004Jy-AZ; Thu, 20 May 2004 11:04:34 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BQmKo-0004Y9-0l for icar@optimus.ietf.org; Thu, 20 May 2004 08:07:38 -0400 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id IAA13589 for ; Thu, 20 May 2004 08:07:36 -0400 (EDT) Received: from ietf-mx.ietf.org ([132.151.6.1] helo=ietf-mx) by ietf-mx with esmtp (Exim 4.32) id 1BQmKn-00009o-1K for icar@ietf.org; Thu, 20 May 2004 08:07:37 -0400 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1BQmJs-0007lY-00 for icar@ietf.org; Thu, 20 May 2004 08:06:40 -0400 Received: from rwcrmhc11.comcast.net ([204.127.198.35]) by ietf-mx with esmtp (Exim 4.12) id 1BQmJL-0007Xh-00 for icar@ietf.org; Thu, 20 May 2004 08:06:08 -0400 Received: from dfnjgl21 (c-24-1-99-5.client.comcast.net[24.1.99.5]) by comcast.net (rwcrmhc11) with SMTP id <20040520120536013004u9k0e> (Authid: sdawkins@comcast.net); Thu, 20 May 2004 12:05:37 +0000 Message-ID: <082e01c43e62$e0e30cf0$0200a8c0@DFNJGL21> Reply-To: "Spencer Dawkins" From: "Spencer Dawkins" To: References: <20040518180234.313BA77AB51@guns.icir.org> <07f601c43e1b$4ce1ebd0$0200a8c0@DFNJGL21> Subject: Re: [Icar] an early review experiment Date: Thu, 20 May 2004 07:06:26 -0500 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2800.1409 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1409 Content-Transfer-Encoding: 7bit Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.0 required=5.0 tests=AWL autolearn=no version=2.60 Content-Transfer-Encoding: 7bit Content-Transfer-Encoding: 7bit From: "Alex Rousskov" To: "Spencer Dawkins" Cc: Sent: Wednesday, May 19, 2004 11:20 PM Subject: Re: [Icar] an early review experiment > On Wed, 19 May 2004, Spencer Dawkins wrote: > > > My dream (and perhaps the dream of others) is that ICAR reviews are > > accessible from the document tracker. If this happens, it seems > > trivial enough for the IESG, or whomever we finally decide actually > > approves publication, to e-mail the ICAR reviewers and ask if there > > are still outstanding issues in the reviewer's opinion. > > IMO, the above would give us 80% of the benefit with 20% of the > effort, which is a wonderful ratio for any pilot experiment. There are > two tricky points here, I think: > > 1) The comments must be individually identifiable (like > DISCUSS items). A long blob of text without specific > items for WG to address is not good enough. > > 2) ID Tracker would have to authenticate reviewers > one way or another to prevent unauthorized access > and reviewer identity theft. If the experiment is > limited to few super-qualified SIRs, then this step is > easy. If the experiment is democratic, then it is not. I would think that we would have this as an ICAR requirement because people are sending unauthenticated private e-mail to ADs containing forged comments today at Last Call time ("Hi, this is Geoff Huston, and you need to DISCUSS this draft, because it *totally* overlooks this really cool technique, please overlook the patent issues"). I have no idea whether this is happening (and if the fiendish pseudo-reviewers are really gifted identity thieves, neither do the ADs), but I am comfortable not worrying about it in ICAR, especially if the ADs contact the apparent reviewers about any remaining issues ("no, I didn't write that review, thanks for asking"). _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Thu May 20 12:32:57 2004 Received: from optimus.ietf.org (www.iesg.org [132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA29684 for ; Thu, 20 May 2004 12:32:57 -0400 (EDT) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BQq1X-0006w5-F3 for icar-archive@odin.ietf.org; Thu, 20 May 2004 12:03:59 -0400 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i4KG3xwA026662 for icar-archive@odin.ietf.org; Thu, 20 May 2004 12:03:59 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BQplB-0001Ga-Ep for icar-web-archive@optimus.ietf.org; Thu, 20 May 2004 11:47:05 -0400 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA25762 for ; Thu, 20 May 2004 11:47:02 -0400 (EDT) Received: from ietf-mx.ietf.org ([132.151.6.1] helo=ietf-mx) by ietf-mx with esmtp (Exim 4.32) id 1BQplA-0006fT-Cr for icar-web-archive@ietf.org; Thu, 20 May 2004 11:47:04 -0400 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1BQpj6-0006Ly-00 for icar-web-archive@ietf.org; Thu, 20 May 2004 11:44:57 -0400 Received: from optimus.ietf.org ([132.151.1.19]) by ietf-mx with esmtp (Exim 4.12) id 1BQphR-00067u-00 for icar-web-archive@ietf.org; Thu, 20 May 2004 11:43:13 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BQp7h-0004dW-KW; Thu, 20 May 2004 11:06:17 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BQoYP-0005lR-8F for icar@optimus.ietf.org; Thu, 20 May 2004 10:29:49 -0400 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id KAA19918 for ; Thu, 20 May 2004 10:29:45 -0400 (EDT) Received: from ietf-mx.ietf.org ([132.151.6.1] helo=ietf-mx) by ietf-mx with esmtp (Exim 4.32) id 1BQoYM-0006KV-SM for icar@ietf.org; Thu, 20 May 2004 10:29:46 -0400 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1BQoVi-0005ke-00 for icar@ietf.org; Thu, 20 May 2004 10:27:03 -0400 Received: from wyvern.icir.org ([192.150.187.14]) by ietf-mx with esmtp (Exim 4.12) id 1BQoU5-0005LB-00 for icar@ietf.org; Thu, 20 May 2004 10:25:21 -0400 Received: from guns.icir.org (adsl-68-76-113-50.dsl.bcvloh.ameritech.net [68.76.113.50]) by wyvern.icir.org (8.12.9p1/8.12.8) with ESMTP id i4KEPFim076026; Thu, 20 May 2004 07:25:16 -0700 (PDT) (envelope-from mallman@guns.icir.org) Received: from guns.icir.org (localhost [127.0.0.1]) by guns.icir.org (Postfix) with ESMTP id EAEB777A6D5; Thu, 20 May 2004 10:25:14 -0400 (EDT) To: Brian E Carpenter From: Mark Allman Reply-To: mallman@icir.org Cc: icar WG Subject: Re: [Icar] Re: I-D ACTION:draft-allman-icar-wg-revcomm-00.txt In-Reply-To: <408FA8B6.30704@zurich.ibm.com> Organization: ICSI Center for Internet Research (ICIR) Song-of-the-Day: Ghost of a Texas Ladies Man MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha1; protocol="application/pgp-signature" Date: Thu, 20 May 2004 10:25:14 -0400 Message-Id: <20040520142514.EAEB777A6D5@guns.icir.org> Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.0 required=5.0 tests=AWL autolearn=no version=2.60 --=-=-= Content-Type: text/plain; charset=us-ascii (catching up; alas ...) (and, using my author, not wg-co-chair, hat) > > http://www.ietf.org/internet-drafts/draft-allman-icar-wg-revcomm-00.txt > > This is an interesting concept and I agree it is worth a try. > > How will this apply to non-WG drafts? The easy answer is that "it doesn't". It's not clear to me that we need to mix WG and non-WG drafts in an early review scheme. That's not to say that non-WG drafts should not be reviewed early. It's to say that this draft (and more generally the effort, IMO) is about helping the IETF work more smoothly. Of course, the IETF is involved in non-WG documents. But, it seems like that could work as a completely seperate process, with seperate review bodies and criteria and etc. That's not to say there couldn't be some common ground, too --- just that I don't imagine every early review proposal must apply to every situation. > One element of SIRs that is missing, if we implemented both this > and draft-alvestrand-icar-xarea-00.txt, is the notion of continuity. > SIRs suggested that a reviewer should stay with a document > through its life. If we set up both WG review teams (for early > review) and Area review teams (for late review), we will lose > this continuity. Also, we run a risk of not finding enough reviewers > to staff all these teams. Yep, that's a good assessment. On the other hand, there is no explicit coupling between the two drafts (that I know of). I.e., if one setup a WG review committee then maybe the later review bodies wouldn't be needed in practice; maybe the IESG could handle things as they do today, but more effectively because they are getting better quality due to cross-area review during the life-cycle of the document. > > A committee is formed by consultation between the WG chairs and > > shepherding AD. The WG chairs generate a list of potential members, > > and discuss with the AD about who might be appropriate. After an > > agreed upon list of members has been generated, the WG chairs > > contact the committee members to assess their willingness to serve. > > I would add here that there will be a pointer to the team > from the WG's charter page. Good call. allman -- Mark Allman -- ICIR -- http://www.icir.org/mallman/ --=-=-= Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.3 (FreeBSD) iD8DBQFArL/KWyrrWs4yIs4RAn4RAJwPhR4NaDYCvU071xKHp3SAL+JpVQCePpw7 0z4gbFiGEYW9g47K8vYVgTI= =8V5N -----END PGP SIGNATURE----- --=-=-=-- _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Thu May 20 12:34:35 2004 Received: from optimus.ietf.org (www.iesg.org [132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA29861 for ; Thu, 20 May 2004 12:34:35 -0400 (EDT) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BQq96-0001ys-Lu for icar-archive@odin.ietf.org; Thu, 20 May 2004 12:11:48 -0400 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i4KGBmNd007614 for icar-archive@odin.ietf.org; Thu, 20 May 2004 12:11:48 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BQpnH-00027G-GN for icar-web-archive@optimus.ietf.org; Thu, 20 May 2004 11:49:15 -0400 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA26220 for ; Thu, 20 May 2004 11:49:12 -0400 (EDT) Received: from ietf-mx.ietf.org ([132.151.6.1] helo=ietf-mx) by ietf-mx with esmtp (Exim 4.32) id 1BQpnG-00076v-Cb for icar-web-archive@ietf.org; Thu, 20 May 2004 11:49:14 -0400 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1BQplQ-0006iw-00 for icar-web-archive@ietf.org; Thu, 20 May 2004 11:47:23 -0400 Received: from optimus.ietf.org ([132.151.1.19]) by ietf-mx with esmtp (Exim 4.12) id 1BQpjJ-0006O1-00 for icar-web-archive@ietf.org; Thu, 20 May 2004 11:45:09 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BQp7z-0004iq-LC; Thu, 20 May 2004 11:06:35 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BQocg-0006BM-4X for icar@optimus.ietf.org; Thu, 20 May 2004 10:34:15 -0400 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id KAA20589 for ; Thu, 20 May 2004 10:34:10 -0400 (EDT) Received: from ietf-mx.ietf.org ([132.151.6.1] helo=ietf-mx) by ietf-mx with esmtp (Exim 4.32) id 1BQocd-0007P9-QL for icar@ietf.org; Thu, 20 May 2004 10:34:11 -0400 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1BQobZ-00077K-00 for icar@ietf.org; Thu, 20 May 2004 10:33:06 -0400 Received: from wyvern.icir.org ([192.150.187.14]) by ietf-mx with esmtp (Exim 4.12) id 1BQoZK-0006Vy-00 for icar@ietf.org; Thu, 20 May 2004 10:30:46 -0400 Received: from guns.icir.org (adsl-68-76-113-50.dsl.bcvloh.ameritech.net [68.76.113.50]) by wyvern.icir.org (8.12.9p1/8.12.8) with ESMTP id i4KEUhim076086; Thu, 20 May 2004 07:30:44 -0700 (PDT) (envelope-from mallman@guns.icir.org) Received: from guns.icir.org (localhost [127.0.0.1]) by guns.icir.org (Postfix) with ESMTP id 205F977A6D5; Thu, 20 May 2004 10:30:43 -0400 (EDT) To: Basavaraj.Patil@nokia.com From: Mark Allman Reply-To: mallman@icir.org Cc: kempf@docomolabs-usa.com, icar@ietf.org Subject: Re: [Icar] new review commitee i-d In-Reply-To: <697DAA22C5004B4596E033803A7CEF44024DC12F@daebe007.americas.nokia.com> Organization: ICSI Center for Internet Research (ICIR) Song-of-the-Day: Ghost of a Texas Ladies Man MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha1; protocol="application/pgp-signature" Date: Thu, 20 May 2004 10:30:43 -0400 Message-Id: <20040520143043.205F977A6D5@guns.icir.org> Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.0 required=5.0 tests=AWL autolearn=no version=2.60 --=-=-= Content-Type: text/plain; charset=us-ascii (again, catching up - sorry for the delay in responding) > http://www.ietf.org/internet-drafts/draft-allman-icar-wg-revcomm-00.txt Thanks! > 2. The rule-of-thumb of 2 reviewers per WG document may be a bit > excessive. I think most reviewers will be on review committees of > more than 1 WG and hence I am sceptical about large review > committees. 2 is a WAG. We'll add some words to basically indicate that some experience will help guide what that number really ought to be. > 3. Your idea of getting people who are not actively involved in the > IETF may be difficult. Sure. I agree. I think the general point is that the committee membership would not be strictly based on IETF participation or attendance or whatever. I.e., there are lots of non-IETFers that use IETF documents and could provide a useful set of eyes. If we can harness such talent, great! Is it a challenge? Yes, quite likely. > 4. Also it would be good to get some clarification regarding the stage > in the WG document that the review committee would be activated. Is > it before WG LC? Or is ot even prior to that? OK. We'll try to clean this up. (I think the vision is that the team would get a crack at the document at -00 or before and then at regular intervals until the WG forwards the document.) Thanks! allman -- Mark Allman -- ICIR -- http://www.icir.org/mallman/ --=-=-= Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.3 (FreeBSD) iD8DBQFArMETWyrrWs4yIs4RAiOuAJ9BGoDpXFyF2xOOhh8IlPdbS1HrOgCfZyC9 ZL0NBG//5+bZp/VnKWGoxE8= =2Y+j -----END PGP SIGNATURE----- --=-=-=-- _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Thu May 20 12:46:41 2004 Received: from optimus.ietf.org (www.iesg.org [132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA00700 for ; Thu, 20 May 2004 12:46:41 -0400 (EDT) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BQqUR-0000sv-Kr for icar-archive@odin.ietf.org; Thu, 20 May 2004 12:33:52 -0400 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i4KGXpEr003402 for icar-archive@odin.ietf.org; Thu, 20 May 2004 12:33:51 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BQq5a-0000Lq-LL for icar-web-archive@optimus.ietf.org; Thu, 20 May 2004 12:08:10 -0400 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA28176 for ; Thu, 20 May 2004 12:08:07 -0400 (EDT) Received: from ietf-mx.ietf.org ([132.151.6.1] helo=ietf-mx) by ietf-mx with esmtp (Exim 4.32) id 1BQq5Z-0001Yd-EL for icar-web-archive@ietf.org; Thu, 20 May 2004 12:08:09 -0400 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1BQq4g-0001SY-00 for icar-web-archive@ietf.org; Thu, 20 May 2004 12:07:15 -0400 Received: from optimus.ietf.org ([132.151.1.19]) by ietf-mx with esmtp (Exim 4.12) id 1BQq45-0001LC-00 for icar-web-archive@ietf.org; Thu, 20 May 2004 12:06:37 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BQpmD-0001nR-9w; Thu, 20 May 2004 11:48:09 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BQpI2-0006wU-PB for icar@optimus.ietf.org; Thu, 20 May 2004 11:16:58 -0400 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA22647 for ; Thu, 20 May 2004 11:16:56 -0400 (EDT) Received: from ietf-mx.ietf.org ([132.151.6.1] helo=ietf-mx) by ietf-mx with esmtp (Exim 4.32) id 1BQpI1-0001UA-T6 for icar@ietf.org; Thu, 20 May 2004 11:16:57 -0400 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1BQpGH-00017I-00 for icar@ietf.org; Thu, 20 May 2004 11:15:10 -0400 Received: from eikenes.alvestrand.no ([158.38.152.233]) by ietf-mx with esmtp (Exim 4.12) id 1BQpE2-0000WQ-00 for icar@ietf.org; Thu, 20 May 2004 11:12:50 -0400 Received: from localhost (localhost.localdomain [127.0.0.1]) by eikenes.alvestrand.no (Postfix) with ESMTP id 2AD7E622BB; Thu, 20 May 2004 17:12:20 +0200 (CEST) Received: from eikenes.alvestrand.no ([127.0.0.1]) by localhost (eikenes.alvestrand.no [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 32137-05; Thu, 20 May 2004 17:12:16 +0200 (CEST) Received: from halvestr-w2k02.emea.cisco.com (localhost.localdomain [127.0.0.1]) by eikenes.alvestrand.no (Postfix) with ESMTP id 8C70A61B8E; Thu, 20 May 2004 17:11:47 +0200 (CEST) Date: Thu, 20 May 2004 06:27:35 -0700 From: Harald Tveit Alvestrand To: "Joel M. Halpern" , icar@ietf.org Cc: Alex Rousskov Subject: Re: [Icar] an early review experiment Message-ID: <247198773.1085034455@localhost> In-Reply-To: <5.1.0.14.0.20040519161941.0241a008@localhost> References: <20040519144950.EB9D977AA5C@guns.icir.org> <20040519144950.EB9D977AA5C@guns.icir.org> <5.1.0.14.0.20040519161941.0241a008@localhost> X-Mailer: Mulberry/3.1.0 (Win32) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline X-Virus-Scanned: by amavisd-new at alvestrand.no Content-Transfer-Encoding: 7bit Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.0 required=5.0 tests=AWL autolearn=no version=2.60 Content-Transfer-Encoding: 7bit Content-Transfer-Encoding: 7bit --On 19. mai 2004 16:22 -0400 "Joel M. Halpern" wrote: > Most of these comments, while fair concerns, seem to be primarily > addressed to the general question of how working groups manage comments. > General working group process is probably outside of the scope of this > working group. > Yours, > Joel M. Halpern > > PS: To be specific, I do not think that the ICAR working group can direct > that working groups should (much less must) have problem trackers, and > should (or must) address all comments received, regardless of source. > Even though those may be good ideas. hmmmm...... I think we have a little more power here than that.... the charter says: > It is an explicit goal of the WG to come up with mechanisms > encouraging early review of the documents while ideas are still > in the formation stage. ..... > The WG will cooperate with others in starting and evaluating > experiments with early community and structured reviews. The > evaluation of such experiments may be published as Informational > RFCs if the group so desires. When this text was originally drafted, I think we thought we'd have an MPOWR WG looking at WG processes in general; we did not find traction for that. But - this group is certainly chartered to make experiments. It is a valid part of an experiment to ask a WG to volunteer to commit to address all comments raised in review (and to have another WG commit to NOT committing to that, if a control is needed), and it's within the charter of this WG to evaluate this experiment and conclude (if we still think so) that mandating that comments be addressed is a Good Thing (or not). Then we can figure out how the IESG should act on that recommendation. But - get the experiment running! Harald _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Thu May 20 23:30:15 2004 Received: from optimus.ietf.org (optimus.ietf.org [132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id XAA16204 for ; Thu, 20 May 2004 23:30:15 -0400 (EDT) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BR0fs-0007wJ-KN for icar-archive@odin.ietf.org; Thu, 20 May 2004 23:26:20 -0400 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i4L3QKcj030520 for icar-archive@odin.ietf.org; Thu, 20 May 2004 23:26:20 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BR0a1-0006yt-Tk for icar-web-archive@optimus.ietf.org; Thu, 20 May 2004 23:20:17 -0400 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id XAA15600 for ; Thu, 20 May 2004 23:20:15 -0400 (EDT) Received: from ietf-mx.ietf.org ([132.151.6.1] helo=ietf-mx) by ietf-mx with esmtp (Exim 4.32) id 1BR0a0-0005je-1l for icar-web-archive@ietf.org; Thu, 20 May 2004 23:20:16 -0400 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1BR0ZB-0005eQ-00 for icar-web-archive@ietf.org; Thu, 20 May 2004 23:19:26 -0400 Received: from optimus.ietf.org ([132.151.1.19]) by ietf-mx with esmtp (Exim 4.12) id 1BR0Yg-0005YY-00 for icar-web-archive@ietf.org; Thu, 20 May 2004 23:18:54 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BR0Wq-00063a-Fb; Thu, 20 May 2004 23:17:00 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BR0VT-0005rN-OK for icar@optimus.ietf.org; Thu, 20 May 2004 23:15:35 -0400 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id XAA15192 for ; Thu, 20 May 2004 23:15:32 -0400 (EDT) Received: from ietf-mx.ietf.org ([132.151.6.1] helo=ietf-mx) by ietf-mx with esmtp (Exim 4.32) id 1BR0VR-00057i-PM for icar@ietf.org; Thu, 20 May 2004 23:15:33 -0400 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1BR0UE-0004zM-00 for icar@ietf.org; Thu, 20 May 2004 23:14:21 -0400 Received: from wyvern.icir.org ([192.150.187.14]) by ietf-mx with esmtp (Exim 4.12) id 1BR0Sw-0004p5-00 for icar@ietf.org; Thu, 20 May 2004 23:12:58 -0400 Received: from guns.icir.org (adsl-68-76-113-50.dsl.bcvloh.ameritech.net [68.76.113.50]) by wyvern.icir.org (8.12.9p1/8.12.8) with ESMTP id i4L3Cuim088802; Thu, 20 May 2004 20:12:56 -0700 (PDT) (envelope-from mallman@icir.org) Received: from lawyers.icir.org (adsl-68-76-113-50.dsl.bcvloh.ameritech.net [68.76.113.50]) by guns.icir.org (Postfix) with ESMTP id C653E77A6D5; Thu, 20 May 2004 23:12:53 -0400 (EDT) Received: from lawyers.icir.org (localhost [127.0.0.1]) by lawyers.icir.org (Postfix) with ESMTP id 8C949150C05; Thu, 20 May 2004 23:12:54 -0400 (EDT) To: "Spencer Dawkins" From: Mark Allman Reply-To: mallman@icir.org Cc: icar@ietf.org Subject: Re: [Icar] an early review experiment In-Reply-To: <082e01c43e62$e0e30cf0$0200a8c0@DFNJGL21> Organization: ICSI Center for Internet Research (ICIR) Song-of-the-Day: Ghost of a Texas Ladies Man MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha1; protocol="application/pgp-signature" Date: Thu, 20 May 2004 23:12:54 -0400 Message-Id: <20040521031254.8C949150C05@lawyers.icir.org> Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.0 required=5.0 tests=AWL autolearn=no version=2.60 --=-=-= Content-Type: text/plain > I would think that we would have this as an ICAR requirement because > people are sending unauthenticated private e-mail to ADs containing > forged comments today at Last Call time ("Hi, this is Geoff Huston, > and you need to DISCUSS this draft, because it *totally* overlooks > this really cool technique, please overlook the patent issues"). > > I have no idea whether this is happening (and if the fiendish > pseudo-reviewers are really gifted identity thieves, neither do the > ADs), but I am comfortable not worrying about it in ICAR, especially > if the ADs contact the apparent reviewers about any remaining issues > ("no, I didn't write that review, thanks for asking"). I agree that this doesn't seem to be something to tackle in ICAR. But, I will note that we could encouraged signed reviews. And, also, I think the more public the review the less chance of it being forged without someone noticing and making a fuss. I.e., if Spencer is asked for a review and a fradulent review comes and then the real review comes then there is a very visible conflict. And, Spencer might notice the fraudulent version himself (i.e., "hey, I'd have never written that slop"). Now, if someone were to impersonate me, I might just let it slide since it'd probably make me look smarter. :-) allman --=-=-= Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.2 (Darwin) iD8DBQFArXO2WyrrWs4yIs4RAuSIAJwPzu6C1mA8qOVOkhW96UFqGg/mugCeN3YQ PGN32cofWdmEHFspJDRobdU= =/oRW -----END PGP SIGNATURE----- --=-=-=-- _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Mon May 24 08:43:29 2004 Received: from optimus.ietf.org (www.iesg.org [132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id IAA02294 for ; Mon, 24 May 2004 08:43:29 -0400 (EDT) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BSEdO-0008Ii-6U for icar-archive@odin.ietf.org; Mon, 24 May 2004 08:32:50 -0400 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i4OCWoQF031909 for icar-archive@odin.ietf.org; Mon, 24 May 2004 08:32:50 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BSETs-000770-46 for icar-web-archive@optimus.ietf.org; Mon, 24 May 2004 08:23:00 -0400 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id IAA01333 for ; Mon, 24 May 2004 08:22:57 -0400 (EDT) Received: from ietf-mx.ietf.org ([132.151.6.1] helo=ietf-mx) by ietf-mx with esmtp (Exim 4.32) id 1BSETr-0001sT-0B for icar-web-archive@ietf.org; Mon, 24 May 2004 08:22:59 -0400 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1BSESb-0001SW-00 for icar-web-archive@ietf.org; Mon, 24 May 2004 08:21:42 -0400 Received: from [65.246.255.50] (helo=mx2.foretec.com) by ietf-mx with esmtp (Exim 4.12) id 1BSERC-0000ru-02 for icar-web-archive@ietf.org; Mon, 24 May 2004 08:20:14 -0400 Received: from optimus22.ietf.org ([132.151.6.22] helo=optimus.ietf.org) by mx2.foretec.com with esmtp (Exim 4.24) id 1BSECn-0007JE-Dm for icar-web-archive@ietf.org; Mon, 24 May 2004 08:05:21 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BSE2t-0002Wc-G7; Mon, 24 May 2004 07:55:07 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BSDns-0000jZ-1M for icar@optimus.ietf.org; Mon, 24 May 2004 07:39:36 -0400 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id HAA28636 for ; Mon, 24 May 2004 07:39:34 -0400 (EDT) Received: from ietf-mx.ietf.org ([132.151.6.1] helo=ietf-mx) by ietf-mx with esmtp (Exim 4.32) id 1BSDnr-0003Fw-Af for icar@ietf.org; Mon, 24 May 2004 07:39:35 -0400 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1BSDms-0002v9-00 for icar@ietf.org; Mon, 24 May 2004 07:38:35 -0400 Received: from penguin.ericsson.se ([193.180.251.47]) by ietf-mx with esmtp (Exim 4.12) id 1BSDmB-0002aM-00 for icar@ietf.org; Mon, 24 May 2004 07:37:52 -0400 Received: from esealmw142.al.sw.ericsson.se ([153.88.254.119]) by penguin.ericsson.se (8.12.10/8.12.10/WIREfire-1.8b) with ESMTP id i4OBboPA015645 for ; Mon, 24 May 2004 13:37:51 +0200 (MEST) Received: from esealnt610.al.sw.ericsson.se ([153.88.254.120]) by esealmw142.al.sw.ericsson.se with Microsoft SMTPSVC(6.0.3790.0); Mon, 24 May 2004 13:37:50 +0200 Received: from lmlinux01.lmera.ericsson.se ([150.132.87.113]) by esealnt610.al.sw.ericsson.se with SMTP (Microsoft Exchange Internet Mail Service Version 5.5.2657.72) id LFSYKTFV; Mon, 24 May 2004 13:37:50 +0200 From: David Partain Reply-To: david.partain@ericsson.com Organization: Ericsson To: icar@ietf.org Date: Mon, 24 May 2004 13:37:31 +0200 User-Agent: KMail/1.6.2 MIME-Version: 1.0 Content-Disposition: inline Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Message-Id: <200405241337.31199.david.partain@ericsson.com> X-OriginalArrivalTime: 24 May 2004 11:37:50.0862 (UTC) FILETIME=[8B6882E0:01C44183] Content-Transfer-Encoding: 7bit Subject: [Icar] New draft - draft-partain-icar-pear-00.txt Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.0 required=5.0 tests=none autolearn=no version=2.60 Content-Transfer-Encoding: 7bit Content-Transfer-Encoding: 7bit Greetings, I realize that I'm a lurker, but I decided a while ago that I wanted to cobble together something that could perhaps be useful as conversation fodder for ICAR. So I did. I've just submitted draft-partain-icar-pear-00.txt to the I-D folks. It's basically my current thoughts on the task at hand in ICAR. There is a sort-of outline of how it might work in practice and several open questions. I'm afraid that it hasn't been updated since the recent flurry of mail. I suspect that it would be a good thing to do so but have simply not have time and would rather get this out now than wait 'til I do. If you don't want to wait for the I-D to appear in your neighborhood repository, you can find it temporarily at http://standards.ericsson.net/partain/drafts/draft-partain-icar-pear-00.txt Hope it's useful. Cheers, David _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Mon May 24 08:59:59 2004 Received: from optimus.ietf.org (www.iesg.org [132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id IAA03197 for ; Mon, 24 May 2004 08:59:59 -0400 (EDT) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BSEqt-0002YM-4O for icar-archive@odin.ietf.org; Mon, 24 May 2004 08:46:47 -0400 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i4OCklfJ009815 for icar-archive@odin.ietf.org; Mon, 24 May 2004 08:46:47 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BSEmn-0001k9-TZ for icar-web-archive@optimus.ietf.org; Mon, 24 May 2004 08:42:33 -0400 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id IAA02217 for ; Mon, 24 May 2004 08:42:31 -0400 (EDT) Received: from ietf-mx.ietf.org ([132.151.6.1] helo=ietf-mx) by ietf-mx with esmtp (Exim 4.32) id 1BSEmm-00004K-Hc for icar-web-archive@ietf.org; Mon, 24 May 2004 08:42:32 -0400 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1BSEli-0007Yi-00 for icar-web-archive@ietf.org; Mon, 24 May 2004 08:41:27 -0400 Received: from optimus.ietf.org ([132.151.1.19]) by ietf-mx with esmtp (Exim 4.12) id 1BSElN-0007H5-00 for icar-web-archive@ietf.org; Mon, 24 May 2004 08:41:05 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BSETv-00077d-0N; Mon, 24 May 2004 08:23:03 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BSEKc-0005do-UK for icar@optimus.ietf.org; Mon, 24 May 2004 08:13:26 -0400 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id IAA00891 for ; Mon, 24 May 2004 08:13:24 -0400 (EDT) Received: from ietf-mx.ietf.org ([132.151.6.1] helo=ietf-mx) by ietf-mx with esmtp (Exim 4.32) id 1BSEKb-0006qw-TL for icar@ietf.org; Mon, 24 May 2004 08:13:26 -0400 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1BSEJh-0006Yt-00 for icar@ietf.org; Mon, 24 May 2004 08:12:29 -0400 Received: from wyvern.icir.org ([192.150.187.14]) by ietf-mx with esmtp (Exim 4.12) id 1BSEJ3-0006Gh-00 for icar@ietf.org; Mon, 24 May 2004 08:11:50 -0400 Received: from guns.icir.org (adsl-68-76-113-50.dsl.bcvloh.ameritech.net [68.76.113.50]) by wyvern.icir.org (8.12.9p1/8.12.8) with ESMTP id i4OCBkim048077 for ; Mon, 24 May 2004 05:11:46 -0700 (PDT) (envelope-from mallman@guns.icir.org) Received: from guns.icir.org (localhost [127.0.0.1]) by guns.icir.org (Postfix) with ESMTP id E9A4877ACFC for ; Mon, 24 May 2004 08:11:45 -0400 (EDT) To: icar@ietf.org From: Mark Allman Reply-To: mallman@icir.org Organization: ICSI Center for Internet Research (ICIR) Song-of-the-Day: Lodi MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha1; protocol="application/pgp-signature" Date: Mon, 24 May 2004 08:11:45 -0400 Message-Id: <20040524121145.E9A4877ACFC@guns.icir.org> Subject: [Icar] early review consensus (part 1) Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.0 required=5.0 tests=AWL autolearn=no version=2.60 --=-=-= Content-Type: text/plain; charset=us-ascii In reading through everyone's thoughts from the last week, I want to assert that we have a decent amount of consensus on the following notions... * ICAR-type reviews should be openly published (on the WG list at a minimum) * ICAR-type reviews are not binding on WGs -- i.e., just because a reviewer says "you should do X" doesn't mean the WG is obligated to do so (i.e., reviewers do not have veto power) * WGs should be obligated to consider and discuss each suggestion from the reviewer's comments I meant to leave out specifics from the above list for now. What I'd like to do is to ensure that we do, in fact, have pretty good agreement on the notions first. Please yell if you think one of the above items is off-base from what folks have argued for. Thanks, allman --=-=-= Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.3 (FreeBSD) iD8DBQFAseaBWyrrWs4yIs4RAnZ+AJ44p/Z4/e1lWEJgQNXuPytLW6M9sgCgmB3H KeJqxd0R8MYkBZxMsgU+ReE= =OiFG -----END PGP SIGNATURE----- --=-=-=-- _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Mon May 24 09:08:52 2004 Received: from optimus.ietf.org (www.iesg.org [132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id JAA03667 for ; Mon, 24 May 2004 09:08:52 -0400 (EDT) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BSF3C-0003ui-AE for icar-archive@odin.ietf.org; Mon, 24 May 2004 08:59:30 -0400 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i4OCxUtM015045 for icar-archive@odin.ietf.org; Mon, 24 May 2004 08:59:30 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BSEqU-0002Sj-Gl for icar-web-archive@optimus.ietf.org; Mon, 24 May 2004 08:46:22 -0400 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id IAA02476 for ; Mon, 24 May 2004 08:46:19 -0400 (EDT) Received: from ietf-mx.ietf.org ([132.151.6.1] helo=ietf-mx) by ietf-mx with esmtp (Exim 4.32) id 1BSEqT-0001Ky-3R for icar-web-archive@ietf.org; Mon, 24 May 2004 08:46:21 -0400 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1BSEpU-00011x-00 for icar-web-archive@ietf.org; Mon, 24 May 2004 08:45:21 -0400 Received: from optimus.ietf.org ([132.151.1.19]) by ietf-mx with esmtp (Exim 4.12) id 1BSEoU-0000ft-00 for icar-web-archive@ietf.org; Mon, 24 May 2004 08:44:18 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BSEdZ-0008LM-HI; Mon, 24 May 2004 08:33:01 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BSETC-0006u5-4S for icar@optimus.ietf.org; Mon, 24 May 2004 08:22:18 -0400 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id IAA01223 for ; Mon, 24 May 2004 08:22:15 -0400 (EDT) Received: from ietf-mx.ietf.org ([132.151.6.1] helo=ietf-mx) by ietf-mx with esmtp (Exim 4.32) id 1BSETA-0001YV-EC for icar@ietf.org; Mon, 24 May 2004 08:22:16 -0400 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1BSERV-00016k-00 for icar@ietf.org; Mon, 24 May 2004 08:20:34 -0400 Received: from wyvern.icir.org ([192.150.187.14]) by ietf-mx with esmtp (Exim 4.12) id 1BSER6-0000nL-00 for icar@ietf.org; Mon, 24 May 2004 08:20:08 -0400 Received: from guns.icir.org (adsl-68-76-113-50.dsl.bcvloh.ameritech.net [68.76.113.50]) by wyvern.icir.org (8.12.9p1/8.12.8) with ESMTP id i4OCK7im048160 for ; Mon, 24 May 2004 05:20:07 -0700 (PDT) (envelope-from mallman@guns.icir.org) Received: from guns.icir.org (localhost [127.0.0.1]) by guns.icir.org (Postfix) with ESMTP id 8BF8077ACFC for ; Mon, 24 May 2004 08:20:06 -0400 (EDT) To: icar@ietf.org From: Mark Allman Reply-To: mallman@icir.org Organization: ICSI Center for Internet Research (ICIR) Song-of-the-Day: Lodi MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha1; protocol="application/pgp-signature" Date: Mon, 24 May 2004 08:20:06 -0400 Message-Id: <20040524122006.8BF8077ACFC@guns.icir.org> Subject: [Icar] reviewer pools Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.0 required=5.0 tests=AWL autolearn=no version=2.60 --=-=-= Content-Type: text/plain; charset=us-ascii I want to see if we can get some discussion going on how to design pool of reviewers for ICAR-type experiments. Some specific questions that jump to mind: + What should the granularity of the reviewer pool be? Should we have one big pool for the IETF (ala SIRS)? Or, should their be an area review team? Or, a WG review team? Or, a review team specific to each i-d? (That seems to be the space. Or, what else could their be?) Is there any value in having more than one granularity of team and having handoffs or overlap in some fashion? + Who should be eligible for these review teams? E.g., the kind of concrete track record of IETF work outlined in SIRS? E.g., teams populated by AD "nominations"? E.g., teams populated by WG chair "nominations"? E.g., purely voluntary teams with no hard requirements on who can volunteer? If folks could give their opinions on the above questions (and, anything else in this space, naturally), that'd be wonderful! Thanks, allman --=-=-= Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.3 (FreeBSD) iD8DBQFAseh2WyrrWs4yIs4RAmesAJ4y+I63GSjZtryMlXMb4YLzq4G4mACgkRGe wUlvzCriQ5vZUkSq6sFIJHk= =NFGt -----END PGP SIGNATURE----- --=-=-=-- _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Mon May 24 09:30:55 2004 Received: from optimus.ietf.org (optimus.ietf.org [132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id JAA05102 for ; Mon, 24 May 2004 09:30:55 -0400 (EDT) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BSFO8-0007U9-Ab for icar-archive@odin.ietf.org; Mon, 24 May 2004 09:21:08 -0400 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i4ODL8W0028774 for icar-archive@odin.ietf.org; Mon, 24 May 2004 09:21:08 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BSFDo-0005oa-6w for icar-web-archive@optimus.ietf.org; Mon, 24 May 2004 09:10:28 -0400 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id JAA03777 for ; Mon, 24 May 2004 09:10:25 -0400 (EDT) Received: from ietf-mx.ietf.org ([132.151.6.1] helo=ietf-mx) by ietf-mx with esmtp (Exim 4.32) id 1BSFDm-00011B-KN for icar-web-archive@ietf.org; Mon, 24 May 2004 09:10:26 -0400 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1BSFCr-0000hu-00 for icar-web-archive@ietf.org; Mon, 24 May 2004 09:09:30 -0400 Received: from optimus.ietf.org ([132.151.1.19]) by ietf-mx with esmtp (Exim 4.12) id 1BSFC9-0000OA-00 for icar-web-archive@ietf.org; Mon, 24 May 2004 09:08:45 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BSF3h-0003yL-0x; Mon, 24 May 2004 09:00:01 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BSEpj-0002L8-5r for icar@optimus.ietf.org; Mon, 24 May 2004 08:45:35 -0400 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id IAA02472 for ; Mon, 24 May 2004 08:45:32 -0400 (EDT) Received: from ietf-mx.ietf.org ([132.151.6.1] helo=ietf-mx) by ietf-mx with esmtp (Exim 4.32) id 1BSEph-00013h-SM for icar@ietf.org; Mon, 24 May 2004 08:45:33 -0400 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1BSEos-0000lA-00 for icar@ietf.org; Mon, 24 May 2004 08:44:43 -0400 Received: from penguin.ericsson.se ([193.180.251.47]) by ietf-mx with esmtp (Exim 4.12) id 1BSEoI-0000S1-00 for icar@ietf.org; Mon, 24 May 2004 08:44:07 -0400 Received: from esealmw143.al.sw.ericsson.se ([153.88.254.118]) by penguin.ericsson.se (8.12.10/8.12.10/WIREfire-1.8b) with ESMTP id i4OCi2PA002234 for ; Mon, 24 May 2004 14:44:06 +0200 (MEST) Received: from esealnt611.al.sw.ericsson.se ([153.88.254.121]) by esealmw143.al.sw.ericsson.se with Microsoft SMTPSVC(6.0.3790.0); Mon, 24 May 2004 14:44:02 +0200 Received: from lmlinux01.lmera.ericsson.se ([150.132.87.113]) by esealnt611.al.sw.ericsson.se with SMTP (Microsoft Exchange Internet Mail Service Version 5.5.2657.72) id LFSZ885M; Mon, 24 May 2004 14:44:02 +0200 From: David Partain Reply-To: david.partain@ericsson.com Organization: Ericsson To: icar@ietf.org Subject: Re: [Icar] early review consensus (part 1) Date: Mon, 24 May 2004 14:43:41 +0200 User-Agent: KMail/1.6.2 References: <20040524121145.E9A4877ACFC@guns.icir.org> In-Reply-To: <20040524121145.E9A4877ACFC@guns.icir.org> MIME-Version: 1.0 Content-Disposition: inline Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Message-Id: <200405241443.42813.david.partain@ericsson.com> X-OriginalArrivalTime: 24 May 2004 12:44:02.0536 (UTC) FILETIME=[CAB5DE80:01C4418C] Content-Transfer-Encoding: quoted-printable Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.0 required=5.0 tests=none autolearn=no version=2.60 Content-Transfer-Encoding: quoted-printable Content-Transfer-Encoding: quoted-printable =2D----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi, Since I didn't chime in last week, I'll do so now. On Monday 24 May 2004 14.11, Mark Allman wrote: > In reading through everyone's thoughts from the last week, I want to > assert that we have a decent amount of consensus on the following > notions... >=20 > * ICAR-type reviews should be openly published (on the WG list at a > minimum) > > * ICAR-type reviews are not binding on WGs -- i.e., just because a > reviewer says "you should do X" doesn't mean the WG is obligated to > do so (i.e., reviewers do not have veto power) >=20 > * WGs should be obligated to consider and discuss each suggestion from > the reviewer's comments Yes, I agree with all of the above. Thanks for summarizing. David =2D --=20 GPG key: http://standards.ericsson.net/partain/partain.asc =2D----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.4 (GNU/Linux) iD8DBQFAse39QVz1dWq9gegRAmrNAJ9OnVq/uO/jchb98vu7R5eSj6pSgQCffnI9 qMafNXKyaQpXH+GMMcGaehY=3D =3DjtM5 =2D----END PGP SIGNATURE----- _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Mon May 24 09:33:16 2004 Received: from optimus.ietf.org (optimus.ietf.org [132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id JAA05240 for ; Mon, 24 May 2004 09:33:16 -0400 (EDT) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BSFQN-0007lX-Pu for icar-archive@odin.ietf.org; Mon, 24 May 2004 09:23:27 -0400 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i4ODNRfF029846 for icar-archive@odin.ietf.org; Mon, 24 May 2004 09:23:27 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BSFKu-0006p9-Af for icar-web-archive@optimus.ietf.org; Mon, 24 May 2004 09:17:48 -0400 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id JAA04346 for ; Mon, 24 May 2004 09:17:45 -0400 (EDT) Received: from ietf-mx.ietf.org ([132.151.6.1] helo=ietf-mx) by ietf-mx with esmtp (Exim 4.32) id 1BSFKs-0003It-Mi for icar-web-archive@ietf.org; Mon, 24 May 2004 09:17:46 -0400 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1BSFJf-0002yU-00 for icar-web-archive@ietf.org; Mon, 24 May 2004 09:16:32 -0400 Received: from optimus.ietf.org ([132.151.1.19]) by ietf-mx with esmtp (Exim 4.12) id 1BSFIs-0002gJ-00 for icar-web-archive@ietf.org; Mon, 24 May 2004 09:15:42 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BSFBU-00059m-0Q; Mon, 24 May 2004 09:08:04 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BSF1A-0003lN-2b for icar@optimus.ietf.org; Mon, 24 May 2004 08:57:24 -0400 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id IAA03111 for ; Mon, 24 May 2004 08:57:21 -0400 (EDT) Received: from ietf-mx.ietf.org ([132.151.6.1] helo=ietf-mx) by ietf-mx with esmtp (Exim 4.32) id 1BSF18-0004mA-Lr for icar@ietf.org; Mon, 24 May 2004 08:57:22 -0400 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1BSF09-0004TQ-00 for icar@ietf.org; Mon, 24 May 2004 08:56:22 -0400 Received: from eagle.ericsson.se ([193.180.251.53]) by ietf-mx with esmtp (Exim 4.12) id 1BSEz6-0003zq-00 for icar@ietf.org; Mon, 24 May 2004 08:55:16 -0400 Received: from esealmw140.al.sw.ericsson.se ([153.88.254.121]) by eagle.ericsson.se (8.12.10/8.12.10/WIREfire-1.8b) with ESMTP id i4OCtEAh031239 for ; Mon, 24 May 2004 14:55:14 +0200 Received: from esealnt611.al.sw.ericsson.se ([153.88.254.121]) by esealmw140.al.sw.ericsson.se with Microsoft SMTPSVC(6.0.3790.0); Mon, 24 May 2004 14:55:14 +0200 Received: from lmlinux01.lmera.ericsson.se ([150.132.87.113]) by esealnt611.al.sw.ericsson.se with SMTP (Microsoft Exchange Internet Mail Service Version 5.5.2657.72) id LFSZ9BA9; Mon, 24 May 2004 14:55:14 +0200 From: David Partain Reply-To: david.partain@ericsson.com Organization: Ericsson To: icar@ietf.org Subject: Re: [Icar] reviewer pools Date: Mon, 24 May 2004 14:54:54 +0200 User-Agent: KMail/1.6.2 References: <20040524122006.8BF8077ACFC@guns.icir.org> In-Reply-To: <20040524122006.8BF8077ACFC@guns.icir.org> MIME-Version: 1.0 Content-Disposition: inline Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <200405241454.54899.david.partain@ericsson.com> X-OriginalArrivalTime: 24 May 2004 12:55:14.0852 (UTC) FILETIME=[5B712640:01C4418E] Content-Transfer-Encoding: 7bit Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.0 required=5.0 tests=none autolearn=no version=2.60 Content-Transfer-Encoding: 7bit Content-Transfer-Encoding: 7bit Hi, On Monday 24 May 2004 14.20, Mark Allman wrote: > I want to see if we can get some discussion going on how to design pool > of reviewers for ICAR-type experiments. Some specific questions that > jump to mind: > > + What should the granularity of the reviewer pool be? Should we have > one big pool for the IETF (ala SIRS)? Or, should their be an area > review team? Or, a WG review team? Or, a review team specific to > each i-d? (That seems to be the space. Or, what else could their > be?) Is there any value in having more than one granularity of team > and having handoffs or overlap in some fashion? My take is that we should (at least initially) use the current area granularity. It's what we have above the WG level and I think WG granularity will be unmanageable. Areas give us a reasonable administrative level to organize around. > + Who should be eligible for these review teams? E.g., the kind of > concrete track record of IETF work outlined in SIRS? E.g., teams > populated by AD "nominations"? E.g., teams populated by WG chair > "nominations"? E.g., purely voluntary teams with no hard > requirements on who can volunteer? I think this is a hard issue. Given the "volunteer" nature of the IETF, I think we should let volunteers prove their worth. That's why my draft suggests a volunteer pool. I believe that adding requirements on who may review and who may not is going to reduce the chances of getting sufficient numbers of reviewers. I realize that this opens the door to people who perhaps have no business giving "expert" review, but it also may allow people to gain a track record as reviewers despite lack of previous experience. Cheers, David _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Mon May 24 11:43:50 2004 Received: from optimus.ietf.org (www.iesg.org [132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA15302 for ; Mon, 24 May 2004 11:43:50 -0400 (EDT) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BSHBT-00089G-2r for icar-archive@odin.ietf.org; Mon, 24 May 2004 11:16:11 -0400 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i4OFGBQx031317 for icar-archive@odin.ietf.org; Mon, 24 May 2004 11:16:11 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BSH41-0006sm-9Y for icar-web-archive@optimus.ietf.org; Mon, 24 May 2004 11:08:29 -0400 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA12700 for ; Mon, 24 May 2004 11:08:24 -0400 (EDT) Received: from ietf-mx.ietf.org ([132.151.6.1] helo=ietf-mx) by ietf-mx with esmtp (Exim 4.32) id 1BSH3y-0004FC-JA for icar-web-archive@ietf.org; Mon, 24 May 2004 11:08:26 -0400 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1BSH36-0004Ab-00 for icar-web-archive@ietf.org; Mon, 24 May 2004 11:07:33 -0400 Received: from optimus.ietf.org ([132.151.1.19]) by ietf-mx with esmtp (Exim 4.12) id 1BSH2Z-00045k-00 for icar-web-archive@ietf.org; Mon, 24 May 2004 11:06:59 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BSGvy-00053B-Ie; Mon, 24 May 2004 11:00:10 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BSGY4-0001RN-4U for icar@optimus.ietf.org; Mon, 24 May 2004 10:35:28 -0400 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id KAA10535 for ; Mon, 24 May 2004 10:35:24 -0400 (EDT) Received: from ietf-mx.ietf.org ([132.151.6.1] helo=ietf-mx) by ietf-mx with esmtp (Exim 4.32) id 1BSGY1-0001dM-Lk for icar@ietf.org; Mon, 24 May 2004 10:35:25 -0400 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1BSGXE-0001Yz-00 for icar@ietf.org; Mon, 24 May 2004 10:34:37 -0400 Received: from sj-iport-2-in.cisco.com ([171.71.176.71] helo=sj-iport-2.cisco.com) by ietf-mx with esmtp (Exim 4.12) id 1BSGWL-0001Oh-00 for icar@ietf.org; Mon, 24 May 2004 10:33:41 -0400 Received: from sj-core-4.cisco.com (171.68.223.138) by sj-iport-2.cisco.com with ESMTP; 24 May 2004 06:39:57 +0000 Received: from mira-sjc5-c.cisco.com (IDENT:mirapoint@mira-sjc5-c.cisco.com [171.71.163.17]) by sj-core-4.cisco.com (8.12.10/8.12.6) with ESMTP id i4OEX8Vh008153; Mon, 24 May 2004 07:33:09 -0700 (PDT) Received: from cisco.com ([10.25.65.178]) by mira-sjc5-c.cisco.com (MOS 3.4.5-GR) with SMTP id AUL64382; Mon, 24 May 2004 07:33:07 -0700 (PDT) Date: Mon, 24 May 2004 10:33:00 -0400 Subject: Re: [Icar] reviewer pools Content-Type: text/plain; charset=US-ASCII; format=flowed Mime-Version: 1.0 (Apple Message framework v553) Cc: icar@ietf.org To: mallman@icir.org From: Melinda Shore In-Reply-To: <20040524122006.8BF8077ACFC@guns.icir.org> Message-Id: <41EAFA5C-AD8F-11D8-82A2-000A95E35274@cisco.com> Content-Transfer-Encoding: 7bit X-Mailer: Apple Mail (2.553) Content-Transfer-Encoding: 7bit Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.0 required=5.0 tests=AWL autolearn=no version=2.60 Content-Transfer-Encoding: 7bit Content-Transfer-Encoding: 7bit On Monday, May 24, 2004, at 08:20 AM, Mark Allman wrote: > + What should the granularity of the reviewer pool be? Should we > have > one big pool for the IETF (ala SIRS)? Or, should their be an area > review team? Or, a WG review team? Or, a review team specific to > each i-d? (That seems to be the space. Or, what else could their > be?) Is there any value in having more than one granularity of > team > and having handoffs or overlap in some fashion? It seems to me that this question is at least a little related to the question of the role of the "technical adviser" assigned to some working groups. I'm not sure that function is working very well, but assuming it is, having a per-wg review team would appear to overlap with the technical advisor. That's not necessarily a bad thing. Another related question is how the reviewers are selected or assigned. My preference is to have a general pool of reviewers and have the reviewers get per-document review responsibility, but that implies that there's someone shepherding the document review, as well, and making sure that it happens in a timely way and doesn't just drop through the cracks. That person would request the review. My primary concern with how the reviewer pool functions is the possibility of gaps in coverage, either due to expertise or availability, and I think I would tend to optimize how reviewers are selected/provided around that issue - how do you ensure that every document that needs review gets it? Melinda _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Mon May 24 11:46:45 2004 Received: from optimus.ietf.org (www.iesg.org [132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA15546 for ; Mon, 24 May 2004 11:46:45 -0400 (EDT) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BSHIY-0000Kf-UQ for icar-archive@odin.ietf.org; Mon, 24 May 2004 11:23:30 -0400 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i4OFNU8u001264 for icar-archive@odin.ietf.org; Mon, 24 May 2004 11:23:30 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BSH92-0007Vo-PD for icar-web-archive@optimus.ietf.org; Mon, 24 May 2004 11:13:40 -0400 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA12987 for ; Mon, 24 May 2004 11:13:37 -0400 (EDT) Received: from ietf-mx.ietf.org ([132.151.6.1] helo=ietf-mx) by ietf-mx with esmtp (Exim 4.32) id 1BSH91-0004dM-OJ for icar-web-archive@ietf.org; Mon, 24 May 2004 11:13:39 -0400 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1BSH7t-0004XO-00 for icar-web-archive@ietf.org; Mon, 24 May 2004 11:12:30 -0400 Received: from optimus.ietf.org ([132.151.1.19]) by ietf-mx with esmtp (Exim 4.12) id 1BSH7a-0004TC-00 for icar-web-archive@ietf.org; Mon, 24 May 2004 11:12:10 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BSGws-0005R8-R1; Mon, 24 May 2004 11:01:06 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BSGnU-0003lt-2N for icar@optimus.ietf.org; Mon, 24 May 2004 10:51:24 -0400 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id KAA11341 for ; Mon, 24 May 2004 10:51:19 -0400 (EDT) Received: from ietf-mx.ietf.org ([132.151.6.1] helo=ietf-mx) by ietf-mx with esmtp (Exim 4.32) id 1BSGnR-0002hs-Jl for icar@ietf.org; Mon, 24 May 2004 10:51:21 -0400 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1BSGmW-0002e3-00 for icar@ietf.org; Mon, 24 May 2004 10:50:25 -0400 Received: from wyvern.icir.org ([192.150.187.14]) by ietf-mx with esmtp (Exim 4.12) id 1BSGlh-0002af-00 for icar@ietf.org; Mon, 24 May 2004 10:49:33 -0400 Received: from guns.icir.org (adsl-68-76-113-50.dsl.bcvloh.ameritech.net [68.76.113.50]) by wyvern.icir.org (8.12.9p1/8.12.8) with ESMTP id i4OEnWim049746 for ; Mon, 24 May 2004 07:49:32 -0700 (PDT) (envelope-from mallman@guns.icir.org) Received: from guns.icir.org (localhost [127.0.0.1]) by guns.icir.org (Postfix) with ESMTP id A4EC977ACFC for ; Mon, 24 May 2004 10:49:30 -0400 (EDT) To: icar@ietf.org From: Mark Allman Reply-To: mallman@icir.org Organization: ICSI Center for Internet Research (ICIR) Song-of-the-Day: Lodi MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha1; protocol="application/pgp-signature" Date: Mon, 24 May 2004 10:49:30 -0400 Message-Id: <20040524144930.A4EC977ACFC@guns.icir.org> Subject: [Icar] technical advisor subroutine call Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.0 required=5.0 tests=AWL autolearn=no version=2.60 --=-=-= Content-Type: text/plain; charset=us-ascii Melinda brought up the technical advisor role. As this is a form of early review from outsiders, I think it would be useful to collect up some data points on whether this works, doesn't work, is used in some WGs successfully, etc. (I am particularly interested because none of the groups I work in have advisors. And, even when they were assigned one, the advisor wasn't much used, in my experience. So, I am pretty clueless on how the larger community uses these roles.) All input appreciated! allman --=-=-= Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.3 (FreeBSD) iD8DBQFAsgt6WyrrWs4yIs4RAuSYAJ9eblTjGbzSEcp+0h2jQL+WHOnFGwCfQPkj QiHwQF8h5yQklJcR3f/ldZA= =Z+s4 -----END PGP SIGNATURE----- --=-=-=-- _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Mon May 24 12:14:34 2004 Received: from optimus.ietf.org (www.iesg.org [132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA17245 for ; Mon, 24 May 2004 12:14:33 -0400 (EDT) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BSHka-0007UM-Hm for icar-archive@odin.ietf.org; Mon, 24 May 2004 11:52:28 -0400 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i4OFqSZP028782 for icar-archive@odin.ietf.org; Mon, 24 May 2004 11:52:28 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BSHcB-0005Zh-Q3 for icar-web-archive@optimus.ietf.org; Mon, 24 May 2004 11:43:47 -0400 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA15279 for ; Mon, 24 May 2004 11:43:44 -0400 (EDT) Received: from ietf-mx.ietf.org ([132.151.6.1] helo=ietf-mx) by ietf-mx with esmtp (Exim 4.32) id 1BSHcA-00009r-Kb for icar-web-archive@ietf.org; Mon, 24 May 2004 11:43:46 -0400 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1BSHb0-0007nn-00 for icar-web-archive@ietf.org; Mon, 24 May 2004 11:42:35 -0400 Received: from optimus.ietf.org ([132.151.1.19]) by ietf-mx with esmtp (Exim 4.12) id 1BSHaR-0007ij-00 for icar-web-archive@ietf.org; Mon, 24 May 2004 11:41:59 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BSHAh-0007qX-Av; Mon, 24 May 2004 11:15:23 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BSGzc-0005qG-TP for icar@optimus.ietf.org; Mon, 24 May 2004 11:03:56 -0400 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA12163 for ; Mon, 24 May 2004 11:03:52 -0400 (EDT) Received: from ietf-mx.ietf.org ([132.151.6.1] helo=ietf-mx) by ietf-mx with esmtp (Exim 4.32) id 1BSGza-0003dc-BM for icar@ietf.org; Mon, 24 May 2004 11:03:54 -0400 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1BSGyR-0003Uy-00 for icar@ietf.org; Mon, 24 May 2004 11:02:44 -0400 Received: from wyvern.icir.org ([192.150.187.14]) by ietf-mx with esmtp (Exim 4.12) id 1BSGxL-0003Oi-00 for icar@ietf.org; Mon, 24 May 2004 11:01:35 -0400 Received: from guns.icir.org (adsl-68-76-113-50.dsl.bcvloh.ameritech.net [68.76.113.50]) by wyvern.icir.org (8.12.9p1/8.12.8) with ESMTP id i4OF16im049915; Mon, 24 May 2004 08:01:06 -0700 (PDT) (envelope-from mallman@guns.icir.org) Received: from guns.icir.org (localhost [127.0.0.1]) by guns.icir.org (Postfix) with ESMTP id 2C70A77ACFC; Mon, 24 May 2004 11:01:05 -0400 (EDT) To: Melinda Shore From: Mark Allman Reply-To: mallman@icir.org Cc: icar@ietf.org Subject: Re: [Icar] reviewer pools In-Reply-To: <41EAFA5C-AD8F-11D8-82A2-000A95E35274@cisco.com> Organization: ICSI Center for Internet Research (ICIR) Song-of-the-Day: Lodi MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha1; protocol="application/pgp-signature" Date: Mon, 24 May 2004 11:01:05 -0400 Message-Id: <20040524150105.2C70A77ACFC@guns.icir.org> Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.0 required=5.0 tests=AWL autolearn=no version=2.60 --=-=-= Content-Type: text/plain; charset=us-ascii I am not arguing for one way or the other, just trying to explore ... > My preference is to have a general pool of reviewers and have the > reviewers get per-document review responsibility, but that implies > that there's someone shepherding the document review, as well, and > making sure that it happens in a timely way and doesn't just drop > through the cracks. That person would request the review. My primary > concern with how the reviewer pool functions is the possibility of > gaps in coverage, either due to expertise or availability, and I think > I would tend to optimize how reviewers are selected/provided around > that issue - how do you ensure that every document that needs review > gets it? I'm interested in hearing your thoughts on area review teams(*) then. That makes things much more explicit ... i.e., "we need a reviewer from the security team" rather than fishing around a large general pool (as you note is your preference) for someone who can cover security documents. Does pre-sorting make for a system that is less prone to "gaps"? (Of course, with some alternate downsides, maybe.) (*) A thought that just popped into my head is that "area teams" wouldn't necessarily have to fall along traditional boundaries - but, might. Call them "topic teams" or something. We could have topic teams on security (traditional IETF area) and architecture (not an area), for instance. Just throwing it out there ... I am not sold on it. Thanks! allman --=-=-= Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.3 (FreeBSD) iD8DBQFAsg4xWyrrWs4yIs4RAt/IAJ9I/L42CdFSqVz63TRvLh8pv7DSkACeJ0We gnD3FDOiQD6qXLIrunVEpas= =oHzF -----END PGP SIGNATURE----- --=-=-=-- _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Mon May 24 12:28:24 2004 Received: from optimus.ietf.org (optimus.ietf.org [132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA18125 for ; Mon, 24 May 2004 12:28:23 -0400 (EDT) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BSI8Q-0003ZX-Ht for icar-archive@odin.ietf.org; Mon, 24 May 2004 12:17:06 -0400 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i4OGH69a013731 for icar-archive@odin.ietf.org; Mon, 24 May 2004 12:17:06 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BSHql-0000Q9-3C for icar-web-archive@optimus.ietf.org; Mon, 24 May 2004 11:58:51 -0400 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA16429 for ; Mon, 24 May 2004 11:58:47 -0400 (EDT) Received: from ietf-mx.ietf.org ([132.151.6.1] helo=ietf-mx) by ietf-mx with esmtp (Exim 4.32) id 1BSHqj-0001n9-Qf for icar-web-archive@ietf.org; Mon, 24 May 2004 11:58:49 -0400 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1BSHpk-0001hw-00 for icar-web-archive@ietf.org; Mon, 24 May 2004 11:57:49 -0400 Received: from optimus.ietf.org ([132.151.1.19]) by ietf-mx with esmtp (Exim 4.12) id 1BSHoz-0001bj-00 for icar-web-archive@ietf.org; Mon, 24 May 2004 11:57:01 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BSHfK-0006HA-Cs; Mon, 24 May 2004 11:47:02 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BSHIR-0000Iu-Fu for icar@optimus.ietf.org; Mon, 24 May 2004 11:23:23 -0400 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA13490 for ; Mon, 24 May 2004 11:23:20 -0400 (EDT) Received: from ietf-mx.ietf.org ([132.151.6.1] helo=ietf-mx) by ietf-mx with esmtp (Exim 4.32) id 1BSHIQ-0005U0-GJ for icar@ietf.org; Mon, 24 May 2004 11:23:22 -0400 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1BSHHT-0005QN-00 for icar@ietf.org; Mon, 24 May 2004 11:22:23 -0400 Received: from sj-iport-3-in.cisco.com ([171.71.176.72] helo=sj-iport-3.cisco.com) by ietf-mx with esmtp (Exim 4.12) id 1BSHGo-0005JY-00 for icar@ietf.org; Mon, 24 May 2004 11:21:42 -0400 Received: from sj-core-4.cisco.com (171.68.223.138) by sj-iport-3.cisco.com with ESMTP; 24 May 2004 07:29:37 +0000 Received: from mira-sjc5-c.cisco.com (IDENT:mirapoint@mira-sjc5-c.cisco.com [171.71.163.17]) by sj-core-4.cisco.com (8.12.10/8.12.6) with ESMTP id i4OFL9Vh026755; Mon, 24 May 2004 08:21:09 -0700 (PDT) Received: from cisco.com ([10.25.65.178]) by mira-sjc5-c.cisco.com (MOS 3.4.5-GR) with SMTP id AUL68782; Mon, 24 May 2004 08:21:07 -0700 (PDT) Date: Mon, 24 May 2004 11:21:00 -0400 Subject: Re: [Icar] reviewer pools Content-Type: text/plain; charset=US-ASCII; format=flowed Mime-Version: 1.0 (Apple Message framework v553) Cc: icar@ietf.org To: mallman@icir.org From: Melinda Shore In-Reply-To: <20040524150105.2C70A77ACFC@guns.icir.org> Message-Id: Content-Transfer-Encoding: 7bit X-Mailer: Apple Mail (2.553) Content-Transfer-Encoding: 7bit Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.0 required=5.0 tests=AWL autolearn=no version=2.60 Content-Transfer-Encoding: 7bit Content-Transfer-Encoding: 7bit On Monday, May 24, 2004, at 11:01 AM, Mark Allman wrote: > I'm interested in hearing your thoughts on area review teams(*) then. > That makes things much more explicit ... i.e., "we need a reviewer from > the security team" rather than fishing around a large general pool (as > you note is your preference) for someone who can cover security > documents. Does pre-sorting make for a system that is less prone to > "gaps"? (Of course, with some alternate downsides, maybe.) I'm not sure. We're stretched as thin as can be on security reviewers today, for example. Will having a more formal review process make more security reviewers available? Probably not. I asked SIRS to review one document for midcom. We got three reviews out of it. They were all useful, and one of the really interesting things that happened is that at least one of the reviewers hadn't been familiar with our work or, I think, with the area (border policy enforcement) in general, and that brought a valuable fresh pair of eyes to the document. Part of the question is related to when the review happens, too - earlier in the process you need someone that brings a macro perspective to the entire problem, and later in the process, presumably the macro questions have been answered and you need someone with more specific, focused expertise to address particular problems within the documents. The issue I'm interested in seeing cross-area review deal with is the broad problem of working groups starting out well and somehow going off the rails early on. That, I think, benefits from wisdom and experience rather than specific area expertise. Melinda _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar From exim@www1.ietf.org Mon May 24 12:36:55 2004 Received: from optimus.ietf.org (optimus.ietf.org [132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA18459 for ; Mon, 24 May 2004 12:36:55 -0400 (EDT) Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BSII5-0005nL-1h for icar-archive@odin.ietf.org; Mon, 24 May 2004 12:27:05 -0400 Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i4OGR59n022270 for icar-archive@odin.ietf.org; Mon, 24 May 2004 12:27:05 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BSI7r-0003Ei-CU for icar-web-archive@optimus.ietf.org; Mon, 24 May 2004 12:16:31 -0400 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA17371 for ; Mon, 24 May 2004 12:16:27 -0400 (EDT) Received: from ietf-mx.ietf.org ([132.151.6.1] helo=ietf-mx) by ietf-mx with esmtp (Exim 4.32) id 1BSI7p-0003aQ-S8 for icar-web-archive@ietf.org; Mon, 24 May 2004 12:16:29 -0400 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1BSI6z-0003U9-00 for icar-web-archive@ietf.org; Mon, 24 May 2004 12:15:38 -0400 Received: from optimus.ietf.org ([132.151.1.19]) by ietf-mx with esmtp (Exim 4.12) id 1BSI61-0003O5-00 for icar-web-archive@ietf.org; Mon, 24 May 2004 12:14:37 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BSHlF-0007js-Ni; Mon, 24 May 2004 11:53:09 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BSHcq-0005l9-Vi for icar@optimus.ietf.org; Mon, 24 May 2004 11:44:29 -0400 Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA15388 for ; Mon, 24 May 2004 11:44:25 -0400 (EDT) Received: from ietf-mx.ietf.org ([132.151.6.1] helo=ietf-mx) by ietf-mx with esmtp (Exim 4.32) id 1BSHcp-0000Gm-RW for icar@ietf.org; Mon, 24 May 2004 11:44:27 -0400 Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1BSHbr-00006l-00 for icar@ietf.org; Mon, 24 May 2004 11:43:28 -0400 Received: from [65.246.255.50] (helo=mx2.foretec.com) by ietf-mx with esmtp (Exim 4.12) id 1BSHao-0007iy-04 for icar@ietf.org; Mon, 24 May 2004 11:42:22 -0400 Received: from wyvern.icir.org ([192.150.187.14]) by mx2.foretec.com with esmtp (Exim 4.24) id 1BSHUl-0003jB-Vn for icar@ietf.org; Mon, 24 May 2004 11:36:08 -0400 Received: from guns.icir.org (adsl-68-76-113-50.dsl.bcvloh.ameritech.net [68.76.113.50]) by wyvern.icir.org (8.12.9p1/8.12.8) with ESMTP id i4OFZeim050370; Mon, 24 May 2004 08:35:40 -0700 (PDT) (envelope-from mallman@guns.icir.org) Received: from guns.icir.org (localhost [127.0.0.1]) by guns.icir.org (Postfix) with ESMTP id CABED77ACFC; Mon, 24 May 2004 11:35:39 -0400 (EDT) To: Melinda Shore From: Mark Allman Reply-To: mallman@icir.org Cc: icar@ietf.org Subject: Re: [Icar] reviewer pools In-Reply-To: Organization: ICSI Center for Internet Research (ICIR) Song-of-the-Day: Lodi MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha1; protocol="application/pgp-signature" Date: Mon, 24 May 2004 11:35:39 -0400 Message-Id: <20040524153539.CABED77ACFC@guns.icir.org> Sender: icar-admin@ietf.org Errors-To: icar-admin@ietf.org X-BeenThere: icar@ietf.org X-Mailman-Version: 2.0.12 Precedence: bulk List-Unsubscribe: , List-Id: Improved Cross-Area Review List-Post: List-Help: List-Subscribe: , X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org X-Spam-Status: No, hits=0.0 required=5.0 tests=AWL autolearn=no version=2.60 --=-=-= Content-Type: text/plain; charset=us-ascii > On Monday, May 24, 2004, at 11:01 AM, Mark Allman wrote: > > I'm interested in hearing your thoughts on area review teams(*) then. > > That makes things much more explicit ... i.e., "we need a reviewer from > > the security team" rather than fishing around a large general pool (as > > you note is your preference) for someone who can cover security > > documents. Does pre-sorting make for a system that is less prone to > > "gaps"? (Of course, with some alternate downsides, maybe.) > > I'm not sure. We're stretched as thin as can be on security > reviewers today, for example. Will having a more formal review > process make more security reviewers available? Probably not. I was trying to ask about your thoughts on using smaller teams (i.e., not one large pool) as a way to manage the process with finer granularity and hence avoiding the "gaps" you noted. Clearly, there are resource issues no matter which way you go. You thoughts on those are useful (as always), thanks! allman --=-=-= Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.3 (FreeBSD) iD8DBQFAshZLWyrrWs4yIs4RAp3gAKCT4YbNJ4t55u4Yq/EXSKKlXMfkyQCghrkD JGlzCm6D4IrJB8mjtnqVrjs= =kpBN -----END PGP SIGNATURE----- --=-=-=-- _______________________________________________ Icar mailing list Icar@ietf.org https://www1.ietf.org/mailman/listinfo/icar