Subject: [secdir] Fwd: timing of reviews

From: Stephen Farrell <[email-masked]>
Date: Fri, 24 May 2013 10:11:25 +0100

Folks,

Jari's mail below says it better than I could.

What do you think about this?

Thanks,
S.

-------- Original Message --------
Subject: timing of reviews
Date: Fri, 24 May 2013 01:28:48 +0300
From: Jari Arkko <[email-masked]>
To: General Area Review Team <[email-masked]>
CC: The IESG <[email-masked]>

Thanks again for all your hard work in doing the reviews. They make it
possible for me to concentrate on reviewing just those documents where
there are problems or I have deeper expertise. And the quality of the
protocol specifications coming out of the IETF is obviously very
important, particularly for protocols that are gaining significant use.

As you may have seen, the IESG and the community has been wondering if
there'd be something that we could do about the IETF process, in the
sense that there's quite many things happening at the very end of the
document's life cycle. This results in some surprises, and often also
moves some important decisions out of the working group and to
author/shepherd/AD hands. A while ago we met for the IESG retreat and
wanted to experiment with three specific changes:

- sending work back to the WGs when significant changes are needed, and
making the WG the central place of the edits
- moving some directorate reviews earlier, without impact reviewer
effort too much
- inviting some of the shepherds onto tele chats

I am writing to you in order to discuss the second item. How big of a
change would it be to have Gen-ART reviews be invoked during WGLC, while
the documents are still in the working groups? The goal of the reviews
would still be the same, e.g., I would be checking that the reviews were
positive and/or that the issues brought up have been properly addressed.

There are important details to consider, however, and I would like to
get your feedback on how you would seem them having an effect, and what
would be the best way to organise this, if we decide to go ahead with
the change for the Gen-ART.

Triggering the review would have to be done by something else than IETF
last call announcement. Is the best approach is to have the WG chairs
manually request for this? Note that sometimes there are multiple WGLCs.
I presume that it would be preferable to have a Gen-ART review be done
only once at this stage, as otherwise the work load would increase. The
chairs may have some idea of whether they are likely to need another
WGLC before they start one.

There may be possibly bigger changes and time lag between the first
Gen-ART review and the one that checks that the changes are ok.

Some specs may not make it through WGLC, and a review at that stage may
increase the effort you guys are putting in, by reviewing those specs
that will fail.

Jari

---

From: Tero Kivinen <[email-masked]>
Date: Fri, 24 May 2013 15:19:32 +0300

> What do you think about this?

I think one of the problems is that lately the IETF LC end time, and
Telechat have been VERY close to each other. There are lots of
documents coming in lately, where the IETF last call end date and the
telechat date are same (for example draft-ietf-dhc-dhcpv6-radius-opt
in last assignment) or there has been only one or two weeks between
them (draft-ietf-ipsecme-dh-checks, draft-ietf-softwire-public-4over6
in last assignment).

This makes it so that if reviewer finds some problems, there is no
time to fix them before telechat. It used to be so that there was much
bigger difference between the IETF LC and Telechat, and in that case
there was even useful to do re-review at telechat time. Now there is
no point to assign document to be re-reviewd at telechat time, as it
was reviewed week or two earlier in the IETF LC time. 

> positive and/or that the issues brought up have been properly

That would make it easier to address bigger issues, especially as the
WG would at that point be more wiling to change things. Sometimes it
seems that suggesting some bigger changes week or two before the
telechat time, is not going to be accepted well by the document
authors.

> WGLC before they start one.

The review tool currently takes the
http://datatracker.ietf.org/feed/last-call/ rss feed and uses that to
find the last calls. If there would be similar feed for WGLCs that
would work fine from the tool point of view (I am not sure if there is
anything like that now).

We would still need to follow IETF last call status as not all
documents go through WGLC.

The review tool already supports quite easy way for secretary to add
new documents to the list, i.e. WG can request earlier review and the
document will be added to the tool for review at that point. The
review tool also supports delegating the permission to add new
documents to the review queue, so area-directors or selected members
of the review team could add documents themselves.

The next question would be how many reviews we need. Currently we
officially do two, one for IETF LC, and re-review at telechat time,
but quite often there is no need to do the second review: reviewer
already said document is ready, or his changes were already done. In
those case I will not re-assign it for re-review (I usually do check
the diffs between the last reviewed version and current version).
-- 
[email-masked]

---

From: Stephen Farrell <[email-masked]>
Date: Fri, 24 May 2013 13:30:27 +0100

Thanks Tero

That's useful info about the mechanics.

Also good to know you reckon WGLC might be a feasible
time for secdir reviews.

Others?

Cheers,
S.

---

From: Jari Arkko <[email-masked]>
Date: Fri, 24 May 2013 16:24:10 +0300

Thanks for your thoughts on this, Tero.

Jari

---

From: Yaron Sheffer <[email-masked]>
Date: Fri, 24 May 2013 17:09:10 +0300

I disagree with Tero. I think it would be a waste of my time to review 
drafts going into WGLC as a general process, since they are often not 
baked enough at this stage.

I would definitely honor requests from WG chairs to do a secdir review 
at the WGLC stage, but this should be the exception.

Thanks,
	Yaron

On 2013-05-24 15:30, Stephen Farrell wrote:
> _______________________________________________
> secdir mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/secdir
> wiki: http://tools.ietf.org/area/sec/trac/wiki/SecDirReview
>

---

From: Paul Hoffman <[email-masked]>
Date: Fri, 24 May 2013 08:15:40 -0700

On May 24, 2013, at 7:09 AM, Yaron Sheffer <[email-masked]> wrote:

> I disagree with Tero. I think it would be a waste of my time to review drafts going into WGLC as a general process, since they are often not baked enough at this stage.
> 
> I would definitely honor requests from WG chairs to do a secdir review at the WGLC stage, but this should be the exception.

And I disagree with my co-chair. :-)

The primary purpose of secdir reviews is to review the security properties of the document. The fact that many secdir reviewers feel a need to comment on other aspects is fine, but not part of the primary goal.

The security properties of a document are generally fairly established by WG LC. That seems like a fine time to tell if there are problems.

Historically, area reviews tell the IETF if there are issues with a document. It is more useful (if possible) to tell a WG of the problems before the IETF-wide review.

--Paul Hoffman

---

From: Yoav Nir <[email-masked]>
Date: Fri, 24 May 2013 15:33:41 +0000

> Historically, area reviews tell the IETF if there are issues with a document. It is more useful (if possible) to tell a WG of the problems before the IETF-wide review.

I guess this depends on the working group. Where WGLC is used to get the participants to read the document for the first time, it can be half-baked. And in that case the WGLC comments may include things like, "we need to add this feature and that capability", or "let's allow a non-authenticated mode (because PKI is too onerous)". So in that case the security properties may change.

But when WGLC are use for their intended purpose, I agree that this should not be a problem.

Yoav

---

From: Jari Arkko <[email-masked]>
Date: Fri, 24 May 2013 18:59:11 +0300

I feel bad about getting you guys all disagree with each other :-)

But back to substance. There are obvious situations where WGLC is done on a document that is far from ready. It comes as a surprise, or the WG chairs are doing it on purpose to solicit reviews that do not appear to be forthcoming otherwise. While early help with half-finished documents can be useful, I don't think we should waste secdir, Gen-ART, etc. reviewer's time on it at that point. When we talked about this in the IESG, it was brought up that the WG chairs should be involved in the decision to call for review. I agree with that. There are details to think about, however. Perhaps we can request review only for docs that the chairs believe are well baked, and use the old process for others, for instance.

Jari

---

From: Paul Hoffman <[email-masked]>
Date: Fri, 24 May 2013 09:16:24 -0700

> I feel bad about getting you guys all disagree with each other :-)

If WG co-chairs always agreed, you would only need one chair per WG. :-)

> But back to substance. There are obvious situations where WGLC is done on a document that is far from ready. It comes as a surprise, or the WG chairs are doing it on purpose to solicit reviews that…

Maybe. But I also think that if the WG chair knows that the WG is going to get all their out-of-area reviews when they press the "WG LC" button, they might do a "pre-LC" call for more reviewers. This is a topic that Yaron and I are actively working on in our less-than-active WG.

A possibility for what you want is a new state or sub-state of "outside reviews requested" that the WG chair can put in the tracker. I bet this means that most chairs will forget to do this and Tero won't hear about most documents until they are in IETF LC, so I'm hesitant to propose this.

--Paul Hoffman

---

From: Yaron Sheffer <[email-masked]>
Date: Fri, 24 May 2013 20:35:15 +0300

> A possibility for what you want is a new state or sub-state of "outside reviews requested" that the WG chair can put in the tracker. I bet this means that most chairs will forget to do this and Ter…
And where's the harm? If the WG wants to speed up a particular document, 
AND they feel it is ready for external review, they will "push the 
button". Otherwise, why not have the secdir review during IETF LC?

Thanks,
	Yaron