Re: [Solutions] Re: I-D ACTION:draft-zinin-early-review-00.txt

[Alex Zinin <zinin@psg.com>]

Pekka,

Thanks for reading the doc! See inline below, pls.

> I like this.  Similar to sirs, which hasn't yielded many reviews but maybe
> been a success otherwise, but more formalized and attractive (I hope).
> Note that a problem of SIRS is folks actually have little time for 
> reviews -- it's difficult ot balance the level of sufficient review skills
> and having the time to actually do them...

> Btw, can I call these people ARTists? :-)

Whatever makes people more interested in doing more reviews ;)

> A few more comments...

> 1) Just an idea: would it be useful to execute something like ART on different
> stages of chartering?   A real problem I've noticed is that sometimes it
> would have been much better not to charter the work at all, or frame it
> differently, than trying to fix documents one by one.  Caveat: this isn't
> really needed, as the different chartering processes are already open to the
> community input, and are not such long term efforts that there would be a
> dire need for "early input" on chartering.

Yes, the architectural review that IAB and IESG does for BOFs and new
WGs is already happening, but I don't see why we couldn't ask ARTs for
their review in this process. At the end, it should also help scale
the architectural review function.

> 2) Note that the 2 weeks time frame is pretty tight e.g. during the holiday
> seasons -- e.g., consider a WG chair requesting ART review on Dec 18th or
> the like?

Yep, I got this comments from another person as well. I'll change it
to be the default that can be changed on a case-by-case basis.

> 3) One particular place where I'd concentrate in applying early review is
> before/immediately-after a document is adopted as a WG item.  That is a
> critical step in the lifetime of a document; it has reached sufficient
> maturity, and should be readable and reasonable enough that the number of
> reviews doesn't jump up to the sky.  WG last call, IETF last call, etc. are
> both pretty late in the process -- if one wants to avoid "late surprises",
> the WG adoption is about the last good chance to do that!  Of course, if
> reducing IESG load is more important, the others are fine as well.

Agreed, I'll add this to the next rev.

> semi-substantial
> ----------------

>      o    Operations directorate (ops-dir) has also been used for early
>           document review and during the IESG review period.

==>> my observation from a relatively short time on ops-dir is that it's
> performing only IESG reviews, not early review as such.

I had at least a couple of cases where I got reviews from ops-dir
at the pre-IESG stage...

>    When a document needs to be reviewed by ART, the AD assigns two ART
>    members as "token holders". All ART members are encouraged to review
>    the document, however, the token holders are held responsible for
>    providing comments within a 2-week time frame and following up on
>    them with the document authors and/or the hosting WG. The token hold-
>    ers will also provide the ADs with their recommendation including the
>    summary of the discussion, the list of issues and how they have been
>    addressed.

==>> assigns how?  I guess the ART members should have some say in what
> they should be reviewing as well.. :-)

I used the "running code" for rtg-dir, but if you have a suggestion on
how to better assign the reviewers, I'd be very interested. My
experience shows that explicit assignment by ADs works much better
than the self-pickup method where each reviewers decides if he needs
to review the referred doc or not.

>      7.   If a document returns to an ART (e.g., the document is under
>           the IETF Last Call and was reviewed during the WG Last Call),
>           the same token holders will "own" the document whenever possi-
>           ble. The token holders check that the new revision of the doc-
>           ument reflects the previous discussion correctly. If no addi-
>           tional concerns arise, the recommendation to the ADs of the
>           ART remain the same.

==>> note that in some cases, it may also make sense to have new ARTists ;-)
> take a look at the document at different stages.  That should ensure that
> the reviews don't get "blind" to the document, and fresh views are ensured
> at the different stages of review.

Note that the fact that 2 ARTists are assigned to a doc does not mean
that other ART members should not review it. More diverse reviews are
better at the early stages. On the other hand, the IESG experience
shows that changing reviewers later in the process is usually
frustrating.


> editorial
> ---------

==>> many periods, capital chars, etc. missing especially in the comments
> section.

Thanks. I wrote it on the plane, rushing to put my thoughts down :)
I'll fix these.

Alex