Re: I-D ACTION:draft-zinin-early-review-00.txt

[Pekka Savola <pekkas@netcore.fi>]

On Mon, 27 Oct 2003 Internet-Drafts@ietf.org wrote:
> 	Title		: Area Review Teams for Early Cross-functional Reviews
> 	Author(s)	: A. Zinin
> 	Filename	: draft-zinin-early-review-00.txt
> 	Pages		: 11
> 	Date		: 2003-10-27
>
> 	This document contains a proposal for cross-functional IETF review
> process that can be initiated at early stages of a document life cycle.
> The approach is based on existing experience with area directorates and
> other expert groups within the IETF.

I like this.  Similar to sirs, which hasn't yielded many reviews but maybe 
been a success otherwise, but more formalized and attractive (I hope).  
Note that a problem of SIRS is folks actually have little time for 
reviews -- it's difficult ot balance the level of sufficient review skills 
and having the time to actually do them...

Btw, can I call these people ARTists? :-)

A few more comments...

1) Just an idea: would it be useful to execute something like ART on different
stages of chartering?   A real problem I've noticed is that sometimes it
would have been much better not to charter the work at all, or frame it
differently, than trying to fix documents one by one.  Caveat: this isn't
really needed, as the different chartering processes are already open to the
community input, and are not such long term efforts that there would be a
dire need for "early input" on chartering.

2) Note that the 2 weeks time frame is pretty tight e.g. during the holiday
seasons -- e.g., consider a WG chair requesting ART review on Dec 18th or
the like?

3) One particular place where I'd concentrate in applying early review is
before/immediately-after a document is adopted as a WG item.  That is a
critical step in the lifetime of a document; it has reached sufficient
maturity, and should be readable and reasonable enough that the number of
reviews doesn't jump up to the sky.  WG last call, IETF last call, etc. are
both pretty late in the process -- if one wants to avoid "late surprises",
the WG adoption is about the last good chance to do that!  Of course, if
reducing IESG load is more important, the others are fine as well.



semi-substantial
----------------

     o    Operations directorate (ops-dir) has also been used for early
          document review and during the IESG review period.

==> my observation from a relatively short time on ops-dir is that it's
performing only IESG reviews, not early review as such.


   When a document needs to be reviewed by ART, the AD assigns two ART 
   members as "token holders". All ART members are encouraged to review
   the document, however, the token holders are held responsible for
   providing comments within a 2-week time frame and following up on   
   them with the document authors and/or the hosting WG. The token hold-
   ers will also provide the ADs with their recommendation including the
   summary of the discussion, the list of issues and how they have been
   addressed.

==> assigns how?  I guess the ART members should have some say in what
they should be reviewing as well.. :-)

     7.   If a document returns to an ART (e.g., the document is under 
          the IETF Last Call and was reviewed during the WG Last Call),
          the same token holders will "own" the document whenever possi-
          ble. The token holders check that the new revision of the doc-
          ument reflects the previous discussion correctly. If no addi- 
          tional concerns arise, the recommendation to the ADs of the   
          ART remain the same.

==> note that in some cases, it may also make sense to have new ARTists ;-)
take a look at the document at different stages.  That should ensure that
the reviews don't get "blind" to the document, and fresh views are ensured
at the different stages of review.

editorial
---------

==> many periods, capital chars, etc. missing especially in the comments
section.

          bers, which will have bearings on the method of ART member

==> s/bearings/bearing/

         The above gives the ADs the ability to insist on fixing cer-  
          tain comments that they believe represent serious issues if
          they were discarded while processing the cross-area review
          feedback during the WG process as described above. 

==> only cross-area, no intra-area?

   Consultation with the AD is a sanity check to make sure the set of   
   engage ARTs is chosen right

==> s/engage/engaged/

   For WG documents this is ensures by the WG chairs who keep track of
 
==> s/ensures/ensured/ ?

   members (hire more members or fire ill-performing ones) to maintain
   adequacy of the review process and require level of off-loading.

==> s/require/required/


-- 
Pekka Savola                 "You each name yourselves king, yet the
Netcore Oy                    kingdom bleeds."
Systems. Networks. Security. -- George R.R. Martin: A Clash of Kings