RE: comments on mib review guidelines 01 - draft-ietf-ops-mib-review-guidelines-XX-20030707.txt

["C. M. Heard" <heard@pobox.com>]

[ mreview added to cc list.  "the document that Mike sent me" 
  referred to below is draft-ietf-ops-mib-review-guidelines-01.txt
  with the diffs posted to the mreview list applied.  ]

On Thu, 10 Jul 2003, Wijnen, Bert (Bert) wrote:
> Mike/Juergen. I have reviewed that document that Mike sent me
> w.r.t. the "OPS Area" text. I think what is in this text is
> fine. Maybe to make things a bit clearer:
> 
> - In abstract: s/of IETF/of all IETF/

If the idea is to say that all IETF MIB documents (informational
and experimental as well as standards-track) go through such
review, then the change should be:

s/IETF standards-track specifications/all IETF specifications/

(BUT ... see my counter-proposal below ... I do not think that we
should make this change).

> - In adstract add (repeat some text from intro:
>    The IESG, always asks a MIB doctor to review an IETF specification 
>    that contains a MIB module.  These reviews are done to ensure that
>    such specifications follow established IETF documentation practices
>    and that the MIB modules they contain meet certain generally accepted
>    standards of quality, including (but not limited to) compliance with
>    all syntactic and semantic requirements of SMIv2 (STD 58) (RFC2578,
>    RFC2579, RFC2580) that are applicable to "standard" MIB modules.
>    The purpose of this memo is to document the guidelines that are
>    followed in such reviews.

That's way too much text for an abstract.  The following shorter
version would accomplish the same purpose:

   IESG policy requires OPS area review of all IETF specifications
   containing MIB modules.  This memo documents the guidelines
   used in those reviews.  Applicable portions of these guidelines
   may used as a basis for reviews of non-IETF MIB documents.

(again, though, see my counter-proposal below ... I like parts of
this, but I think the official scope should be standards-track
documents, as decided back in February).

Note that I have deliberately avoided using the term "MIB doctor"
because it is not currently used in the document and would need to
be defined before use.  Providing a definition before use would be
awkward in an abstract (which is supposed to be self-contained).

> - In 2nd line in sect 1: s/of IETF/of all IETF/
> 
> I'd have to check if IESG would be OK with that addition to the
> abstract, but I would think they would agree. 
> By the abocve 3 changes, I think it should be very clear that ALL
> MIB documents (in fact informational and experimental go 
> through this as well) are being checked at the request of the IESG.

Again, the change would be

s/IETF standards-track specifications/all IETF specifications/

if this is supposed to apply to all IETF MIB documents.

> Makes sense? If so I will check with IESG.

I'm not sure that it does.  Allow me to point out that the original
abstract in the -00 draft did not limit the scope to standards-track
documents;  rather, it said:

   This memo provides guidelines for authors and reviewers of IETF
   specifications containing MIB modules.

We changed the scope to standards-track documents deliberately,
after a discussion that took place last February.  Here are the URLs
of the e-mail messages that we exchanged (sorry, I can't point to a
single URL where the thread begins, as the it is broken up in the
thread index):

http://ops.ietf.org/lists/mreview/mreview.2003/msg00184.html
http://ops.ietf.org/lists/mreview/mreview.2003/msg00186.html
http://ops.ietf.org/lists/mreview/mreview.2003/msg00189.html
http://ops.ietf.org/lists/mreview/mreview.2003/msg00190.html
http://ops.ietf.org/lists/mreview/mreview.2003/msg00192.html
http://ops.ietf.org/lists/mreview/mreview.2003/msg00195.html
http://ops.ietf.org/lists/mreview/mreview.2003/msg00199.html
http://ops.ietf.org/lists/mreview/mreview.2003/msg00200.html

So we would be reversing a previous decision, and it would affect
other parts of the document as well.  Please review the
above-referenced mail thread and see if you really do want to
revisit that decision.

My opinion, BTW, is that that the official scope of the review
guidelines should be for standards-track documents.  They should NOT
automatically be applied _in toto_ to experimental and informational
documents, but rather that such parts of the guidelines that are
deemed appropriate MAY be applied ... and that, in fact, is exactly
what we agreed on in the above thread and is also exactly what the
document now says.

Recent discussion on the problem-statement list makes me suspect
that there would be some objections to _automatically_ applying the
same review criteria to experimental/informational documents and
standards-track documents.  And I would agree with such objections
(indeed, I would raise them myself if no-one else did).  Although
it's not done as often as it should be, the option to create an
experimental spec for testing prior to deployment is something that
I think we should have, and such specs should not necessarily have
the same high bar as standards-track documents.  The SMIv2
documents, in fact, exempt experimental MIB modules (ones that are
registered under the experimental tree) from the extra requirements
that apply to "standard" MIB modules (e.g., uniqueness of
descriptors).  I would find it objectionable to decide after the
fact that those extra requirements do apply after all.

So, if we want to tinker with the abstract to make things a little
bit clearer -- without changing the previous decision with respect
to the official scope of this document -- then my counter-proposal
to Bert's suggestions above would be to change the abstract only,
and to change it like this:

   IESG policy requires OPS area review of IETF standards-track
   specifications containing MIB modules.  This memo documents
   the guidelines used in those reviews.  Applicable portions
   of these guidelines may used as a basis for reviews of
   other documents.

> I am also starting to think that maybe this should become a BCP.
> I had seen on problem-statement list that people were complaining
> that they did not know about documented-quality-checks that the IESG
> does, and I believe that this doc is such a documented set of
> quality requirments. I have not checked with IESG if they agree.
> A BCP would cause 4 week IETF Last Call, so all IETFers can then
> also see what is being expected from them, and if we get consensus,
> then it is clear that IETF does agree with these quality checks.

I have more-or-less been assuming from the outset that this document
would be a BCP, and have attempted to write it that way.  Certainly
if it documents a practice that people are obliged to comply with
then it ought to get proper community review (regardless of whether
we call it BCP or Informational).

Now to respond to some off-list comments from Juergen:

> - "OPS area review"
> 
>   Since the policy that MIBs are reviewed is an IESG
>   policy (as the first sentence in the Introduction says),
>   I think "IESG MIB review" is more appropriate.

I don't agree, since the reviews are done in the OPS area, and
reasonably so, since that's where the expertise lies.

> Regarding my original comment, I think it all boils down to the
> following two phrases (and variations of them):
> 
> - "OPS Area MIB reviewers"
> 
>   The actual established term is "MIB doctor".  But if
>   we want a more formal term, then I would say that
>   "IESG MIB reviewers" is more appropriate since we do
>    MIB reviews for the IESG.

No, the MIB doctors are advisors to the OPS AD in charge of network
management.  See http://www.ops.ietf.org/mib-doctors.html

> Perhaps another solution would be to just talk about "MIB reviewers"
> where the text currently talks about "OPS Area MIB reviewers" and
> to talk about "MIB review" instead of "OPS area review". I actually
> like this solution since this document might very well be used by
> other organizations for their reviews. Does this make sense?

That I could live with ... but the following part of the intro

 [ ... ] the IESG instituted a policy of requiring OPS area review
 of IETF standards-track specifications containing MIB modules.

should stay as is, since it accurately reports the facts (and
similarly for the abstract, if my proposal above is accepted).
The specific changes would be:

s/current pool of OPS Area MIB reviewers/current pool of MIB reviewers/

in Section 4.2 and in Section 4.8.

Will these changes, along with the revised abstract I proposed
above, be satisfactory to everyone?

Thanks,

Mike