BMWG Charter Revision

[Randy Bush <randy@psg.com>]

<boilerplate unchanged>

Description of Working Group:

The major goal of the Benchmarking Methodology Working Group is to make
a series of recommendations concerning the measurement of the
performance characteristics of various internetworking technologies;
further, these recommendations may focus on the systems or services that
are built from these technologies.

Each recommendation will describe the class of equipment, system, or
service being addressed; discuss the performance characteristics that
are pertinent to that class; clearly identify a set of metrics that aid
in the description of those characteristics; specify the methodologies
required to collect said metrics; and lastly, present the requirements
for the common, unambiguous reporting of benchmarking results.

To better distinguish the BMWG from other measurement initiatives in the
IETF, the scope of the BMWG is limited to technology characterization
using simulated stimuli in a laboratory environment. Said differently,
the BMWG does not attempt to produce benchmarks for live, operational
networks.  Moreover, the benchmarks produced by this WG shall strive
to be vendor independent or otherwise have universal applicability
to a given technology class.

Because the demands of a particular technology may vary from
deployment to deployment, a specific non-goal of the Working Group is
to define acceptance criteria or performance requirements.

An ongoing task is to provide a forum for discussion regarding the
advancement of measurements designed to provide insight on the operation
internetworking technologies.


Goals and Milestone

Done  Expand the current Ethernet switch benchmarking methodology
       draft to define the metrics and methodologies particular to
       the general class of connectionless, LAN switches.
Done  Edit the LAN switch draft to reflect the input from BMWG.
       Issue a new version of document for comment. If appropriate,
       ascertain consensus on whether to recommend the draft for
       consideration as an RFC.
Done  Take controversial components of multicast draft to mailing
       list for discussion. Incorporate changes to draft and reissue
       appropriately.
Done  Submit workplan for initiating work on Benchmarking
       Methodology for LAN Switching Devices.
Done  Submit workplan for continuing work on the Terminology for
       Cell/Call Benchmarking draft.
Done  Submit initial draft of Benchmarking Methodology for LAN
       Switches.
Done  Submit Terminology for IP Multicast Benchmarking draft for AD
       Review.
Done  Submit Benchmarking Terminology for Firewall Performance for AD
       review
Done  Progress ATM benchmarking terminology draft to AD review.
Done  Submit Benchmarking Methodology for LAN Switching Devices
       draft for AD review.
Done  Submit first draft of Firewall Benchmarking Methodology.
Done  First Draft of Terminology for FIB related Router Performance
       Benchmarking.
Done  First Draft of Router Benchmarking Framework
Done  Progress Frame Relay benchmarking terminology draft to AD
       review.
Done  Methodology for ATM Benchmarking for AD review.
Done  Terminology for ATM ABR Benchmarking for AD review.
Done  Terminology for FIB related Router Performance Benchmarking to
       AD review.
Done  First draft Net Traffic Control Benchmarking Methodology.
Done  First Draft of Methodology for FIB related Router Performance
       Benchmarking.
Done  Firewall Benchmarking Methodology to AD Review
MAR 03  Methodology for IP Multicast Benchmarking to AD Review.
MAR 03  Resource Reservation Benchmarking Terminology to AD Review
MAR 03  First I-D on IPsec Device Benchmarking Terminology
APR 03  Net Traffic Control Benchmarking Terminology to AD Review
APR 03  OSPF Convergence Benchmarking Terminology to AD Review
APR 03  OSPF Convergence Benchmarking Methodology to AD Review
APR 03  OSPF Convergence Benchmarking Applicability to AD Review
APR 03  EGP Convergence Benchmarking Terminology to AD Review
APR 03  Methodology for FIB related Router Performance Benchmarking
         to AD review.
JUL 03  Resource Reservation Benchmarking Methodology to AD Review
JUL 03  Basic BGP Convergence Benchmarking Methodology to AD Review.
DEC 03  Net Traffic Control Benchmarking Methodology to AD Review
DEC 03  IPsec Device Benchmarking Terminology to AD Review