RE: Questions on the Draft

[Burke Chris-CCB007 <Chris.Burke@motorola.com>]

Certainly. As software became important in engineering, many large manufacturing corporations decided that metrics were needed to quantify the "quality" or "goodness" of software. The metrics were developed using a manufacturer's view of the world, which suggested that statistical methods could be applied: count the number of product defects found, divide by the number of "things" manufactured, and set a target of, say 3 defects per million (so called "six sigma").

But in the software world, what is the "thing" manufactured? A software defect, when present, occurs in every radio, every toaster, so that couldn't be used as the denominator. Instead, many companies decided to use something they *could* count, that was loosely related to the work of programmers: the number of lines of source code in the program. Thus, the bigger the program, the more defects it could have and still be considered "good enough."

They had to measure something. Lines of code was easy to measure, and it had a loose correlation to programmer's work product. But, on the balance it was a silly metric. One defect in a million lines of code is enough to crash a satellite. 10 defects in a 20,000 line game don't seriously take away from players' enjoyment. A cunning programmer could use sub-optimal algorithms to increase the size of his code, so as to perform better on the metric although actually making the software less "good". The problem: the metric didn't accurately express the value of the software.

Similarly, per-packet billing is relatively easy to measure, it has a loose correllation to things the end user and carrier care about, but it's not a good measure of the value delivered to the user. Per-packet billing will seem arbitrary to users (just defects per line of code seemed arbitrary both to programmers, and to end-users of software).

Chris

-----Original Message-----
From: Atsushi Takeshita [mailto:takeshita@dcl.docomo-usa.com]
Sent: Wednesday, June 27, 2001 9:48 PM
To: Burke Chris-CCB007; more@ops.ietf.org
Subject: Re: Questions on the Draft


Chris,

> I understand the appeal of per-packet pricing. But to me, it makes about
as much sense as measuring software quality based on defects per line of
code - something done not because it makes sense, or because it correlates
to value, but because its easy and feels "quantitative".

I don't understand what you mean by "measuring software
quality based on defects per line of code".
Could you please explain more?

------------------------------------------
TAKESHITA, Atsushi
DoCoMo Communications Laboratories USA, Inc.
takeshita@dcl.docomo-usa.com
Tel: 408-451-4705 / Fax: 408-573-1090
------------------------------------------