[Python-Dev] Stability Metrics [was Stability and Change]

Raymond Hettinger python@rcn.com
Thu, 11 Apr 2002 00:46:43 -0400


From: akuchlin@mems-exchange.org
> Oh, sure; it's straightforward to write a script to pull log messages
> from a given branch, and if people religiously put 'bug #NNN' in their
> log messages, count up the fixed bugs.  Handling bugs fixed per time 
> would require more script hacking, but nothing terrifying.
>
> I've done this ever since whatsnew21.  My numbers, surely
> underestimates, are:
> 
> 2.1: 117 patches, 136 bugs
> 2.2:    527 patches, 683 bugs
> 2.2.1: 139 patches, 143 bugs

Whoa there!  The conversation is now trending toward bugs
as a metric of merit rather than demerit.  Of course, fixing
bugs is good, but not having them in the first place is better.

Before choosing a publicity metric, consider the significance
of the metric if it is very large or very small.  In the above
example, 0 bugs would likely indicate that no maintenance
is taking place.  Having 10,000 bugs would indicate that
the release process was a disaster.

Whatever metric is choosen (and I DO think having a stability
metric is good), it should have a clear interpretation that
a higher number is good and a lower number is bad or vice-versa.

it-takes-an-accountant-to-notice-these-things-ly yours,


Raymond Hettinger, CPA