[Python-Dev] Stability Metrics [was Stability and Change]
Thu, 11 Apr 2002 00:46:43 -0400
> Oh, sure; it's straightforward to write a script to pull log messages
> from a given branch, and if people religiously put 'bug #NNN' in their
> log messages, count up the fixed bugs. Handling bugs fixed per time
> would require more script hacking, but nothing terrifying.
> I've done this ever since whatsnew21. My numbers, surely
> underestimates, are:
> 2.1: 117 patches, 136 bugs
> 2.2: 527 patches, 683 bugs
> 2.2.1: 139 patches, 143 bugs
Whoa there! The conversation is now trending toward bugs
as a metric of merit rather than demerit. Of course, fixing
bugs is good, but not having them in the first place is better.
Before choosing a publicity metric, consider the significance
of the metric if it is very large or very small. In the above
example, 0 bugs would likely indicate that no maintenance
is taking place. Having 10,000 bugs would indicate that
the release process was a disaster.
Whatever metric is choosen (and I DO think having a stability
metric is good), it should have a clear interpretation that
a higher number is good and a lower number is bad or vice-versa.
Raymond Hettinger, CPA