[Python-Dev] Stability Metrics [was Stability and Change]
Guido van Rossum
Thu, 11 Apr 2002 10:10:51 -0400
> > I've done this ever since whatsnew21. My numbers, surely
> > underestimates, are:
> > 2.1: 117 patches, 136 bugs
> > 2.2: 527 patches, 683 bugs
> > 2.2.1: 139 patches, 143 bugs
> Whoa there! The conversation is now trending toward bugs
> as a metric of merit rather than demerit. Of course, fixing
> bugs is good, but not having them in the first place is better.
> Before choosing a publicity metric, consider the significance
> of the metric if it is very large or very small. In the above
> example, 0 bugs would likely indicate that no maintenance
> is taking place. Having 10,000 bugs would indicate that
> the release process was a disaster.
> Whatever metric is choosen (and I DO think having a stability
> metric is good), it should have a clear interpretation that
> a higher number is good and a lower number is bad or vice-versa.
I think it's nice to keep track of this for ourselves (despite the
interpretational problems). But I'm strengthening my position about
the use of statistics in communicating to the users: I think that
would be bad. Statistics can easily be misused, misinterpreted, and
--Guido van Rossum (home page: http://www.python.org/~guido/)