[Python-Dev] Stability Metrics [was Stability and Change]

Guido van Rossum guido@python.org
Thu, 11 Apr 2002 10:10:51 -0400


[AMK]
> > I've done this ever since whatsnew21.  My numbers, surely
> > underestimates, are:
> > 
> > 2.1: 117 patches, 136 bugs
> > 2.2:    527 patches, 683 bugs
> > 2.2.1: 139 patches, 143 bugs

[RH]
> Whoa there!  The conversation is now trending toward bugs
> as a metric of merit rather than demerit.  Of course, fixing
> bugs is good, but not having them in the first place is better.
> 
> Before choosing a publicity metric, consider the significance
> of the metric if it is very large or very small.  In the above
> example, 0 bugs would likely indicate that no maintenance
> is taking place.  Having 10,000 bugs would indicate that
> the release process was a disaster.
> 
> Whatever metric is choosen (and I DO think having a stability
> metric is good), it should have a clear interpretation that
> a higher number is good and a lower number is bad or vice-versa.

I think it's nice to keep track of this for ourselves (despite the
interpretational problems).  But I'm strengthening my position about
the use of statistics in communicating to the users: I think that
would be bad.  Statistics can easily be misused, misinterpreted, and
countered.

--Guido van Rossum (home page: http://www.python.org/~guido/)