[Catalog-sig] Package comments

[Laura Creighton]

I think that a rating system -- so many stars -- can only be effective
if there are hundreds, perhaps thousands of raters.  Otherwise the
variation between 'what I think is worth 5 stars' and 'what
somebody else does' is too large to make even non-trollish
responses meaningful.  I am thinking about the rating system that
amazon has for its books.

I find it useful to read the reviews, sometimes, but never the number
of stars.  Practically every technical book that I consider excellent
has some readers who rank it poorly; while many books I consider
rubbish get high marks.  Some of this reflects my personal taste in
technical books 'give me a concise summary rather than an
encyclopedia, please' but mostly this reflects how well or poorly the
goals the author of the book had when writing the book fit my goaks in
reading the book.  The weirder the thing is that I wanted to do, the
less useful, in general, I found the book, as is only to be expected.

There are two great problems I see with a perfectly working rating
system.  The first is that people tend to use it instead of developing
their own judgment.  If there are X packages out there for doing
something, one can sympathise with a desire to know 'which one is
the best' -- but it is rare that you can get an answer to this
question outside of the context of 'and what do you intend to do
with it?'.  So-and-so is simpler, such-and-such is more complete,
this one runs very quickly but eats memory like a pig, while that
one is tiny but slow; which one you prefer depends on what you are
doing.

The second problem is related to the first.  If, by some chance, one
of the X packages gets great reviews by its first users, it will look
better than its alternates to the casual browser, who will then
choose it.  Thus you can generate momentum rather easily this way,
even when the preferred package is arguably one of the weakest of 
the choices.  Conversely, a great package that has been reviewed
by some exacting people -- or some people who tried to use it
for something for which it was not suited -- may be skipped over
in favour of an inferior package.

I don't think that anything can substitute for developing one's own
judgement, and mechanisms that attempt to do so should be resisted
because they damage all of us in the long run.

Laura