[Catalog-sig] Please turn off ratings

"Martin v. Löwis" martin at v.loewis.de
Wed Apr 6 01:45:13 CEST 2011

> I was talking with the people at Yelp, the ratings company in San Francisco
> this March.  They are crowdsourcing experts.  And they say that there is
> a particular problem with people giving ratings when they don't also come
> with a review.

I certainly agree with that, and that's why the PyPI rating system had
the comments feature from the beginning. There was objection to that,
on reasonable grounds, so it's opt-out now. I would certainly agree
that it better wasn't opt-out, but alas, package authors disagree. It's
their choice.

> Thus we have a situation here where we are in more desparate need of
> a way to check that the ratings are fair, no resources to do so, and,
> given than people give votes without needing to state a reason or
> an explanation for their number, no way to check that the rating is
> fair.

I believe that anybody remotely familiar with the internet is aware
of this. *Of course* ratings may be misleading, and I'll use some
software anyway despite everybody telling me that Apache is crap
and Nginx is the one and only true web server that any sane person
would use.

> All we have done is undermined the credibility of 
> http://pypi.python.org/pypi  - yes there is a rating system, but there
> is no quality control of the raters, so its all pretty meaningless.

User's aren't so dumb to not know that. They will read the reviews if
present, and the rebuttals of the reviews if present. If there are
overwhelmingly negative reviews, and no comments even though the
reviewers could have commented, they assume gaming. If there are
overwhelmingly negative reviews and commenting disabled, they assume
the author has something to hide. If there is a single negative review,
they download Django, anyway.

> So I think that the rating system is a serious disservice to the people
> it was supposed to help, the users who asked for it, as well as being
> a source of considerable angst for the software developers.

I know that you believe that, and I still disagree.


More information about the Catalog-SIG mailing list