[Python-Dev] PyPI comments and ratings, *really*?

[Robert Kern]

[We really should be discussing this on catalog-sig, but moving things 
mid-thread never works, so here we go. I apologize to python-dev. I also 
apologize for the length.]

On 2009-11-13 17:18 PM, Steven D'Aprano wrote:
> On Sat, 14 Nov 2009 09:57:18 am Ben Finney wrote:
>> "A.M. Kuchling"<amk at amk.ca>  writes:
>>> If popular vote is ruled out, I don't see who else could possibly
>>> make the decision to disable comments and/or ratings.
>>
>> Reasoned argument with the person who decides. A bad idea with many
>> people's support is still a bad idea; a good idea with few people's
>> support is still a good idea.
>
> Okay, let's talk reasoned debate.
>
> I understand the reason for making comments compulsory: they're for the
> benefit of the users, not the package owner. It helps prevent
> information about the package from being fragmented: there is One
> Obvious place to find out about a package on PyPI, which is the PyPI
> page, instead of having to search for blogs where people may or may not
> have made comments about the package. If individual package owners
> don't like this, too bad, because PyPI is run for the benefit of the
> community, not individual package owners.
>
> I understand the reason for making comments optional: personal choice of
> the package owner is valuable in and of itself, even if it is against
> the best interests of the community.
>
> But for the life of me, I can't understand the 1/3 of the votes that
> have been cast in favour of prohibiting comments for everybody, even
> those who want comments.

While I do have a couple of packages on PyPI, I use PyPI as a consumer of 
packages much more frequently, every day in fact. I voted against comments and 
ratings because I think it is a detriment to my experience of PyPI as a user (I 
also think that they will be a detriment to the community, but that's a 
prediction, not a fact). Short form comments are simply not useful to me. 
Ratings are worse. They do not carry reliable information; they carry short 
statements of opinion from a wide variety of experiences, most of which are 
entirely unrelated to my own needs.

To make an example, I have a lot of experience making ornery stuff build. A lot 
of other people don't. Their personal experience of not managing to install a 
package correctly turns into a weird, objective-seeming statement that the 
"package is difficult to build". People have different thresholds, different 
experiences, and different standards.

When such opinions get numerically aggregated by a monolithic rating system, 
that's even worse. Even with short-form comments, they have the ability, though 
often not the propensity, to give a little bit of information ("I had trouble 
building it on Windows.") that can help me tease out whether their experiences 
will be relevant to mine, but one star is just one star.

I *do* like to read long-form reviews where people relate what their needs were, 
what they tried to use the package for, and exactly where the package succeeded 
and where it failed. I can compare that context with my own needs and extract 
the relevant parts of their experience. Blogs are great for that.

Now I do appreciate ratings and comments on shopping sites if they don't provide 
the capability for long-form reviews. But that's because the product is locked 
behind the barrier of payment and often shipping. There is no such barrier on 
PyPI. If I can get to a web view of their repository, thirty seconds with it 
will give a much better idea of whether the package is worth trying than any 
amount of comments I could read in that time. I can easily see how much 
documentation they have, if they have funny build requirements, if they are just 
prototyping, etc. without needing to trust that everyone else has needs and 
standards like mine. That's the key difference between comments and ratings on 
shopping sites and why I don't think that their success in that field translates 
to here.

If you want one idea that would make my use of PyPI much more pleasant and 
informative, it would be to add a "Repository-URL" entry to the recommended PyPI 
metadata so that I could always start looking at the code in one click. Or 
integrate the source code browsing into PyPI itself; it could open up the sdists 
and eggs and show a WebVCS-like browser interface.

Now, these are all reasons why commenting and rating are not beneficial to me. 
Where I think they are harmful is that when one is exposed to them, one cannot 
just ignore them. They have an emotional, unreasonable impact whether I want 
them to or not. And that's why I do not want to see them. Give me access to 
information, not opinions. If authors do want comments and ratings on their 
packages, let them use the services that already exist for *just* that purpose 
like Ohloh. They have the time and energy to implement these mechanisms with the 
care and attention that they need.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco