[Python-ideas] Package reputation system

[Chris Angelico]

On Sat, Apr 2, 2016 at 8:16 PM, Pavol Lisy <pavol.lisy at gmail.com> wrote:
> 2. Art critics are not to shame artists. Constructive criticism could
> help authors too. It is not benefit only for "customers". One who
> wants invent index to get rich could probably invent this job (Python
> critic) too.

A reputation system is simply a score, though. If you are low on the
rankings, you don't automatically know why - and you can't look at the
high ranked results to figure out what to do better. That's why the
inventor of the index gets a lot of money - the consultancy fees to
help people get higher on the index will be gladly paid, because
there's no other way to learn.

> 3. pypi has number of downloads for day, week and month. If there is
> public api to data - one could make graphs, trends, etc and public
> them. If they will be nice then they probably could be included on
> pypi site too.

That's popularity, which is different again from reputation. It
suffers from several brutal limitations:

1) By definition, it cannot count *usage*. If I download something
only to find that it completely fails me, it's still a download; if I
download something and use it every day for years, it's still only one
download.

2) It can only count downloads from the ultimate origin. If a Python
package gets included in a downstream collection such as the Debian
repositories, people who apt-get the package won't ting the stats.

3) Installations cascade into their dependencies. Does that count?
What if you 'pip freeze' and now explicitly list all those
dependencies in your requirements.txt - should they NOW count?

4) Even if you solve all of those problems, popularity begets
popularity in many ways. Two competing packages may fight for a while,
but one of them will "win", and the other languish - and it's
virtually impossible for a new competitor to get to the point where it
can be properly respected.

Choosing which thing to use based on how many other people use it is
sometimes important (the network effect - using Python rather than
Bob's Obscure Scripting Language means it's a lot easier to find
collaborators, no matter how technically superior BOSL might be), but
it's almost completely orthogonal to quality.

ChrisA