Stating a new thread with a correct title.
On 2 Jul 2023, at 10:12, Paul Moore <
p.f.moore@gmail.com> wrote:
Unfortunately, too much of this discussion is framed as “someone should”, or “it would be good if”. No-one is saying “I will”. Naming groups, like “the PyPA should” doesn’t help either - groups don’t do things, people do. Who in the PyPA? Me? Nope, sorry, I don’t have the time or interest - I’d *use* a curated index, I sure as heck couldn’t *create* one.
Well, I started this topic, and I don't *think* I ever wrote "someone should", and I certainly didn't write "PyPa should".
But whatever I or anyone else wrote, my intention was to discuss what might be done to address what I think is a real problem/limitation in the discoverability of useful packages for Python.
And I think of it not so much as "someone should" but as "it would be nice to have".
Of course, any of these ideas would take a lot of work to implement -- and even though there are a lot of folks, on this list and elsewhere, that would like to help, I don't think any substantial open-source project has gotten anywhere without a concerted effort by a very small group (often just 1) of people doing a lot of work to get it to a useful state before a larger group can contribute. So I"m fully aware that nothings going to happen unless *someone* really puts the work in up front. That someone *might* be me, but I'm really good at over-committing myself, and not so great at keeping my nose to the grindstone, so ....
And I think this particular problem calls for a solution that would have to be pretty well established before reaching critical mass to actually be useful -- after all, we already have PyPi -- why go anywhere else that is less comprehensive?
All that being said, it's still worth having a conversation about what a good solution might look like -- there are a lot of options, and hashing out some of the ideas might inspire someone to rise to the occasion.
The :problem", as I see it.
- The Python standard library is not, and will never be fully comprehensive -- most projects require *some* third party packages.
- There are a LOT of packages available on PyPi -- with a very wide range of usefulness, quality and maintenance -- everything from widely used packages with a huge community (e.g. numpy) to packages that are release 0.0.1, and never seen an update, and may not even work.
So the odds that there's a package that does what you need are good, but it can be pretty hard to find them sometimes -- and can be a fair bit of work to sift through to find the good ones -- and many folks don't feel qualified to do so.
This can result in two opposite consequences:
1) People using a package that really isn't reliable or maintained (or not supported on all platforms, or ..) and getting stuck with it (I've had that on some of my projects -- I'm pretty careful, but not everyone on my team is)
2) People writing their own code - wasting time, and maybe not getting a very good solution either. I've also had that problem on my projects...
To avoid this -- SOME way for folks to find packages that have at least has some level of vetting would be good -- exactly what level of vetting, is a very open question, but I think "even a little" could be very helpful.
A few ideas that have come up in the previous thread -- more or less in order of level of effort.
1) A "clearing house" of package reviews, for lack of a better word -- a single web site that would point to various resources on the internet -- blog posts, etc, that would help people select packages. So one might go to the "JSON: section, and find links to some of the comparisons of JSON parsers, to help you choose. The authors of this site could even solicit folks to write reviews, etc that they could then point to.
Chris A: this is my interpretation of your decentralized idea -- please do correct me if I got it wrong.
2) A Python package review site. This could be setup and managed with, e.g. a gitHub repo, so that there could be a small number of editors, but anyone could contribute a review via PR. The ultimiate goal would be reviews/recommendations of all the major package categories on PyPi. If well structured and searchable, this could be very useful.
- This was proposed by someone on the previous thread -- again, correct me if I'm wrong.
3) A rating system built into PyPi -- This could be a combination of two things:
A - Automated analysis -- download stats, dependency stats, release frequency, etc, etc, etc.
B - Community ratings -- upvotes. stars, whatever.
If done well, that could be very useful -- search on PyPi listed by rating. However -- :done well" ios a huge challenge -- I don't think there's a way to do the automated system right, and community scoring can be abused pretty easily. But maybe folks smarter than me could make it work with one or both of these approaches.
4) A self contained repository of packages that you could point pip to -- it would contain only the packages that had met some sort of "vetting" criteria. In theory, anyone could run it, but a stamp of approval from the PSF would make it far more acceptable to people. This would be a LOT of work to get set up, and still a lot of work to maintain.
Personally, I think (4) is the best end result, but probably the most work as well[*], so ???
(1) and (2) have the advantage that they could be useful even without being comprehensive -- tthey's need to have some critical mass to get any traction, but maybe not THAT much.
Anyway, I'd love to hear your thoughts on these ideas (or others?) - both technical and social.
And of course, if anyone wants to join forces on getting something going.
-CHB
[*] maybe not, actually -- once the infrastructure was in place, adding a package would require someone requesting that it be added, and someone on a "core team" approving it. That could be a pretty low level of effort, actually. The actual mechanism would be to simply copy it from PyPi once approved -- not that hard to automate. hmmm....
Probably the biggest challenge would be coming up with the criteria for approval -- not an easy question.
And it would require substantial critical mass to be at all useful.
-- Christopher Barker, PhD (Chris)
Python Language Consulting
- Teaching
- Scientific Software Development
- Desktop GUI and Web Development
- wxPython, numpy, scipy, Cython
Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/XFJL36Y373RESCKGJ47EXQ5Q3FTDNRZB/