[Distutils] Maintaining a curated set of Python packages

[Nick Coghlan]

On 15 December 2016 at 03:41, Chris Barker <chris.barker at noaa.gov> wrote:
[Barry wrote]
>> Ubuntu has an elaborate automated system for testing some dimension of
>> compatibility issues between packages, not just Python packages.  Debian
>> has
>> the same system but isn't gated on the results.
>
> This brings up the larger issue -- PyPi is inherently different than these
> efforts -- PyPi has always been about each package author maintaining their
> own package -- Linux distros and conda-forge, and ??? all have a small set
> of core contributions that do the package maintenance.

Fedora at least has just shy of 1900 people in the "packager" group,
so I don't know that "small" is the right word in absolute terms :)

However, relatively speaking, even a packager group that size is still
an order of magnitude smaller than the 30k+ publishers on PyPI (which
is in turn an order of magnitude smaller than the 180k+ registered
PyPI accounts)

> This is a large
> effort, and wold be insanely hard with the massive amount of stuff on
> PyPi....
>
> In fact, I think the kinda-sort curation that comes from individual
> communities is working remarkably well:
>
> the scipy community
> the django community
> ...

Exactly. Armin Ronacher and a few others have also started a new
umbrella group on GitHub, Pallets, collecting together some of the key
infrastructure projects in the Flask ecosystem:
https://www.palletsprojects.com/blog/hello/

Dell/EMC's John Mark Walker has a recent article about this
"downstream distribution" formation process on opensource.com, where
it's an emergent phenomenon arising from the needs of people that are
consuming open source components to achieve some particular purpose
rather than working on them for their own sake:
https://opensource.com/article/16/12/open-source-software-supply-chain

It's a fairly different activity from pure upstream development -
where upstream is a matter of "design new kinds and versions of Lego
bricks" (e.g. the Linux kernel, gcc, CPython, PyPI projects),
downstream integration is more "define new Lego kits using the already
available bricks" (e.g. Debian, Fedora, conda-forge), while commercial
product and service development is "We already put the Lego kit
together for you, so you can just use it" (e.g. Ubuntu, RHEL, Amazon
Linux, ActivePython, Enthought Canopy, Wakari.io).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia