[Distutils] Maintaining a curated set of Python packages

Nick Coghlan ncoghlan at gmail.com
Thu Dec 15 23:18:45 EST 2016


On 16 December 2016 at 07:14, Glyph Lefkowitz <glyph at twistedmatrix.com>
wrote:

> On Dec 15, 2016, at 6:39 AM, Donald Stufft <donald at stufft.io> wrote:
> Theoretically we could allow people to not just select packages, but also
> package specifiers for their “curated package set”, so instead of saying
> “requests”, you could say “requests~=2.12” or “requests==2.12.2”. If we
> really wanted to get slick we could even provide a requirements.txt file
> format, and have people able to install the entire set by doing something
> like:
>
>     $ pip install -r https://pypi.org/sets/dstufft/
> my-cool-set/requirements.txt
>
>
> Can't people already do this by publishing a package that just depends on
> their whole 'package set'?
>

Technically, sure, but it adds a lot of overhead. The closest equivalent
right now would be maintaining a source control repo somewhere with various
requirements files in it.

However, at an ecosystem level, that doesn't have the same user experience
impact. The idea of building this into PyPI itself would be to *reshape the
learning curve of how people learn about dependency management as they're
introduced to Python*.

Going back to the CurseGaming example, I actually use the free version of
their client to manage the Warcraft addons on my gaming PC. The basic usage
model is really simple and (not coincidentally) very similar to the way the
Add-on manager works in Firefox and other GUI apps with integrated plugin
managers:

- you have an "Installed" tab for the addons you have installed
- when you start the client, it checks for updates for all your installed
addons and the out of date ones gain an "Update" button
- there's a separate tab where you can search all the available addons and
install new ones

I've never used any of Curse's other clients (like the Minecraft or Kerbal
Space Program ones), but I assume they operate in a similar way.

The paid tier of the Curse Client, and the account sync feature of Firefox,
then offer the ability to synchronize your installed addons across
machines. (There are also a lot of similarities between this model and the
way mobile app stores work)

A comparable UX for Python/PyPI/pip would focus less on the
library-and-application development cases (where the presence of source
control is assumed), and more on the ad hoc scripting and
learning-to-program use cases, where you're typically more interested in
"--user" installations and the question of which parts of the Python
ecosystem are just an import away than you are in reproducability and
maintainability.

The ecosystem level learning curve then becomes:

- did you know you can back up your list of user installed packages to PyPI?
- did you know you can use PyPI to sync your user installs between systems?
- did you know you can use PyPI to categorise your user installs and share
them with others?
- OK, now it's time to start learning about version control, virtual
environments and automated testing

It wouldn't necessarily make sense to bake this *directly* into Warehouse,
and the Mozilla folks responsible for Firefox Sync could no doubt offer
real word guidance on the infrastructure and support needed to operate a
service like that at scale, but the core concept of allowing package
management to be introduced independently of both version control and
virtual environments sounds potentially valuable to me.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20161216/47dc4b89/attachment.html>


More information about the Distutils-SIG mailing list