[Distutils] Beyond wheels 1.0: helping downstream, FHS and more

Donald Stufft donald at stufft.io
Tue Apr 14 04:29:22 CEST 2015

> On Apr 13, 2015, at 10:17 PM, Robert Collins <robertc at robertcollins.net> wrote:
> On 14 April 2015 at 09:35, David Cournapeau <cournape at gmail.com> wrote:
> ...
> One of the earlier things mentioned here - {pre,post}{install,remove}
> scripts - raises a red flag for me.
> In Debian at least, the underlying system has the ability to run such
> turing complete scripts, and they are a rich source of bugs - both
> correctness and performance related.
> Nowadays nearly all such scripts are machine generated from higher
> level representations such as 'this should be the default command' or
> 'configure X if Y is installed', but because the plumbing is turing
> complete, they all need to be executed, which slows down
> install/upgrade paths, and any improvement to the tooling requires a
> version bump on *all* the packages using it - because effectively the
> package is itself a compiled artifact.
> I'd really prefer it if we keep wheels 100% declarative, and instead
> focus on defining appropriate metadata for the things you need to
> accomplish {pre,post}{install,remove} of a package.

A possible way to implement {pre,post}{install,remove} scripts is to
instead turn them into extensions. One example is that Twisted uses
a setup.py hack to regenerate a cache file of all of the registered
plugins. This needs to happen at install time due to permission issues.
Currently you can’t get this speedup when installing something that
uses twisted plugins and installing via Wheel.

So a possible way for this to work is in a PEP 426 world, simply define
a twisted.plugins extension that says, in a declarative way, “hey when
you install this Wheel, if there’s a plugin that understands this extension
installed, let it do something before you actually move the files into
place”. This let’s Wheels themselves still be declarative and moves the
responsibility of implementing these bits into their own PyPI projects
that can be versioned and independently upgraded and such. We’d probably
need some method of marking an extension as “critical” (e.g. bail out and
don’t install this Wheel if you don’t have something that knows how to handle
it) and then non critical extensions just get ignored if we don’t know
how to handle it.

Popular extensions could possibly be added directly to pip at some point
if a lot of people are using them (or even moved from third party extension
to officially supported extension).

Donald Stufft
PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20150413/41c621db/attachment.sig>

More information about the Distutils-SIG mailing list