[Distutils] Beyond wheels 1.0: helping downstream, FHS and more

David Cournapeau cournape at gmail.com
Mon Apr 13 16:54:23 CEST 2015

On Mon, Apr 13, 2015 at 10:44 AM, Donald Stufft <donald at stufft.io> wrote:

> On Apr 13, 2015, at 10:39 AM, David Cournapeau <cournape at gmail.com> wrote:
> Hi there,
> During pycon, Nick mentioned there was interest in updating the wheel
> format to support downstream distributions. Nick mentioned Linux
> distributions, but I would like to express interest for other kind of
> downstream distributors like Anaconda from Continuum or Canopy from
> Enthought (disclaimer: I work for Enthought).
> Right now, wheels have the following limitations for us:
> 1. lack of post/pre install/removing
> 2. more fine-grained installation scheme
> 3. lack of clarify on which tags vendors should use for custom wheels:
> some packages we provide would not be installable on "normal" python, and
> it would be nice to have a scheme to avoid confusion there as well.
> At least 1. and 2. are of interest not just for us.
> Regarding 2., it looks like anything in the <wheel_name>.data/data
> directory will be placed as is in sys.prefix by pip. This is how distutils
> scheme is defined ATM, but I am not sure whether that's by design or
> accident ?
> I would suggest to use something close to autotools, with some tweaks to
> work well on windows. I implemented something like this in my project bento
> (
> https://github.com/cournape/Bento/blob/master/bento/core/platforms/sysconfig.py),
> but we could of course tweak that.
> For 1., I believe it was a conscious decision not to include them in wheel
> 1.0 ? Would it make sense to start a discussion to add it to wheel ?
> I will be at the pycon sprints until wednesday evening, so that we can
> flesh some concrete proposal first, if there is enough interest.
> As a background: at Enthought, we have been using eggs to distribute
> binaries of python packages and other packages (e.g. C libraries, compiled
> binaries, etc...) for a very long time. We had our own extensions to the
> egg format to support this, but I want to get out of eggs so as to make our
> own software more compatible with where the community is going. I would
> also like to avoid making ad-hoc extensions to wheels for our own purposes.
> To my knowledge, (1) was purposely punted until a later revision of Wheel
> just to make it easier to land the “basic” wheel.

Great. Was there any proposal made to support it at all ? Or should I just
work from scratch there ?

> I think (2) is a reasonable thing as long as we can map it sanely on all
> platforms.

Yes. We support all platforms at Enthought, and Windows is important for us

> I’m not sure what (3) means exactly. What is a “normal” Python, do you
> modify Python in a way that breaks the ABI but which isn’t reflected in the
> standard ABI tag?

It could be multiple things. The most obvious one is that generally.
cross-platforms python distributions will try to be "relocatable" (i.e. the
whole installation can be moved and still work). This means they require
python itself to be built a special way. Strictly speaking, it is not an
ABI issue, but the result is the same though: you can't use libraries from
anaconda or canopy on top of a normal python

More generally, we could be modifying python in a way that is not forward
compatible with upstream python: a binary that works on our python may not
work on the python from python.org (though the opposite is true). It would
be nice if one could make sure pip will not try to install those eggs when
installed on top of a python that does not advertise itself as "compatible".

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20150413/91377690/attachment-0001.html>

More information about the Distutils-SIG mailing list