[Distutils] Multi-version import support for wheel files

[Paul Moore]

On 25 August 2013 06:57, Nick Coghlan <ncoghlan at gmail.com> wrote:

> I'm currently working on the docs for the __main__.__requires__
> feature of pkg_resources, and have been generally poking around inside
> pkg_resources before I started on that. It gave me an idea for a
> question that has come up a few times: how should we support parallel
> installation of multiple versions of the same distribution in the same
> Python installation, *without* requiring complete isolation in the
> form of virtual environments.
>
> The current solution (at least in Fedora), is to use the multi-version
> support in pkg_resources by installing unpacked egg files. To use
> CherryPy as an example, Fedora currently provides RPMs for both
> CherryPy 2 and 3.
>
> CherryPy 3 is the default and installed directly into site-packages,
> with an appropriate .egg-info directory
>
> CherryPy 2 is *not* the default, and is instead installed as an
> unpacked "CherryPy-2.3.0-py2.7.egg" directory. You can force this
> directory to be added to sys.path by doing the following in your
> __main__ module:
>
>     __requires__ = ["CherryPy < 3"]
>     import pkg_resources
>
> (__main__.__requires__ *has* to be set before importing pkg_resources
> or the default CherryPy 3 module will be activated automatically, and
> conflict with a later call to pkg_resources.requires that asks for
> CherryPy 2)
>
> While I'm not a fan (to put it mildly) of non-trivial side effects
> when importing a module, this approach to multi-version imports *does*
> work well (and, as noted, I'm currently working on improving the docs
> for it), and I think the approach to the filesystem layout in
> particular makes sense - the alternative versions are installed to the
> usual location, but pushed down a level in a subdirectory or zip
> archive.
>
> So, it seems to me that only two new pieces are needed to gain
> multi-version import support for wheel files:
>
> 1. An option (or options) to pip, telling it to just drop a wheel file
> (or the unpacked contents of the wheel as a directory) into
> site-packages instead of installing the distribution directly as the
> default version. The "root_is_purelib" setting in the wheel metadata
> would indicate whether the destination was purelib or platlib. A wheel
> installed this way wouldn't have script wrappers generated, etc - it
> would only allow the contents to be used as an importable library.
>
> 2. Support for the ".whl" filename format and internal layout in
> pkg_resources, modelling after the existing support for the ".egg"
> filename format. For wheels that include both purelib and platlib,
> this would involved adding both directories to the path.
>
> That means there wouldn't be an major design work to be done - just
> replicating what easy_install and pkg_resources already support for
> the egg format using the wheel format instead.
>
> Regardless of my personal feelings about the current *front end* API
> for multi-version imports (and PJE has put up with my ranting about
> that with remarkably good grace!), the current egg-based back end
> design looks solid to me and worth keeping rather than trying to
> invent something new for the sake of being different.
>

Like Donald, I'm going to need to look into the proposal a bit more before
commenting.

My main concern is that for people who *don't* need multi-versioning, or
who only use it for one or two special cases, there is no detrimental
impact on user interface or runtime (the "if you don't use it, you don't
pay for it" principle). A lot of the FUD around setuptools'
multi-versioning was around precisely this point (path munging, etc) and I
want to understand how the proposed solution avoids repeating those
problems (to the extent that they were real and not imagined).

Paul
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20130825/e2f1ae0e/attachment.html>