[Distutils] Multi-version import support for wheel files

[Jim Fulton]

On Sun, Aug 25, 2013 at 1:57 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> I'm currently working on the docs for the __main__.__requires__
> feature of pkg_resources, and have been generally poking around inside
> pkg_resources before I started on that. It gave me an idea for a
> question that has come up a few times: how should we support parallel
> installation of multiple versions of the same distribution in the same
> Python installation, *without* requiring complete isolation in the
> form of virtual environments.
>
> The current solution (at least in Fedora), is to use the multi-version
> support in pkg_resources by installing unpacked egg files.

This is also the approach used by buildout.

(It's also the approach (except for the unpacked part) used in
modern Java-ecosystem-based deployments, FWIW.  Collect
jar files, typically in a cache, and set application-specific classpaths
to point to the right ones.)

...

> CherryPy 2 is *not* the default, and is instead installed as an
> unpacked "CherryPy-2.3.0-py2.7.egg" directory. You can force this
> directory to be added to sys.path by doing the following in your
> __main__ module:
>
>     __requires__ = ["CherryPy < 3"]
>     import pkg_resources

I'd never see this. Interesting.

> While I'm not a fan (to put it mildly) of non-trivial side effects
> when importing a module,

Me neither.

> this approach to multi-version imports *does*
> work well (and, as noted, I'm currently working on improving the docs
> for it), and I think the approach to the filesystem layout in
> particular makes sense - the alternative versions are installed to the
> usual location, but pushed down a level in a subdirectory or zip
> archive.

Note that buildout takes a different approach that I think is worth
keeping in mind.  In buildout, of course, there's a specific "build"
step that assembles an application from its parts. In the case of
Python distributions, this means creating an application specific
Python path as a list of installed eggs.

This works well. It's explicit and pretty non-invasive.  No import
magic, .pth files or funny site.py files*.

Buildout really wants to have self-contained distribution
installations, whether they be eggs or wheels or whatever,
to function properly.

Jim

* There was a well-intention but unfortunate deviation
  in later buildout 1 versions.  This was, somewhat ironically
  in pursuit of better integration with system Python installations.



-- 
Jim Fulton
http://www.linkedin.com/in/jimfulton