[Distutils] Making pip and PyPI work with conda packages

[Nick Coghlan]

On 20 May 2015 at 23:30, Daniel Holth <dholth at gmail.com> wrote:
> It occurs to me that the setuptools packaging in general is more like
> a shared library format .so or .dll, aka libraries searched for along
> a path, than an OS level package manager.

Yep, that was what PJE was after for Chandler, so that's what he
built. It was just useful enough for other folks that it was adopted
well beyond that original use case.

The key benefit it offered at the time was that pkg_resources could
use sys.path + the assumption of directory or zip archive based
installation to implement searching for metadata, which avoided the
need to come up with an alternative cross-platform approach to
metadata storage and retrieval.

A related potentially interesting project I've never had time to
pursue is an idea for a virtualenv friendly installation layout that
doesn't quite lead to the same kind of version proliferation as
setuptools did (or as something like NixOS does), while remaining
compatible with sys.path based metadata discovery.

The essential concept would be to assume semantic versioning for
shared installations, and install the shared packages into directories
named as "package/MAJOR_VERSION/<what-would-otherwise-go-into-site-packages>".
In each virtualenv that opts in to using the shared package rather
than its own bundled copy, you'd then install a *.pth file that added
"package/MAJOR_VERSION" to sys.path in that environment.

This would be similar in principle to the way Nix user profiles work
(http://nixos.org/releases/nix/nix-0.12/manual/#sec-profiles), but
adapted to be compatible with Python's existing *.pth file mechanism.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia