[Distutils] Making pip and PyPI work with conda packages

Chris Barker chris.barker at noaa.gov
Sat May 16 22:38:50 CEST 2015

On Sat, May 16, 2015 at 12:04 PM, David Mertz <dmertz at continuum.io> wrote:

> Continuum has a great desire to make 'pip' work with conda packages.
> Obviously, we love for users to choose the Anaconda Python distribution but
> many will not for a variety of reasons (many good reasons).

Hmm -- this strikes me as very, very , tricky -- and of course, tied in to
the other thread I've been spending a bunch of time on...

However, we would like for users of other distros still to be able to
> benefit from our creation of binary packages for many platforms in the
> conda format.

Frankly, if you want your efforts at building binaries to get used outside
of Anaconda, then you shoudl be building wheels in the first place. While
conda does more than pip + wheel can do -- I suppose you _could_ use wheels
for the things it can support..

But on to the technical issues:

conda python packages depend on other conda packages, and some of those
packages are not python packages at all. The common use case here are
non-python dynamic libs -- exactly the use case I've been going on in the
other thread about...

And conda installs those dynamic libs in a conda environment -- outside of
the python environment. So you can't really use a conda package without a
conda enviroment, and an installer that understands that environment (I
think conda install does some lib path re-naming, yes?), i.e. conda itself.
So I think that's kind of a dead end.

So what about the idea of a conda-package-to-wheel converter? conda
packages an wheels have a bit in common -- IIUC, they are both basically a
zip of all the files you need installed. But again the problem is those
dependencies on third party dynamic libs.

So far that to work -- pip+wheel would have to grow a way to deal with
installing, managing and using dynamic libs. See the other thread for the
nightmare there...

And while I'd love to see this happen, perhaps an easier route would be for
conda_build to grow a "static" flag that will statically link stuff and get
to somethign already supported by pip, wheel, and pypi.


> It is true that right now, a user can in principle type:
>   % pip install conda
>   % conda install some_conda_package
> But that creates two separate systems for tracking what's installed and
> what dependencies are resolved;

Indeed -- which is why some folks are working on making it easier to use
conda for everything....converting a wheel to a conda package is probably
easier than the other way around..

Funny -- just moments ago I wrote that it didn't seem that anyone other
than me was interested in extending pip_wheel to support this kind of thing
-- I guess I was wrong!

Great to see you and continuum thinking about this.



Christopher Barker, Ph.D.

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20150516/fa90bd1f/attachment.html>

More information about the Distutils-SIG mailing list