[Distutils] Making pip and PyPI work with conda packages

[David Cournapeau]

On Mon, May 18, 2015 at 9:05 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:

>
> On 18 May 2015 07:32, "Chris Barker" <chris.barker at noaa.gov> wrote:
> >
> > On Sun, May 17, 2015 at 12:05 AM, Nick Coghlan <ncoghlan at gmail.com>
> wrote:
> >>
> >> >   % pip install --upgrade pip
> >> >   % pip install some_conda_package
> >>
> >> This gets the respective role of the two tools reversed - it's like my
> >> asking for "pip install some_fedora_rpm" to be made to work.
> >
> >
> > I agree here -- I was thinking there was some promise in a
> conda_package_to_wheel converter though. It would, of course, only work in
> a subset of conda packages, but would be nice.
> >
> > The trick is that conda packages for the hard-to-build python packages
> (the ones we care about) often (always?) depend on conda packages for
> dynamic libs, and pip+wheel have no support for that.
> >
> > And this is a trick, because while I have some ideas for supporting
> just-for-python dynamic libs, conda's are not just-for-python -- so that
> might be hard to mash together.
> >
> > Continuum has a bunch of smart people, though.
> >
> >> However, having conda use "pip install" in its build scripts so that
> >> it reliably generates pip compatible installation metadata would be a
> >> possibility worth discussing - that's what we've started doing in
> >> Fedora, so that runtime utilities like pkg_resources can work
> >> correctly.
> >
> >
> > Hmm -- that's something ot look into -- you can put essentially anything
> into a conda bulid script --  so this would be a matter of convention,
> rather than tooling. (of course the conventions used by Continuum for the
> "offical" conda packages is the standard).
> >
> > But I'm confused as to the roles of pip vs setuptools, vs wheel, vs ???
> >
> > I see pip has handling the dependency resolution, and finding and
> downloading of packages part of the problem -- conda does those already.
> >
> > So what would using pip inside a conda build script buy you that using
> setuptools does not?
>
> Indirection via pip injects the usage of setuptools even for plain
> distutils projects, and generates
> https://www.python.org/dev/peps/pep-0376/ compliant metadata by default.
>

Note that some packages will push hard against injecting setuptools, at
least until it does not offer a way to prevent from installing as an egg
directory. Most of the core scientific packages avoid setuptools because of
this.

David


However, looking at the current packaging policy, I think I misremembered
> the situation - it looks like we *discussed* recommending indirection via
> pip & attaining PEP 376 compliance, but haven't actually moved forward with
> the idea yet. That makes sense, since pursuing it would have been gated on
> ensurepip, and the Python 3 migration has been higher priority recently.
>
> Cheers,
> Nick.
>
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20150518/b45ae4b1/attachment.html>