[Distutils] Making pip and PyPI work with conda packages

[Nick Coghlan]

On 18 May 2015 07:32, "Chris Barker" <chris.barker at noaa.gov> wrote:
>
> On Sun, May 17, 2015 at 12:05 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>>
>> >   % pip install --upgrade pip
>> >   % pip install some_conda_package
>>
>> This gets the respective role of the two tools reversed - it's like my
>> asking for "pip install some_fedora_rpm" to be made to work.
>
>
> I agree here -- I was thinking there was some promise in a
conda_package_to_wheel converter though. It would, of course, only work in
a subset of conda packages, but would be nice.
>
> The trick is that conda packages for the hard-to-build python packages
(the ones we care about) often (always?) depend on conda packages for
dynamic libs, and pip+wheel have no support for that.
>
> And this is a trick, because while I have some ideas for supporting
just-for-python dynamic libs, conda's are not just-for-python -- so that
might be hard to mash together.
>
> Continuum has a bunch of smart people, though.
>
>> However, having conda use "pip install" in its build scripts so that
>> it reliably generates pip compatible installation metadata would be a
>> possibility worth discussing - that's what we've started doing in
>> Fedora, so that runtime utilities like pkg_resources can work
>> correctly.
>
>
> Hmm -- that's something ot look into -- you can put essentially anything
into a conda bulid script --  so this would be a matter of convention,
rather than tooling. (of course the conventions used by Continuum for the
"offical" conda packages is the standard).
>
> But I'm confused as to the roles of pip vs setuptools, vs wheel, vs ???
>
> I see pip has handling the dependency resolution, and finding and
downloading of packages part of the problem -- conda does those already.
>
> So what would using pip inside a conda build script buy you that using
setuptools does not?

Indirection via pip injects the usage of setuptools even for plain
distutils projects, and generates https://www.python.org/dev/peps/pep-0376/
compliant metadata by default.

However, looking at the current packaging policy, I think I misremembered
the situation - it looks like we *discussed* recommending indirection via
pip & attaining PEP 376 compliance, but haven't actually moved forward with
the idea yet. That makes sense, since pursuing it would have been gated on
ensurepip, and the Python 3 migration has been higher priority recently.

Cheers,
Nick.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20150518/4161091e/attachment.html>