[Distutils] Handling the binary dependency management problem

[Nick Coghlan]

On 3 December 2013 21:22, Oscar Benjamin <oscar.j.benjamin at gmail.com> wrote:
> AFAICT conda/binstar are alternatives for pip/PyPI that happen to host
> binaries for some packages that don't have binaries on PyPI. (conda
> also provides a different - incompatible - take on virtualenvs but
> that's not relevant to this proposal).

It sounds like I may have been confusing two presentations at the
packaging mini-summit, as I would have sworn conda used hashes to
guarantee a consistent set of packages. I know I have mixed up
features between hashdist and conda in the past (and there have been
some NixOS features mixed in there as well), so it wouldn't be the
first time that has happened - the downside of mining different
distribution systems for ideas is that sometimes I forget where I
encountered particular features :)

If conda doesn't offer such an internal consistency guarantee for
published package sets, then I agree with the criticism that it's just
an alternative to running a private PyPI index server hosting wheel
files pre-built with particular options, and thus it becomes
substantially less interesting to me :(

Under that model, what conda is doing is *already covered* in the
draft metadata 2.0 spec (as of the changes I posted about the other
day), since that now includes an "integrator suffix" (to indicate when
a downstream rebuilder has patched the software), as well as a
"python.integrator" metadata extension to give details of the rebuild.
The namespacing in the wheel case is handled by not allowing rebuilds
to be published on PyPI - they have to be published on a separate
index server, and thus can be controlled based on where you tell pip
to look.

So, I apologise for starting the thread based on what appears to be a
fundamentally false premise, although I think it has still been useful
despite that error on my part (as the user confusion is real, even
though my specific proposal no longer seems as useful as I first
thought).

I believe helping the conda devs to get it to play nice with virtual
environments is still a worthwhile exercise though (even if just by
pointing out areas where it *doesn't* currently interoperate well, as
we've been doing in the last day or so), and if the conda
bootstrapping issue is fixed by publishing wheels (or vendoring
dependencies), then "try conda if there's no wheel" may still be a
reasonable fallback recommendation.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia