[Distutils] Handling the binary dependency management problem

Nick Coghlan ncoghlan at gmail.com
Tue Dec 3 00:48:36 CET 2013

On 3 Dec 2013 09:03, "Paul Moore" <p.f.moore at gmail.com> wrote:
> On 2 December 2013 22:26, Nick Coghlan <ncoghlan at gmail.com> wrote:
> >>Whether solving the Unix
> >> issues is worth it is the Unix users' call - I'll help solve the
> >> issues, if they choose to, but I won't support abandoning the existing
> >> Windows solution just because it can't be extended to cater for Unix
> >> as well.
> >
> > You appear to still be misunderstanding my proposal, as we're actually
> > violent agreement. All that extra complexity you're worrying about is
> > precisely what I'm saying we should *leave out* of the wheel spec. In
> > cases of accelerator and wrapper modules, the static linking and/or
> > solutions will work fine, and that's the domain I believe we should
> > *deliberately* restrict wheels to, so we don't get distracted trying to
> > solve an incredibly hard external dependency management problem that we
> > don't actually need to solve at the wheel level, since anyone that
> > needs it solved can just bootstrap conda instead.
> OK. I think I've finally seen what you're suggesting, and yes, it's
> essentially the same as I'd like to see (at least for now). I'd hoped
> that wheels could be more useful for Unix users than seems likely now
> - mainly because I really do think that a lot of the benefits of
> binary distributions are *not* restricted to Windows, and if Unix
> users could use them, it'd lessen the tendency to think that
> supporting anything other than source installs was purely "to cater
> for Windows users not having a compiler" :-) But if that's not a
> practical possibility (and I defer to the Unix users' opinions on that
> matter) then so be it.
> On the other hand, I still don't see where the emphasis on conda in
> your original message came from. There are lots of "full stack"
> solutions available - I'd have thought system packages like RPM and
> apt are the obvious first suggestion for users that need a curated
> stack. If they are not appropriate, then there are Enthought,
> ActiveState and Anaconda/conda that I know of. Why single out conda to
> be blessed?
> Also, I'd like the proposal to explicitly point out that 99% of the
> time, Windows is the simple case (because static linking and bundling
> DLLs is common). Getting Windows users to switch to wheels will be
> enough change to ask, without confusing the message. A key point here
> is that packages like lxml, matplotlib, or Pillow would have
> "arbitrary binary dependency issues" on Unix, but (because of static
> linking/bundling) be entirely appropriate for wheels on Windows. Let's
> make sure the developers don't miss this point!

Once we solve the platform tagging problem, wheels will also work on any
POSIX system for the simple cases of accelerator and wrapper modules. Long
term the only persistent problem is with software stacks that need
consistent build settings and offer lots of build options. That applies to
Windows as well - the SSE build variants of NumPy were one of the original
cases brought up as not being covered by the wheel compatibility tag format.

Near term, platform independent stacks *also* serve as a workaround for the
POSIX platform tagging issues and the fact there isn't yet a "default"
build configuration for the scientific stack.

As for "Why conda?":
- open source
- cross platform
- can be installed with pip
- gets new releases of Python components faster than Linux distributions
- uses Continuum Analytics services by default, but can be configured to
use custom servers
- created by the creator of NumPy

For ActiveState and Enthought, as far as I am aware, their package managers
are closed source and tied fairly closely to their business model, while
the Linux distros are not only platform specific, but have spotty coverage
of PyPI packages, and even those which are covered, often aren't reliably
kept up to date (although I hope metadata 2.0 will help improve that
situation by streamlining the conversion to policy compliant system


> Paul
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20131203/3a0d282a/attachment.html>

More information about the Distutils-SIG mailing list