[Distutils] Handling the binary dependency management problem

[Ralf Gommers]

On Fri, Dec 6, 2013 at 1:33 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:

> On 6 December 2013 17:21, Ralf Gommers <ralf.gommers at gmail.com> wrote:
> > On Fri, Dec 6, 2013 at 6:47 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> >> With that approach, the existing wheel model would work (no need for a
> >> variant system), and numpy installations could be freely moved between
> >> machines (or shared via a network directory).
> >
> > Hmm, taking a compile flag and encoding it in the package layout seems
> like
> > a fundamentally wrong approach. And in order to not litter the source
> tree
> > and all installs with lots of empty dirs, the changes to __init__.py will
> > have to be made at build time based on whether you're building Windows
> > binaries or something else. Path manipulation is usually fragile as
> well. So
> > I suspect this is not going to fly.
>
> In the absence of the perfect solution (i.e. picking the right variant
> out of no SSE, SSE2, SSE3 automatically), would it be a reasonable
> compromise to standardise on SSE2 as "lowest acceptable common
> denominator"?
>

Maybe, yes. It's hard to figure out the impact of this, but I'll bring it
up on the numpy list. If no one has a good way to get some statistics on
cpu's that don't support these instruction sets, it may be worth a try for
one of the Python versions and see how many users will run into the issue.

On accident we've released an incorrect binary once before by the way
(scipy 0.8.0 for Python 2.5) and that was a problem fairly quickly:
https://github.com/scipy/scipy/issues/1697. That was 2010 though.


> Users with no sse capability at all or that wanted to take advantage
> of the SSE3 optimisations, would need to grab one of the Windows
> installers or something from conda, but for a lot of users, a "pip
> install numpy" that dropped the SSE2 version onto their system would
> be just fine, and a much lower barrier to entry than "well, first
> install this other packaging system that doesn't interoperate with
> your OS package manager at all...".
>

Well, for most Windows users grabbing a .exe and clicking on it is a lower
barrier that opening a console and typing "pip install numpy":)


> Are we letting perfect be the enemy of better, here? (punting on the
> question for 6 months and seeing if we can deal with the install-time
> variant problem in pip 1.6 is certainly an option, but if we don't
> *need* to wait that long...)
>

Let's first get the OS X wheels up, that can be done now. And then see what
is decided on the numpy list for the compromise you propose above.

Ralf
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20131206/700b807c/attachment.html>