[Distutils] Handling the binary dependency management problem
David Cournapeau
cournape at gmail.com
Fri Dec 6 14:06:07 CET 2013
On Fri, Dec 6, 2013 at 5:47 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> On 6 December 2013 11:52, Donald Stufft <donald at stufft.io> wrote:
> >
> > On Dec 5, 2013, at 8:48 PM, Chris Barker - NOAA Federal <
> chris.barker at noaa.gov> wrote:
> >
> >> What would really be best is run-time selection of the appropriate lib
> >> -- it would solve this problem, and allow users to re-distribute
> >> working binaries via py2exe, etc. And not require opening a security
> >> hole in wheels...
> >>
> >> Not sure how hard that would be to do, though.
> >
> > Install time selectors probably isn’t a huge deal as long as there’s a
> way
> > to force a particular variant to install and to disable the executing
> code.
>
> Hmm, I just had an idea for how to do the runtime selection thing. It
> actually shouldn't be that hard, so long as the numpy folks are OK
> with a bit of __path__ manipulation in package __init__ modules.
>
As Ralf, I think it is overkill. The problem of SSE vs non SSE is because
of one library, ATLAS, which as IMO the design flaw of being arch specific.
I always hoped we could get away from this when I built those special
installers for numpy :)
MKL does not have this issue, and now that openblas (under a BSD license)
can be used as well, we can alleviate this for deployment. Building a
deployment story for this is not justified.
David
>
> Specifically, what could be done is this:
>
> - all of the built SSE level dependent modules would move out of their
> current package directories into a suitable named subdirectory (say
> "_nosse, _sse2, _sse3")
> - in the __init__.py file for each affected subpackage, you would have
> a snippet like:
>
> numpy._add_sse_subdir(__path__)
>
> where _add_sse_subdir would be something like:
>
> def _add_sse_subdir(search_path):
> if len(search_path) > 1:
> return # Assume the SSE dependent dir has already been added
> # Could likely do this SSE availability check once at import time
> if _have_sse3():
> sub_dir = "_sse3"
> elif _have_sse2():
> sub_dir = "_sse2"
> else:
> sub_dir = "_nosse"
> main_dir = search_path[0]
> search_path.append(os.path.join(main_dir, sub_dir)
>
> With that approach, the existing wheel model would work (no need for a
> variant system), and numpy installations could be freely moved between
> machines (or shared via a network directory).
>
> To avoid having the implicit namespace packages in 3.3+ cause any
> problems with this approach, the SSE subdirectories should contain
> __init__.py files that explicitly raise ImportError.
>
> Cheers,
> Nick.
>
> --
> Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia
> _______________________________________________
> Distutils-SIG maillist - Distutils-SIG at python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20131206/fbf23db1/attachment-0001.html>
More information about the Distutils-SIG
mailing list