[Distutils] Handling the binary dependency management problem

[Nick Coghlan]

On 6 December 2013 11:52, Donald Stufft <donald at stufft.io> wrote:
>
> On Dec 5, 2013, at 8:48 PM, Chris Barker - NOAA Federal <chris.barker at noaa.gov> wrote:
>
>> What would really be best is run-time selection of the appropriate lib
>> -- it would solve this problem, and allow users to re-distribute
>> working binaries via py2exe, etc. And not require opening a security
>> hole in wheels...
>>
>> Not sure how hard that would be to do, though.
>
> Install time selectors probably isn’t a huge deal as long as there’s a way
> to force a particular variant to install and to disable the executing code.

Hmm, I just had an idea for how to do the runtime selection thing. It
actually shouldn't be that hard, so long as the numpy folks are OK
with a bit of __path__ manipulation in package __init__ modules.

Specifically, what could be done is this:

- all of the built SSE level dependent modules would move out of their
current package directories into a suitable named subdirectory (say
"_nosse, _sse2, _sse3")
- in the __init__.py file for each affected subpackage, you would have
a snippet like:

    numpy._add_sse_subdir(__path__)

where _add_sse_subdir would be something like:

    def _add_sse_subdir(search_path):
        if len(search_path) > 1:
            return # Assume the SSE dependent dir has already been added
        # Could likely do this SSE availability check once at import time
        if _have_sse3():
            sub_dir = "_sse3"
        elif _have_sse2():
            sub_dir = "_sse2"
        else:
            sub_dir = "_nosse"
        main_dir = search_path[0]
        search_path.append(os.path.join(main_dir, sub_dir)

With that approach, the existing wheel model would work (no need for a
variant system), and numpy installations could be freely moved between
machines (or shared via a network directory).

To avoid having the implicit namespace packages in 3.3+ cause any
problems with this approach, the SSE subdirectories should contain
__init__.py files that explicitly raise ImportError.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia