On Fri, Dec 6, 2013 at 5:47 AM, Nick Coghlan <ncoghlan@gmail.com> wrote:

On 6 December 2013 11:52, Donald Stufft <donald@stufft.io> wrote:
>
> On Dec 5, 2013, at 8:48 PM, Chris Barker - NOAA Federal <chris.barker@noaa.gov> wrote:
>
>> What would really be best is run-time selection of the appropriate lib
>> -- it would solve this problem, and allow users to re-distribute
>> working binaries via py2exe, etc. And not require opening a security
>> hole in wheels...
>>
>> Not sure how hard that would be to do, though.
>
> Install time selectors probably isn’t a huge deal as long as there’s a way
> to force a particular variant to install and to disable the executing code.

Hmm, I just had an idea for how to do the runtime selection thing. It
actually shouldn't be that hard, so long as the numpy folks are OK
with a bit of __path__ manipulation in package __init__ modules.

As Ralf, I think it is overkill. The problem of SSE vs non SSE is because of one library, ATLAS, which as IMO the design flaw of being arch specific. I always hoped we could get away from this when I built those special installers for numpy :)

MKL does not have this issue, and now that openblas (under a BSD license) can be used as well, we can alleviate this for deployment. Building a deployment story for this is not justified.

David

Specifically, what could be done is this:

- all of the built SSE level dependent modules would move out of their
current package directories into a suitable named subdirectory (say
"_nosse, _sse2, _sse3")
- in the __init__.py file for each affected subpackage, you would have
a snippet like:

numpy._add_sse_subdir(__path__)

where _add_sse_subdir would be something like:

def _add_sse_subdir(search_path):
if len(search_path) > 1:
return # Assume the SSE dependent dir has already been added
# Could likely do this SSE availability check once at import time
if _have_sse3():
sub_dir = "_sse3"
elif _have_sse2():
sub_dir = "_sse2"
else:
sub_dir = "_nosse"
main_dir = search_path[0]
search_path.append(os.path.join(main_dir, sub_dir)

With that approach, the existing wheel model would work (no need for a
variant system), and numpy installations could be freely moved between
machines (or shared via a network directory).

To avoid having the implicit namespace packages in 3.3+ cause any
problems with this approach, the SSE subdirectories should contain
__init__.py files that explicitly raise ImportError.

Cheers,

Nick.

--
Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia

_______________________________________________
Distutils-SIG maillist - Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig