[Numpy-discussion] ANN: NumPy 1.8.1 release

[Nathaniel Smith]

On Tue, Apr 1, 2014 at 11:58 PM, David Cournapeau <cournape at gmail.com> wrote:
> On Tue, Apr 1, 2014 at 6:43 PM, Nathaniel Smith <njs at pobox.com> wrote:
>>
>> On Tue, Apr 1, 2014 at 6:26 PM, Matthew Brett <matthew.brett at gmail.com>
>> wrote:
>> > I'm guessing that the LOAD_WITH_ALTERED_SEARCH_PATH means that a DLL
>> > loaded via:
>> >
>> > hDLL = LoadLibraryEx(pathname, NULL,  LOAD_WITH_ALTERED_SEARCH_PATH);
>> >
>> > will in turn (by default) search for its dependent DLLs in their own
>> > directory.    Or maybe in the directory of the first DLL to be loaded
>> > with LOAD_WITH_ALTERED_SEARCH_PATH, damned if I can follow the
>> > documentation.  Looking forward to doing my tax return after this.
>> >
>> > But - anyway - that means that any extensions in the DLLs directory
>> > will get their dependencies from the DLLs directory, but that is only
>> > true for extensions in that directory.
>>
>> So in conclusion, if we just drop our compiled dependencies next to
>> the compiled module files then we're good, even on older Windows
>> versions? That sounds much simpler than previous discussions, but good
>> news if it's true...
>
>
> That does not work very well in my experience:
>
>   - numpy has extension modules in multiple directories, so we would need to
> copy the dlls in multiple subdirectories
>   - copying dlls means that windows will load that dll multiple times, with
> all the ensuing problems (I don't know for MKL/OpenBlas, but we've seen
> serious issues when doing something similar for hdf5 dll and pytables/h5py).

We could just ship all numpy's extension modules in the same directory
if we wanted. It would be pretty easy to stick some code at the top of
numpy/__init__.py to load them from numpy/all_dlls/ and then slot them
into the appropriate places in the package namespace.

Of course scipy and numpy will still both have to ship BLAS etc., and
so I guess it will get loaded at least twice in *any* binary install
system. I'm not sure why this would be a problem (Windows, unlike
Unix, carefully separates DLL namespaces, right?), but if it is a
problem then it's a very fundamental one for any binaries we ship.

Do the binaries we ship now have this problem? Or are we currently
managing to statically link everything?

-n

-- 
Nathaniel J. Smith
Postdoctoral researcher - Informatics - University of Edinburgh
http://vorpus.org