[Numpy-discussion] ANN: NumPy 1.8.1 release

Matthew Brett matthew.brett at gmail.com
Wed Apr 2 02:52:28 EDT 2014


Hi,

On Tue, Apr 1, 2014 at 4:46 PM, David Cournapeau <cournape at gmail.com> wrote:
>
>
>
> On Wed, Apr 2, 2014 at 12:36 AM, Nathaniel Smith <njs at pobox.com> wrote:
>>
>> On Tue, Apr 1, 2014 at 11:58 PM, David Cournapeau <cournape at gmail.com>
>> wrote:
>> > On Tue, Apr 1, 2014 at 6:43 PM, Nathaniel Smith <njs at pobox.com> wrote:
>> >>
>> >> On Tue, Apr 1, 2014 at 6:26 PM, Matthew Brett <matthew.brett at gmail.com>
>> >> wrote:
>> >> > I'm guessing that the LOAD_WITH_ALTERED_SEARCH_PATH means that a DLL
>> >> > loaded via:
>> >> >
>> >> > hDLL = LoadLibraryEx(pathname, NULL,  LOAD_WITH_ALTERED_SEARCH_PATH);
>> >> >
>> >> > will in turn (by default) search for its dependent DLLs in their own
>> >> > directory.    Or maybe in the directory of the first DLL to be loaded
>> >> > with LOAD_WITH_ALTERED_SEARCH_PATH, damned if I can follow the
>> >> > documentation.  Looking forward to doing my tax return after this.
>> >> >
>> >> > But - anyway - that means that any extensions in the DLLs directory
>> >> > will get their dependencies from the DLLs directory, but that is only
>> >> > true for extensions in that directory.
>> >>
>> >> So in conclusion, if we just drop our compiled dependencies next to
>> >> the compiled module files then we're good, even on older Windows
>> >> versions? That sounds much simpler than previous discussions, but good
>> >> news if it's true...
>> >
>> >
>> > That does not work very well in my experience:
>> >
>> >   - numpy has extension modules in multiple directories, so we would
>> > need to
>> > copy the dlls in multiple subdirectories
>> >   - copying dlls means that windows will load that dll multiple times,
>> > with
>> > all the ensuing problems (I don't know for MKL/OpenBlas, but we've seen
>> > serious issues when doing something similar for hdf5 dll and
>> > pytables/h5py).
>>
>> We could just ship all numpy's extension modules in the same directory
>> if we wanted. It would be pretty easy to stick some code at the top of
>> numpy/__init__.py to load them from numpy/all_dlls/ and then slot them
>> into the appropriate places in the package namespace.
>>
>> Of course scipy and numpy will still both have to ship BLAS etc., and
>> so I guess it will get loaded at least twice in *any* binary install
>> system. I'm not sure why this would be a problem (Windows, unlike
>> Unix, carefully separates DLL namespaces, right?)
>
>
> It does not really matter here. For pure blas/lapack, that may be ok because
> the functions are "stateless", but I would not count on it either.
>
> The cleanest solution I can think of is to have 'privately shared DLL', but
> that would AFAIK require patching python, so not really an option.

David - do you know anything about private assemblies [1]?
Might they work for our problem? How about AddDllDirectory [2]?

Cheers,

Matthew

[1] http://msdn.microsoft.com/en-us/library/windows/desktop/ff951638(v=vs.85).aspx
[2] http://msdn.microsoft.com/en-us/library/windows/desktop/hh310513(v=vs.85).asp



More information about the NumPy-Discussion mailing list