[Neuroimaging] Using Python libs from MATLAB

[Nathaniel Smith]

On Jun 14, 2016 12:36 PM, "Marmaduke Woodman" <marmaduke.woodman at univ-amu.fr>
wrote:
>
>
> On Tue, Jun 14, 2016 at 5:03 PM, Nathaniel Smith <njs at pobox.com> wrote:
>>
>> My prediction is that on Windows and OS X, you actually can use
inconsistent versions of hdf5 and blas, and it will work. Can you confirm
this explicitly? It's possible that I'm wrong, but I would be surprised
because it's actually impossible to get symbol collisions on those
operating systems unless you really try hard to configure things in an
extra broken way.
>
> I can confirm that h5py's call into hdf5 results in an abort due to
inconsistent header and library versions (1.8.15 vs 1.8.16 for example). We
can call it something else like ABI mismatch, but nevertheless, it seems
has to use their version of HDF5 or not at all. I wrote to this list partly
so that others interested might help confirm, deny or qualify the
difficulties I stated.

Ok. On Windows, I think the only way this could be happening is if everyone
using hdf5 is linking to a version of the dll without the version number in
the name. The solution is probably for h5py to make sure it links against
and ships a library called libhdf5-1.8.16.dll or similar.

On OS X... not sure what could be causing this. Maybe h5py's build system
is passing some sort of please-give-me-symbol-collisions flag to the linker
(-flat_namespace and friends)? Maybe it has some buggy paths embedded in
the binaries? On OS X I can't immediately think of a way to make h5py break
on MATLAB without making it mostly broken everywhere else too, but I guess
someone found a way. It's probably not hard to fix though.

On Linux, it's also not too hard to avoid these issues, but if things are
broken now then it suggests that MATLAB is doing something bad internally
like loading libhdf5 with the RTLD_GLOBAL flag. Which is the flag that
means "please monkey patch everything loaded after this with these
symbols", equivalent to LD_PRELOAD. And if that's what's going on then
there isn't much that h5py can do about it; really the MATLAB folk would
need to fix that.

>> (I notice they say they're compatible with py27, py33, py34... I wonder
what compilers they used for that on Windows.)
>
> They are using the C-API to libpython.dll, so probably MSVC..?

Normally py27 and py33/py34 use different and incompatible versions of
MSVC. It's possible to mix them under certain limited circumstances, but
this is also another chance to mess things up :-).

This matters because if you want to use wheels from PyPI, the py27 wheels
all assume one version of msvc, and the py33/py34 wheels all assume a
different and incompatible version. (And py35 is yet another different and
incompatible version.)

-n
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/neuroimaging/attachments/20160614/8f83a1ae/attachment.html>