[Neuroimaging] Using Python libs from MATLAB
marmaduke.woodman at univ-amu.fr
Thu Jun 16 13:52:09 EDT 2016
Thanks for the details on linking. I hadn't had time to test all bugs on
On Mac, everything seems ok, including h5py and non-MKL linalg.
On Linux, with h5py I see HDF5 library warning that lib and header versions
don't match, with a stack trace including
[ 1] 0x00007f841ce30fc0
[ 2] 0x00007f840b9c8d75
[ 3] 0x00007f83de4c367f
ldd shows h5py's dll linked to conda's hdf5, so this perhaps confirms your
On Tue, Jun 14, 2016 at 9:53 PM, Nathaniel Smith <njs at pobox.com> wrote:
> On Jun 14, 2016 12:36 PM, "Marmaduke Woodman" <
> marmaduke.woodman at univ-amu.fr> wrote:
> > On Tue, Jun 14, 2016 at 5:03 PM, Nathaniel Smith <njs at pobox.com> wrote:
> >> My prediction is that on Windows and OS X, you actually can use
> inconsistent versions of hdf5 and blas, and it will work. Can you confirm
> this explicitly? It's possible that I'm wrong, but I would be surprised
> because it's actually impossible to get symbol collisions on those
> operating systems unless you really try hard to configure things in an
> extra broken way.
> > I can confirm that h5py's call into hdf5 results in an abort due to
> inconsistent header and library versions (1.8.15 vs 1.8.16 for example). We
> can call it something else like ABI mismatch, but nevertheless, it seems
> has to use their version of HDF5 or not at all. I wrote to this list partly
> so that others interested might help confirm, deny or qualify the
> difficulties I stated.
> Ok. On Windows, I think the only way this could be happening is if
> everyone using hdf5 is linking to a version of the dll without the version
> number in the name. The solution is probably for h5py to make sure it links
> against and ships a library called libhdf5-1.8.16.dll or similar.
> On OS X... not sure what could be causing this. Maybe h5py's build system
> is passing some sort of please-give-me-symbol-collisions flag to the linker
> (-flat_namespace and friends)? Maybe it has some buggy paths embedded in
> the binaries? On OS X I can't immediately think of a way to make h5py break
> on MATLAB without making it mostly broken everywhere else too, but I guess
> someone found a way. It's probably not hard to fix though.
> On Linux, it's also not too hard to avoid these issues, but if things are
> broken now then it suggests that MATLAB is doing something bad internally
> like loading libhdf5 with the RTLD_GLOBAL flag. Which is the flag that
> means "please monkey patch everything loaded after this with these
> symbols", equivalent to LD_PRELOAD. And if that's what's going on then
> there isn't much that h5py can do about it; really the MATLAB folk would
> need to fix that.
> >> (I notice they say they're compatible with py27, py33, py34... I wonder
> what compilers they used for that on Windows.)
> > They are using the C-API to libpython.dll, so probably MSVC..?
> Normally py27 and py33/py34 use different and incompatible versions of
> MSVC. It's possible to mix them under certain limited circumstances, but
> this is also another chance to mess things up :-).
> This matters because if you want to use wheels from PyPI, the py27 wheels
> all assume one version of msvc, and the py33/py34 wheels all assume a
> different and incompatible version. (And py35 is yet another different and
> incompatible version.)
> Neuroimaging mailing list
> Neuroimaging at python.org
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the Neuroimaging