[Numpy-discussion] MKL and OpenBLAS

Carl Kleffner cmkleffner at gmail.com
Thu Jan 30 06:01:11 EST 2014


I agree, building OpenBLAS with mingw-w64 is a snap. The problem is
choosing and adapting a mingw based gcc-toolchain and patching the numpy
sources according to this toolchain. For the last years I was a happy user
of the mingw.org based toolchain. After searching for a 64-bit alternative
I stumbled upon mingw-w64 and its derivatives.

I tried out several mingw-w64 based toolchains, i.e. TDM, equation.com and
more. All mingw-w64 derivatives have there pros and cons. You may know,
that you have to choose not only for bitness (32 vs 64 bit) and gcc
version, but also for exception handling (sjlj, dwarf, seh) and the way
threading is supported (win32 vs. posix threads). Not all of these
derivatives describe what they use in a clearly manner. And the TDM
toolchain i.e. has introduced some API incompatibilities to standard
gcc-toolchains due to its own patches.

A serious problem is gcc linking to runtimes other than msvcrt.dll.
Mingw-w64 HAS import libraries fort msvcr80, msvcr90, msvcr100, msvcr110.
However correct linkage to say to msvcr90 is more than just adding
-lmsvcr90 to the linker command. You have to create a spec file for gcc and
adapt it to your need. It is also very important (especially for msvcr90)
to link manifest files to the binaries you create. This has to do with the
way Microsoft searches for DLLs. "Akruis" (Anselm Kruis, science +
computing AG) did the job to iron out these problems concerning mingw-w64
and python. Unfortunately his blog disappears for some time now.

The maintainers of the mingw-w64 toolchains DO NOT focus on the problem
with alternative runtime linking. A related problem is that symbols are
used by OpenMP and winpthreads you can resolve in msvcrt.dll, but not in
msvcr90.dll, so "_ftime"has to be exchanged with "ftime64" if you want to
use OpenMP or winpthreads.

In the end my solution was to build my own toolchain. This is time
consuming but simple with the help of the set of scripts you can find here:
https://github.com/niXman/mingw-builds/tree/develop

With this set of scripts and msys2
http://sourceforge.net/projects/msys2/and my own "_ftime" patch I
build a 'statically' mingw-w64 toolchain. Let
me say a word about statically build: GCC can be build statically. This
means, that all of the C, C++, Gfortran runtime is statically linked to
every binary. There is not much bloat as you might expect when the binaries
are stripped.

And yes, it is necessary to build an import lib for python. This import lib
is specific to the toolchain you are going to use. My idea is to create and
add all import libs (py2.6 up to py3.4) to the toolchain and do not use any
of the importlibs that might exist in the python/libs/ folder.

My conclusion is: mixing different compiler architectures for building
Python extensions on Windows is possible but makes it necessary to build a
'vendor' gcc toolchain. I did not find the time to put my latest binaries
on the web or make numpy pull requests the github way due to my workload.
Hopefully I find some time next weekend.

with best regards

Carl


2014-01-30 Sturla Molden <sturla.molden at gmail.com>:

> On 27/01/14 12:01, Carl Kleffner wrote:
> > Did you consider to check the experimental binaries on
> > https://code.google.com/p/mingw-w64-static/ for Python-2.7? These
> > binaries has been build with with a customized mingw-w64 toolchain.
> > These builds are fully statically build and are link against the MSVC90
> > runtime libraries (gcc runtime is linked statically) and OpenBLAS.
> >
> > Carl
>
> Building OpenBLAS and LAPACK is very easy. I used TDM-GCC for Win64.
> It's just two makefile (not even a configure script). OpenBLAS and
> LAPACK are probably the easiest libraries to build there is.
>
> The main problem for using OpenBLAS with NumPy and SciPy on Windows is
> that Python 2.7 from www.python.org does not ship with libpython27.a for
> 64-bit Python, so we need to maintain our own. Also, GNU compilers are
> required to build OpenBLAS. This means we have to build our own
> libgfortran as well. The binary is incompatible with the MSVC runtime we
> use. I.e. not impossible, but painful.
>
> http://mail.scipy.org/pipermail/numpy-discussion/2012-August/063740.html
>
>
> Sturla
>
>
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20140130/5f8737ca/attachment.html>


More information about the NumPy-Discussion mailing list