[Python-Dev] Status of C compilers for Python on Windows
njs at pobox.com
Mon Oct 27 20:02:52 CET 2014
On Mon, Oct 27, 2014 at 5:48 PM, Paul Moore <p.f.moore at gmail.com> wrote:
> On 26 October 2014 23:44, Paul Moore <p.f.moore at gmail.com> wrote:
>> On 26 October 2014 23:11, Ray Donnelly <mingw.android at gmail.com> wrote:
>>> I don't know where this "ABI compatible" thing came into being;
>> Simple. If a mingw-built CPython doesn't work with the same extensions
>> as a MSVC-built CPython, then the community gets fragmented (because
>> you can only use the extensions built for your stack). Assuming numpy
>> needs mingw and ultimately only gets built for a mingw-compiled Python
>> (because the issues building for MSVC-built Python are too hard) and
>> assuming that nobody wants to make the effort to build pywin32 under
>> mingw, then what does someone who needs both numpy and pywin32 do?
>> Avoiding that issue is what I mean by ABI-compatible. (And that's all
>> I mean by it, nothing more subtle or controversial).
>> I view it as critical (because availability of binaries is *already*
>> enough of a problem in the Windows world, without making it worse)
>> that we avoid this sort of fragmentation. I'm not seeing an
>> acknowledgement from the mingw side that they agree. That's my
>> concern. If we both agree, there's nothing to argue about.
> I have just done some experiments with building CPython extensions
> with mingw-w64. Thanks to Ray for helping me set this up.
> The bad news is that the support added to the old 32-bit mingw to
> support linking to alternative C runtime libraries (specifically
> -lmsvcr100) has bitrotted, and no longer functions correctly in
> mingw-w64. As a result, not only can mingw-w64 not build extensions
> that are compatible with python.org Python, it can't build extensions
> that function at all . They link incompatibly to *both* msvcrt and
> This is a bug in mingw-w64. I have reported it to Ray, who's passed it
> onto one of the mingw-w64 developers. But as things stand, mingw
> builds will definitely produce binary extensions that aren't
> compatible with python.org Python.
IIUC, getting mingw-w64 to link against msvcr100 instead of msvcrt
requires a custom mingw-w64 build, because by default mingw-w64's
internal runtime libraries (libgcc etc.) are linked against msvcrt. So
by the time you're choosing compiler switches etc., it's already too
late -- your switches might affect how *your* code is built, but your
code will still be linked against pre-existing runtime libraries that
are linked against msvcrt.
It's possible to hack the mingw-w64 build process to build the runtime
libraries against msvcr100 (or whatever) instead of msvcrt, but this
is still not a panacea -- the different msvcr* libraries are, of
course, incompatible with each other, and IIUC the mingw-w64
developers have never tried to make their libraries work against
anything except msvcrt. For example, mingw-w64's gfortran runtime uses
a symbol that's only available in msvcrt, not msvcr90 or msvcrt100:
So my impression is that these issues are all fixable, but they will
require real engagement with mingw-w64 upstream.
>  Note, that's if you just use --compiler=mingw32 as supported by
> distutils. Looking at how the numpy folks build, they seem to hack
> their own version of the distutils C compiler classes. I don't know
> whether that's just to work around this bug, or whether they do it for
> other reasons as well (but I suspect the latter).
numpy.distutils is a massive pile of hacks to handle all kinds of
weird things including recursive builds, fortran, runtime capability
detection (like autoconf), and every random issue anyone ran into at
some point in the last 10 years and couldn't be bothered filing a
proper upstream bug report. Basically no-one knows what it actually
does -- the source is your only hope :-).
Nathaniel J. Smith
Postdoctoral researcher - Informatics - University of Edinburgh
More information about the Python-Dev