[Python-Dev] Status of C compilers for Python on Windows

[Steve Dower]

Ray Donnelly wrote:
> On Sat, Oct 25, 2014 at 11:44 PM, Paul Moore <p.f.moore at gmail.com> wrote:
>> On 25 October 2014 23:22, Chris Angelico <rosuav at gmail.com> wrote:
>>> On Sun, Oct 26, 2014 at 9:19 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
>>>> My point is that your "Windows build" would not have the same behaviour
>>>> as a MSVC-produced Windows build, and so testing it with it would not
>>>> certify that your code would actually be compatible with genuine
>>>> MSVC builds of CPython, which we will not stop supporting.
>>>>
>>>
>>> So you're saying it's impossible to support two compilers?
>>
>> No, rather that Windows currently only has a single supported compiler
>> (excluding cygwin, which is essentially a different OS). Adding a
>> second compiler doesn't just involve adding support for it - which is
>> all that the people offering mingw patches are doing - but also
>> involves going through the whole CPython ecosystem locating the places
>> where there is an implicit assumption that "all Windows builds use the
>> same compiler" and fixing them. I've already pointed out where this is
>> a question for pip and wheel. Whoever wants to add support for a
>> second compiler needs to be willing to do this part of the job as
>> well.
>>
>> Handwaving arguments that "it's binary compatible" aren't enough. Prove it.
> 
> The msvcrt.dlls that MinGW-w64 depends on are those dating back to
> Windows XP SP3 / XP64. Ironically, the official Windows CPython
> doesn't come with any such crt guarantees and you must ensure that the
> same msvcr??.dll is used for *all* extensions. This puts considerable
> strain on extension developers to use the correct (or any) version of
> Visual Studio to build their extensions for CPython on Windows.

We're well aware of this, and it's a big part of why I'm currently migrating CPython to build with VC14, which will not have the same binary compatibility issues. For VC14, the entire CRT has been cleaned up and mostly hidden behind calls into DLLs, so provided the calling conventions match (which they must or everything would crash very quickly), it should be relatively easy to build compatible extensions with MinGW-w64.

> Also, where are the publicly accessible specifications and other technical
> descriptions that MinGW-w64 would need to implement strong binary
> compatibility with MSVC? As a random example, those for C++ name
> mangling and the PDB file format would be very helpful.

C++ name mangling is always an implementation detail and it changes from version to version. Luckily, CPython is entirely in C, so that doesn't matter. PDBs are another red herring - you can build a loadable PE file without PDBs.

The full source code for the MSVCRT is available with any version of Visual Studio (including the free editions, last time I checked), so feel free to check whatever you need to ensure compatibility. I've suggested to the VC team that they could get in touch with the MinGW projects and offer to help them improve compatibility with MSVC, but unfortunately I don't think anyone will take me up on that. I'm happy to research what I can to answer specific questions, but there's very little that isn't already publicly available other than direct access to the devs.

Cheers,
Steve

>> Paul