[Python-Dev] Status of C compilers for Python on Windows

[Tony Kelman]

Stephen J. Turnbull:
> the pain of using Windows is what drives me away from all of them.

Enough that you are not able to make the software you write usable on
Windows? I see your point and agree with it - I don't even like Windows
much at all, but supporting it is important for plenty of reasons.

Steve Dower:
> FWIW, I'm happy to talk specifics off list

Off-list, on-list, either way. I've read the blog post about the CRT
refactoring 
http://blogs.msdn.com/b/vcblog/archive/2014/06/10/the-great-crt-refactoring.aspx
but have not yet experimented with the early CTP versions of Visual Studio
"14." Is there any plan to promote these runtimes to installed-by-default
components of the operating system in future service packs or new releases
of Windows? The fact that the redistributables are a separate download and
not always there by default means MinGW developers are not quite so
optimistic about them suddently solving all problems. In the embedded-
Python-for-IJulia pull request on github for example, it turns out the
Python msi installer does not include the necessary runtimes and wouldn't
work by itself on a brand new computer.

It would also help if there were a preview version of just the runtime
libraries available. More far-fetched and not likely to happen would be a
smaller "command line tools for Visual Studio" type package, like Apple
provides for Xcode, with only the compiler, tools, and libraries needed
to build C/C++ libraries or Python extensions but not the whole IDE.

Nick Coghlan:
> * wanting to build for Windows within the scope of an existing POSIX
> based build automation system
> * folks that prefer to use only open source software themselves
> wanting to make their projects available to Windows users

Don't forget: * MSVC is not capable of compiling the code I need to use.
That's the crux of the scientific-software problem. If it were possible to
use MSVC for everything, I'd probably suck it up and spend my time writing
CMake build systems for every project I wanted to get to work on Windows.
Most software I work with is a library first, Python extension second, if
at all.

Nick Coghlan:
> Lots of folks are happy with POSIX emulation layers on Windows, as
> they're OK with "basically works" rather than "works like any other
> native application". "Basically works" isn't sufficient for many
> Python-on-Windows use cases though, so the core ABI is a platform
> native one, rather than a POSIX emulation.

To clarify here: MinGW does not use a POSIX layer at all. Cygwin does,
MSYS does (it's based on Cygwin with relatively minor changes), but when
we're discussing the MinGW.org or MinGW-w64 compilers, there is no POSIX.
There is a GCC runtime library, which can be linked statically if you
choose, though it's not always a good idea to do so. Cygwin or MSYS provide
a POSIX build environment where you can run bash, make, etc, but with MinGW
you are not targeting or depending on that build environment with the
compiled binaries.

The distinction in runtime libraries is that MinGW links to msvcrt.dll,
which is from an old version of Visual Studio but is installed by default
on every version of Windows since XP. The python.org installers are
compiled with a particular more recent version of Visual Studio, so
they (and hence all extensions used with them) depend on the specific
corresponding msvcr##.dll.

Most MinGW developers and users would rather avoid the dll hell and issues
stemming from choosing a particular MSVC runtime and having to stick with
it for everything. NumPy was able to coax a custom MinGW-w64 build into
linking to CPython's designated runtime and statically link all the GCC
libraries (these are 2 separate issues however). I do think some sensitivity
to the requirements of the existing Windows CPython/PyPI environment from
the MinGW-w64 side would be valuable. If NumPy's needs can be met with
fewer modifications and without having to rebuild a custom GCC from source,
I think that would be beneficial for everyone. We'll see how many of the
MinGW-w64 developers I can get to agree with me on that. Any changes that
may need to be upstreamed to GCC will have their own set of challenges.

> Adding FORTRAN dependencies to the mix (which
> *none* of the platform vendors really support properly)

I'm not sure what you mean by platform vendors? Are you talking Conda,
Enthought, etc? To the best of my knowledge, they're using the Intel
toolchain for Fortran, at least on Windows and probably elsewhere. So
I suspect you'll find a libifcoremd.dll somewhere in those distributions.
That imposes some cost requirements on reproducing their processes that
are a bit high for average users to meet.

Donald Stufft:
> So that people upload a source distribution to PyPI and it kicks off
> Wheel builds on the various platforms automatically.
>
> What this looks like is a complete unknown, all I have is the general
> idea.

I do something similar today for C, C++, and Fortran libraries using
the MinGW-w64 cross-compiler. It looks like this
https://build.opensuse.org/project/show/windows:mingw:win64
I write a spec file and upload source, and get back dll's and exe's -
compressed in RPM's in this case, along with the RPM dependency metadata.
For OSX I would look at what Homebrew does with their "bottle"
infrastructure. A build farm with Vagrant (or similar) VM's could even
be made to do the same basic thing on Windows with MSVC, at least for
binaries that MSVC is capable of compiling.

-Tony