[Python-Dev] Status of C compilers for Python on Windows

[Paul Moore]

On 28 October 2014 14:46, Tony Kelman <kelman at berkeley.edu> wrote:
> Patches should be done well and accompanied with proper documentation
> so new functionality is fully reproducible. If that's what's holding
> up review, comments in the bug threads indicating as much would be
> helpful.

Typically that tends to be expressed as a terse "I can't get this
working". That's not ideal, but is an indication. When the response is
"but it's easy" (as it sometimes is) communication degenerates quite
fast. There's an example here in this thread - it took me a *long*
time, with help from a couple of people, to work out how to get a
mingw toolchain I could use to try things out. Even though the premise
of the whole discussion was "it's easy to set up a mingw toolchain"...

> In my opinion the MSVC toolchain makes that problem worse, as it's far
> harder for unix developers to have any familiarity with how things work.
> But you do have involvement and core developers from Microsoft which is
> obviously incredibly important. Maybe even mandatory for Python on Windows
> to be viable in your eyes.

One problem I've seen a lot on other projects is that when Unix
developers set up a comfortable, Unix-like environment on Windows
using something like mingw, they frequently aren't aware of crucial
differences between Unix and Windows and tend to write software that
even though it's Windows-native, *feels* Unixy to Windows users (don't
ask me to be more specific :-)). That's always going to happen, and
the people with Windows experience have to take up the slack by
keeping an eye out for such things, but setting the bar marginally
higher, to "you have to at least be willing to download and use a
native Windows compiler" can at least remind said Unix developers that
they are in a different environment.

That's *not* a criticism of anyone in the Python community, btw.
Typically the experience of Windows users is very well respected by
python core and package developers. But I tend to think that's partly
*because* we didn't take the "Unix-like toolchain" approach by
default.

> This is not the case at all in the scientific community. NumPy and SciPy
> put in a lot of extra work to come up with something that is compatible
> with the MSVC build of CPython because they have to, not because they're
> "happy to" jump through the necessary hoops. The situation today with
> NumPy AIUI is they already have to build with a custom hacked MinGW
> toolchain that only one person knows how to use. Evidently until very
> recently they were using a many-years-old 32-bit-only build of GCC 3.x.
> Do python-dev and numpy-discussion not talk to one another? I get that
> not everyone uses numpy, or Windows, but it sounds like there's very
> little understanding, or even awareness, of the issues they face.

Sadly, no. The numpy developers have always been a pretty much
separate world. We're seeing a bit more interaction these days, but
it's very limited and far from the level that's needed. The fault (if
that's the right word) probably lies on both sides. It's certainly not
purely the responsibility of the core team to build communication
links. As this thread has demonstrated, python-dev (and distutils-sig,
which is another place that desparately needs more numpy interaction)
is not the most welcoming of places for someone who is 100% focused on
the scientific stack, but conversely the scientific types do tend to
do their own thing, and sometimes when they do surface, they can dive
in with little appreciation of the wider picture. But we can get
along, and we can make progress (albeit not always as fast as either
party might like).

If all this thread has done is raise awareness of each others'
concerns, it's still been a win IMO.

> I'm going to move the "extensions with MinGW-w64" part of this conversation
> over to numpy-discussion, since they have a need for this today and are
> already using workarounds and processes that I don't think anyone is fully
> satisfied with. I do find this combination interesting, worth working on,
> and possible to make work well, but not completely in isolation as it does
> not address my embedding use case.

Please keep distutils-sig in the loop as well. I can't promise we'll
be able to help, but we should at least make sure we're not hindering
you, and we can make you aware of when your solutions won't work
outside your area of interest. Now the door is open, let's not close
it again.

Paul