[Python-Dev] Status of C compilers for Python on Windows

[Nick Coghlan]

(Paul Moore already covered most of this, but I'll go into a bit more
detail in a couple of areas)

On 29 October 2014 00:46, Tony Kelman <kelman at berkeley.edu> wrote:
> Stephen J. Turnbull:
>> It should be evident by now that our belief is that the large majority
>> of Windows users is well-served by the current model
>
>
> This is not the case at all in the scientific community. NumPy and SciPy
> put in a lot of extra work to come up with something that is compatible
> with the MSVC build of CPython because they have to, not because they're
> "happy to" jump through the necessary hoops.

Lots of folks are happy with POSIX emulation layers on Windows, as
they're OK with "basically works" rather than "works like any other
native application". "Basically works" isn't sufficient for many
Python-on-Windows use cases though, so the core ABI is a platform
native one, rather than a POSIX emulation.

This makes Python fit in more cleanly with other Windows applications,
but makes it harder to write Python applications that span both POSIX
and Windows. The degree to which a language runtime abstracts away the
details of the underlying operating system is a complex trade-off,
from C (where the C/C++ ABI is a core part of what *defines* an
operating system), through Python (which lets the developer largely
choose whether to use high level abstractions or lower level platform
specific APIs) to the JVM and CLR (which both largely abstract away
many of the details of the underlying operating system, aside from
carefully contained "native" code snippets).

This approach *does* fragment the community a bit, into at least
"Python-on-POSIX-only", "Python-on-Windows-only",
"platform-independent-Python" and "Python-on-POSIX-and-Windows" (where
the latter code has to deal directly with the differences between
Windows and POSIX because it needs the low level platform specific
APIs).

I personally consider that trade-off worth it, as it gives Python a
scope of applicability that more isolated runtimes lack, as well as
providing access to sophisticated platform level capabilities far more
cheaply than would be possible with a higher level of default
isolation.

> The situation today with
> NumPy AIUI is they already have to build with a custom hacked MinGW
> toolchain that only one person knows how to use. Evidently until very
> recently they were using a many-years-old 32-bit-only build of GCC 3.x.
> Do python-dev and numpy-discussion not talk to one another? I get that
> not everyone uses numpy, or Windows, but it sounds like there's very
> little understanding, or even awareness, of the issues they face.

There have been major ongoing communication problems stemming from the
wildly different assumptions that go into "programming as a tool for
building applications" (what python-dev and distutils-sig mostly do)
and "programming as a tool for thinking" (what scientists and data
analysts mostly do). Adding FORTRAN dependencies to the mix (which
*none* of the platform vendors really support properly) then makes the
whole problem of dealing with scientific tools even more alien to most
professional developers.

Those differing assumptions then meant that the two groups end up
frequently talk past each other because the worldviews are too
different. While that's starting to improve of late (as the above
distinction becomes better recognised), there's still a long way to
go.

Regards,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia