[SciPy-Dev] win32 binaries and scipy 0.17.0 release

[Matthew Brett]

Hi,

On Sat, Jan 2, 2016 at 5:30 PM, Tony Kelman <tony at kelman.net> wrote:
> Matthew Brett <matthew.brett <at> gmail.com> writes:
>
>> It would certainly be useful to share effort or at least discussion about
> this.
>>
>> Are there any good places to go to understand the logic behind
>> upstream Python's lack of interest in these patches / mingw-w64 Python
>> generally?  I'd love to hear how you are dealing with DLL hell and pip
>> on the msys2 system...
>
> I think the last time this was discussed was in the October 2014
> python-dev thread "Status of C compilers for windows"
> https://mail.python.org/pipermail/python-dev/2014-October/136607.html
> My personal conclusion from that was it could be cleaner to turn
> MSYS2's large patch set into a formal fork that anyone could build
> (or cross compile) even if not using MSYS2's pkgbuild system, but I
> didn't get too far there.
>
> My understanding of how the MSYS2 python works is that for pure
> python modules without C extensions, pip should work fine. There
> are issues with virtualenv last time I checked, and I don't know
> how careful the MSYS2 packagers are about checking the status of the
> test suites of all their C extension builds. If you get all C
> extensions from MSYS2 then there is no dll hell, but mixing C
> runtimes can of course cause problems.

The problem we're trying to solve, is making a toolchain that will
allow any project to build their own Windows .whl installers without
having to worry about how other projects were built.

My own feeling is having one central place building all these packages
is not going to scale, partly because continuous testing is so
important, and this can most efficiently be done and responded to by
the projects themselves.

We scientific Python persons are particularly concerned with packages
that need compilation.

That does put us at risk of DLL hell because many packages need to
link to external libraries, and two packages may need the same
library.  Or two versions of one package may get installed which have
different and incompatible versions of the same library.

> Some of their patches are
> in distutils to also make that recognize and use mingw-w64 compilers,
> but that would need some firsthand testing to see how well it works
> on C extension modules that haven't been packaged by MSYS2 yet.
>
> To be clear I think the current approach of trying to make mingwpy
> more robust and eventually get it upstreamed makes sense. But you're
> forking GCC with features (linking against recent msvc runtimes)
> upstream mingw-w64 doesn't want to use, as MSYS2 is forking cpython
> with features upstream python-dev doesn't want. It's kind of
> unfortunate all around.

Yes, it is unfortunate, it would be good to find some way of sharing
effort, and / or lobbying for the Python / pypi folks.

I can well see the attraction of the Mingw-w64-built Python depending
only on a single standard MS VC runtime, the current mix of VC
runtimes is a tiring burden.

Cheers,

Matthew