Re: [Python-ideas] Update the required C compiler for Windows to a supported version.
Stephen J. Turnbull wrote:
Vernon D. Cole writes:
I cannot compile a Python extension module with any Microsoft compiler I can obtain.
Your pain is understood, but it's not simple to address it.
FWIW, I'm working on making the compiler easily obtainable. The VS 2008 link that was posted is unofficial, and could theoretically disappear at any time (I'm not in control of that), but the Windows SDK for Windows 7 and .NET 3.5 SP1 (http://www.microsoft.com/en-us/download/details.aspx?id=3138) should be around for as long as Windows 7 is supported. The correct compiler (VC9) is included in this SDK, but unfortunately does not install the vcvarsall.bat file that distutils expects. (Though it's pretty simple to add one that will switch on %1 and call the correct vcvars(86|64|...).bat.)
The SDK needed for Python 3.3 and 3.4 (VC10) is even worse - there are many files missing. I'm hoping we'll be able to set up some sort of downloadable package/tool that will fix this. While we'd obviously love to move CPython onto our latest compilers, it's simply not possible (for good reason). Python 3.4 is presumably locked to VC10, but hopefully 3.5 will be able to use whichever version is current when that decision is made.
The basic problem is that the ABI changes. Therefore it's going to require a complete new set of *all* C extensions for Windows, and the duplication of download links for all those extensions from quite a few different vendors is likely to confuse a lot of users.
Specifically, the CRT changes. The CRT is an interesting mess of data structures that are exposed in header files, which means while you can have multiple CRTs loaded, they cannot touch each other's data structures at all or things will go bad/crash, and there's no nice way to set it up to avoid this (my colleague who currently owns MSVCRT suggested a not-very-nice way to do it, but I don't think it's going to be reliable enough). Python's stable ABI helps, but does not solve this problem.
The file APIs are the worst culprits. The layout of FILE* objects can and do change between CRT versions, and file descriptors are simply indices into an array of these objects that is exposed through macros rather than function calls. As a result, you cannot mix either FILE pointers or file descriptors between CRTs. The only safe option is to build with the matching CRT, and for MSVCRT, this means with the matching compiler. It's unfortunate, and the responsible teams are well aware of the limitation, but it's history at this point, so we have no choice but to work with it.
Cheers, Steve
Hi Steve,
On 27.11.2013 19:24, Steve Dower wrote:
Stephen J. Turnbull wrote:
Vernon D. Cole writes:
I cannot compile a Python extension module with any Microsoft compiler I can obtain.
Your pain is understood, but it's not simple to address it.
FWIW, I'm working on making the compiler easily obtainable. The VS 2008 link that was posted is unofficial, and could theoretically disappear at any time (I'm not in control of that), but the Windows SDK for Windows 7 and .NET 3.5 SP1 (http://www.microsoft.com/en-us/download/details.aspx?id=3138) should be around for as long as Windows 7 is supported. The correct compiler (VC9) is included in this SDK, but unfortunately does not install the vcvarsall.bat file that distutils expects. (Though it's pretty simple to add one that will switch on %1 and call the correct vcvars(86|64|...).bat.)
The SDK needed for Python 3.3 and 3.4 (VC10) is even worse - there are many files missing. I'm hoping we'll be able to set up some sort of downloadable package/tool that will fix this. While we'd obviously love to move CPython onto our latest compilers, it's simply not possible (for good reason). Python 3.4 is presumably locked to VC10, but hopefully 3.5 will be able to use whichever version is current when that decision is made.
I would expect that Python is not the only OSS tool that requires to use the same compiler version for extensions as the one used to compile the main application.
With Microsoft opening up towards OSS software, wouldn't it be better to keep the express versions available officially for a much longer period or allow 3rd parties to host the express ISOs ?
If there's anything the Python Software Foundation could do to help with this, please let me know.
Thanks,
On 28 November 2013 04:24, Steve Dower Steve.Dower@microsoft.com wrote:
The SDK needed for Python 3.3 and 3.4 (VC10) is even worse - there are many files missing. I'm hoping we'll be able to set up some sort of downloadable package/tool that will fix this. While we'd obviously love to move CPython onto our latest compilers, it's simply not possible (for good reason). Python 3.4 is presumably locked to VC10, but hopefully 3.5 will be able to use whichever version is current when that decision is made.
The main challenge we face on that front is the fact that Visual Studio 2013 is Windows 7+ only, so we'd potentially be causing problems for people still developing on XP or Vista if we update.
Windows XP goes EOL next month, so that will be dropped for 3.5 next year in accordance with PEP 11.
However, Vista is still in extended support until 2017 - under the current PEP 11 guidelines, that means it will be supported for at least 3.5, and likely 3.6 as well. If we update the required compiler we'd end up in a situation where CPython 3.5 nominally supports Vista, but Vista users would still need at least one Windows 7 machine to install the toolchain to create compatible C extensions. That may be OK given the number of people that jumped straight from XP to Windows 7, but it's still a consideration that would need to be taken into account.
Longer term, we'd love to add a build farm to PyPI to take the drudgery out of creating pre-compiled wheel files (and MSI installers for Windows), and that would potentially address this problem as well (since the build farm would include the requisite Windows 7 images). However, I currently think it would be rather optimistic to assume that we'll have that in place by the time 3.5 rolls around (there's a lot of other things that need to be addressed before a build farm will make it to the top of the distutils-sig todo list).
Cheers, Nick.
On 02/03/2014 12:39 am, Nick Coghlan wrote:
Windows XP goes EOL next month, so that will be dropped for 3.5 next year in accordance with PEP 11.
However, Vista is still in extended support until 2017 - under the current PEP 11 guidelines, that means it will be supported for at least 3.5, and likely 3.6 as well.
BTW, Windows Server 2003 has extended support until 14 July 2015. Will it be supported in Python 3.5?
-- Richard
participants (4)
-
M.-A. Lemburg -
Nick Coghlan -
Richard Oudkerk -
Steve Dower