[Python-Dev] PEP 0404 and VS 2010

[Steve Dower]

Nick Coghlan wrote:
> On 21 Nov 2013 10:33, "Antoine Pitrou" <solipsis at pitrou.net> wrote:
>> I think it isn't only about teaching it to build with VS 2010, but
>> providing binaries compatible with the VS 2010 runtime.
>> Otherwise, AFAIU, if extensions are built with VS 2010 but loader with
>> a VS 2008-compiled Python, there will be ABI problems.
>
> Right, this is the problem - building with newer compilers isn't an issue,
> changing the Windows ABI *is* (which is the reason Christian is proposing a
> version bump to denote the ABI incompatibility).
>
> If Visual Studio Express 2010 and later can be told to build against the VS2008
> C runtime, then that is something that can (and probably should) be enabled in
> the next CPython 2.7 maintenance release (for both the main build process and
> for distutils extension building).

Unfortunately, this is not the case. The compiler has intimate knowledge of the associated CRT and can't build against other versions (officially; if there's an unofficial way I genuinely don't know what it is). It is possible to use the older compiler from newer versions of Visual Studio, but that doesn't really solve any issues.

> Doing a 2.8 release *just* to change the Windows ABI to a new version of the MS
> C runtime, on the other hand, seems impossible to do without thoroughly
> confusing people (regardless of whether it's CPython or Stackless making such a
> change).

I agree, this is a bad idea. We'd also need to update distutils to detect the right version, and if we can do this then it's easier to update it to detect VC9 when installed from http://www.microsoft.com/en-us/download/details.aspx?id=3138 - currently it only finds VS2008, which installs things slightly differently, despite being an identical compiler.

I've had one go at making the change (http://bugs.python.org/issue7511), though that (a) had a bug and (b) broke people who monkey patch distutils to return a different vcvarsall.bat (including the distutils tests - my fix removed the dependency on vcvarsall.bat completely). Happy to revise it if there's interest in taking the change.

> I'd certainly want to explore all our alternatives with the Microsoft folks for
> getting our preferred option (new compiler, old C runtime) working in an open
> source friendly way before we went down the path of a 2.x ABI bump.

In my experience, extensions built with later compilers work fine, provided you stay away from the FILE* APIs. One of my side projects is C++11 bindings for hosting/extending Python, and I run my tests with VC12 (Visual Studio 2013) against python.org's 2.6, 2.7, 3.2 and 3.3 binaries without issue.

What may be best is to detect MSVC>9.0 in the headers and simply block the dangerous APIs when building sdists. Then extensions can be built with any available version of MSVC, provided they are "safe", and the build will fail if they aren't.

There are still some ways the differing CRTs can cause issues - alloc/free pairing, for example - so some macros may also have to become exported functions. After that I don't think there are any ways to accidentally cause issues; you'd really have to be doing strange things (such as passing pointers to CRT types between extensions via capsules).

> I've cc'ed Steve Dower directly, as he's the best person I know to ask about
> this kind of problem.

Cheers,
Steve