[New-bugs-announce] [issue24872] Add /NODEFAULTLIB:MSVCRT to _msvccompiler

[Steve Dower]

New submission from Steve Dower:

The C Runtime used for Python 3.5 and later consists of two DLLs. One, ucrtbase.dll, is an operating system component that can be assumed to be available and up to date (or will be installed by the Python installer).

The second DLL is vcruntimeXXX.dll, where XXX is a version number that is directly tied to the version of the compiler that was used to build. 

In order to maintain version-independence from the C Runtime, vcruntime should be statically, rather than dynamically, linked. This ensures that extensions linking to a different vcruntime will run on machines with builds of Python prior to that version being available (i.e. we'd have needed a time machine to be able to ensure all future versions of vcruntime are available). This would essentially put us back in a place where you need to match compilers to build extensions.

To achieve this semi-static linking, the following options must be used.

cl /c /MT /GL <file.c>
link <file.obj> /LTCG /NODEFAULTLIB:libucrt.lib ucrt.lib

Building with /MT links everything statically, and the last two linker options substitute the non-static ucrtbase.dll. /GL and /LTCG (link-time code generation) ensures the correct stubs for dynamic linking are created - compiling with /MD instead of /MT generates these at compile time instead of link time.

One problem appears to be including custom-built static libraries that compiled objects with /MD instead of /MT:

cl /C testlib.c /MD
lib testlib.c
cl /C test.c /MT /GL
link test.obj testlib.lib /LTCG /NODEFAULTLIB:libucrt.lib ucrt.lib
<linker errors occur>

These are because testlib.lib pulls in the MSVCRT library (which indirectly includes the dynamic vcruntime.lib and ucrt.lib) instead of the LIBCMT library (which indirectly includes the static libvcruntime.lib and libucrt.lib). Building test.c with /MT pulls in LIBCMT and conflicts ensue.

This may be fixed by using /MD for test.c, but this causes a dependency on vcruntimeXXX.dll. It appears that it can also be fixed by excluding MSVCRT from the final link step:

link test.obj testlib.lib /LTCG **/NODEFAULTLIB:msvcrt.lib** /NODEFAULTLIB:libucrt.lib ucrt.lib

Christoph - I know you've already patched your _msvccompiler.py to use /MD instead of /MT, but I wonder if you could try reverting that and adding the /NODEFAULTLIB:msvcrt.lib linker option instead and see if that helps? For my test cases it's been fine, but you have a much larger collection of libraries to work with. I'm very keen to see if this is a satisfactory solution.

FWIW, I've spent a lot of time considering ways we could actually include vcruntime*.dll such that old Python versions would include or get future versions of the DLL and have come up with nothing. Requiring users to install every VC redistributable that is released is unacceptable to me, and I would much rather help extension builders to create extensions that do not require extra DLLs.

----------
assignee: steve.dower
components: Windows
messages: 248646
nosy: cgohlke, paul.moore, steve.dower, tim.golden, zach.ware
priority: release blocker
severity: normal
status: open
title: Add /NODEFAULTLIB:MSVCRT to _msvccompiler
type: behavior
versions: Python 3.5, Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24872>
_______________________________________