runtime dlls on Windows

Hi folks, The standard build of Py3.5 for Windows is built with VS2015 (correct??) And it includes the runtime dlls it needs. However, we've found that wxPython wheels for win32 (not sure about win64) also need: MSVCP140.DLL So: wxPython could include that of course, But it looks like it's getting included with the Matplotlib wheels already (so folks with Matplotlib can run wx....). I'm just guessing, but this looks like the standard run time for C++ with that compiler. Python itself doesn't use C++, of course, but maybe we should include that dll with Python anyway -- that way folks can build wheels of packages with C++ extensions in the normal way, and those wheels will "just work", and we don't have to have every individual package ship the same dll. -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov

Wouldn't downloading the Microsoft C++ Runtime 2015 also work? Many recent computers already have it pre-installed. -- Ryan [ERROR]: Your autotools build scripts are 200 lines longer than your program. Something’s wrong. http://kirbyfan64.github.io/ On May 25, 2016 2:31 PM, "Chris Barker" <chris.barker@noaa.gov> wrote:

On Wed, May 25, 2016 at 12:37 PM, Ryan Gonzalez <rymg19@gmail.com> wrote:
Wouldn't downloading the Microsoft C++ Runtime 2015 also work?
I'm sure it would -- I know that installing the entire MSVC2015 Community Edition does... but the point here is that end users should be able to: pip install something and if there is a binary wheel for something, it should work without them having to install something else. (why MS doesn't ship ALL their runtimes with eh OS is beyond me...) As it stands now, there are two options: 1) users need to install something else themselves: the runtime, that particular dll, the compiler... 2) every package maintainer that uses C++ needs to ship that dll with the binary wheels. If we put the dll into the python binary, then it would "just work" -CHB
-- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov

On Thu, May 26, 2016 at 6:26 AM, Chris Barker <chris.barker@noaa.gov> wrote:
Agreed with that, but meh.
Counting your conclusion as option 3, you're offering the status quo and two competing solutions to it. I agree that the status quo is unideal; a binary wheel should either include or be able to fetch everything necessary for a supported platform. But why should CPython package a runtime that it doesn't use? Which is more common - someone uses two C++ modules, or someone uses none of them? I don't know how hard it is for the wheels to ship the DLL ("hard" here including any licensing or versioning issues, as well as the actual effort involved), but from a purely logical standpoint, it seems the most sane option. So I'm in favour of your option 2. ChrisA

On Wed, May 25, 2016 at 1:40 PM, Chris Angelico <rosuav@gmail.com> wrote:
But why should CPython package a runtime that it doesn't use?
Because it IS part of the "standard runtime environment" that cPython is providing. The cPython Windows builds are built with a particular compiler that expects a particular runtime. C++ is part of that runtime, even if the cPython code itself doesn't use that part. I'm sure there are features in the C runtime DLL that cPython doesn't happen to use, either. The fact that it's a separate dll is kind of an implementation detail.
Which is more common - someone uses two C++ modules, or someone uses none of them?
Irrelevant -- which is the bigger problem for users? a lot of folks getting an extra dll as part of a large install that they aren't using, or folks getting cryptic error messages and/or every package builder of C++ packages having to identify and solve the problem? BUT -- Steve Dower seems to have identified that the wonders of dll hell never cease, and this isn't possible anyway. Oh well. I don't know how hard it is for the wheels to ship the DLL ("hard"
here including any licensing or versioning issues, as well as the actual effort involved),
It's not all that hard. but one more step that has to be done. I wonder if setuptools / wheel could automate this somehow? -- hmm. On Wed, May 25, 2016 at 2:05 PM, Bernardo Sulzbach < mafagafogigante@gmail.com> wrote:
I don't know how clear it will be for the average Python user on Windows that a C++ runtime is missing.
That's the key problem -- the error message is NOT helpful -- it makes it look like the extension dll is missing, when it's actually that the extension dll can't link to the runtime dll -- but the message doesn't say what dll it's trying to link to. sigh. You need to be a pretty sophisticated user to figure it out -- or use Google :-) On Wed, May 25, 2016 at 2:52 PM, Nathaniel Smith <njs@pobox.com> wrote:
The other challenge with this proposal is that Python 3.5.0 and 3.5.1 have already shipped without this .dll.
yeah, maybe only 3.6 then, if possible at all.
(unless the user has that dll ;-) ) And to be clear, this is going to effect any package that used C++ -- those two are two I happen to be familiar with. There can't be a small number of such packages. (hmm -- GDAL, anything Boost-based, anything else that uses AGG, .....)
Yes -- if pynativelib becomes standard practice, then yes, that would be a good way to handle this issue -- jsut ike any other lib multiple packages might need. (I really need to get back to that... if anyone wants to help sprint
on this at PyCon let me know :-))
I might -- haven't decided what to sprint on yet -- let's talk. (let's talk anyway) On Wed, May 25, 2016 at 2:20 PM, Steve Dower <steve.dower@python.org> wrote:
got to love Windows! Does the OS do the right thing if they are both installed? ouch!
oh God. Well, that will pretty much kill it. So essentially every single app or lib written in C++ has to ship this dll? And IIUC, folks that have the compiler installed have SOME version on the system paths, and who knows what version other arbitrary stuff is linked to? How can this possibly be considered "Enterprise" software? Wow! Importantly, it will break forward compatibility. We are already on the
wow. So could that be extended to do the same thing with MSVCP140.DLL? then each package would be shipping it -- but it would happen automagically, rather than everyone having to figure out that they have to do it, and how to do it.
yeah, a line has to be drawn, and "created by Microsoft" wouldn't make any sense either. but "standard runtime expected by the compiler" would -- but, as you say above, you have may have to ship a new version with your extension anyway -- so it seems adding the feature for distutils to do that for C++ extensions would both solve the problem and not ship anything extra with the core install.
Though we have to do that for msvcp140.dll anyway, yes? god, what a pain! (and why MS should ship the damn things themselves!) Hopefully that helps explain the position somewhat. I'm happy to go into
Thanks Steve. Will you be at PyCon? if Nathaniel and I look at this during the sprints, maybe you could coach us a bit. Anyway, proposal withdrawn. Replaced with possible proposal to have distutils include the C++ runtime dll like it does now for the C one. -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov

On Fri, May 27, 2016 at 2:42 AM, Chris Barker <chris.barker@noaa.gov> wrote:
I'm not sure that it is an implementation detail; it's indicative that the compiler/runtime designers intended for the C runtime to be able to be distributed without the C++ runtime. You wouldn't expect to have to ship a Fortran runtime and a Cobol runtime, even if MSVC happened to package up compilers for all of the above, would you?
I agree with you that end users getting cryptic error messages is a problem. But the choice of solutions is: 1) CPython ships the C++ runtime that it doesn't use, in case someone pip-installs something that needs it 2) Package authors who choose to use C++ must ship the C++ runtime. Even apart from the DLL Hell that Steve mentioned, the second option appears more sane. Package authors get the freedom to choose any runtime dependencies they want, and then they package them or make it possible to obtain them.
BUT -- Steve Dower seems to have identified that the wonders of dll hell never cease, and this isn't possible anyway. Oh well.
I'm not entirely grasping what's happening here. There are multiple versions of msvcp140.dll floating around out there; what happens if one app requires one of them and another requires another? Can you simply say "take the newer one"? Is there a way to instantly recognize that there's a newer one available, and ignore the older one? Versioning is a *pain*. ChrisA

On Thu, May 26, 2016 at 9:53 AM, Chris Angelico <rosuav@gmail.com> wrote:
me neither :-(
That's pretty much the dll hell that has plagued Windows FOREVER! why they don't do some versioning is totally beyond me, though 8.3 filenames have not been a restriction for a long time.
Can you simply say "take the newer one"?
I *think* that standard Windows policy/practice is that newer versions of a dll with the same name should always be backward compatible -- so something built against an older version should work with the newer one -- so you can ship a dll with a security patch, and it will all work. I presume that MS itself is careful about this with their runtimes.
Is there a way to instantly recognize that there's a newer one available, and ignore the older one?
I don't think so -- the linker will pick the first one it finds on the search path :-( -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov

Chris Angelico <rosuav@gmail.com> wrote:
Windows executables can and should ask for specific versions of the DLLs in the manifest (an XML file embedded in the EXE). Details here: https://en.wikipedia.org/wiki/Side-by-side_assembly Sebastian

On 26May2016 1453, Sebastian Krause wrote:
This has been deprecated. It sounded like a great idea at the time (~8 years ago) but caused more problems than it solved. Python 2.7 and 3.2 (IIRC) were the last versions to use this feature. The way to get the newest version is to run the installer, which will upgrade if it has a newer version or leave the old one alone. Cheers, Steve

On 5/26/2016 3:18 PM, Steve Dower wrote:
This has been deprecated. It sounded like a great idea at the time (~8 years ago) but caused more problems than it solved.
Somehow I missed the announcement of the deprecation. When I first heard of Windows manifests, though, my first reaction was that it sounded like a solution that was more complex than the problem it was trying to solve. Glad to hear they've been deprecated.

On 26May2016 1601, Glenn Linderman wrote:
The feature itself probably hasn't been officially deprecated, but using it for the C and C++ runtime libraries has not been required for a long time now. (I need to be more careful that I don't accidentally announce things that may not be true :) )
Indeed. Cheers, Steve

On 26/05/2016 19:02, Brett Cannon wrote:
Steve writes such an excellent blog that when and if he has time I'm certain that he'll put something together. With people such as him following on from Martin Loewis, Tim Golden, Mark "I've forgotten more about Python on Windows than you'll ever know" Hammond and Andy "ditto" Robinson I can't say that I'll lose too much sleep over it. As I suffer insomnia on top of chronic fatigue syndrome, that is quite a statement for me to make. -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence

On 26May2016 0942, Chris Barker wrote:
This seems like a good solution. I won't be around for sprints, but we can chat about it before then.
If they're installed properly into System32/SysWOW64 (using the official installer), then yes. If you simply drop them into your Python install directory, then no, unless you drop the right one - it has to match the python.exe architecture.
To make it worse, you can only load it once per process. So the first extension to load their private copy wins. The DLL is designed to be installed using the official installer, statically linked, or embedded as part of an entire application. It's not meant to be injected into someone else's application (which is essentially what Python extension modules are doing).
It would be possible. It might be simpler to update the default compiler settings to statically link the C++ runtime while still dynamically linking the C runtime (and any project is already capable of doing this on their own - I'm 99% sure it works).
Maybe, but it's also the same reason why MS doesn't want to ship it themselves either. In fact, all the reasons why we don't want to include it in Python equally apply to Windows - until it's a direct dependency of the product itself, it has a really high bar to be included (SQLite is included in Windows because it is a dependency, not just as a convenience to developers).
The big difference here is that msvcp140.dll contains significant functionality while vcruntime140.dll is a very small amount of shared state and initialization. Changes/additions/fixes to msvcp140.dll are far more likely as a result. I don't recall whether pip extensions are a thing yet, but if they are this would be a great candidate for that (trigger the runtime installer if necessary). Otherwise, having a package that only includes this DLL would probably be fine too, though it really only defers the problem to the next DLL you depend on. Cheers, Steve

Another reason why we don't want to install this DLL with Python :) Per-user install is properly supported in 3.5 for I believe the first time ever. But yes, installing them properly requires admin privileges. An unfortunately messy situation, one that I'm happy to not have to resolve for Python's incredibly diverse user base. Top-posted from my Windows Phone -----Original Message----- From: "Chris Angelico" <rosuav@gmail.com> Sent: 5/26/2016 17:04 Cc: "python-dev" <python-dev@python.org> Subject: Re: [Python-Dev] runtime dlls on Windows On Fri, May 27, 2016 at 5:13 AM, Steve Dower <steve.dower@python.org> wrote:
What if the Python installation isn't being run as admin? Or is that deprecated too? ChrisA _______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/steve.dower%40python.org

On 05/25/2016 04:37 PM, Ryan Gonzalez wrote:
Wouldn't downloading the Microsoft C++ Runtime 2015 also work? Many recent computers already have it pre-installed.
Even though the download seems to be only 14 MB (I don't have a Windows machine, so I cannot assure that just the file on MS website is enough), it is an inconvenience. Additionally, I don't know how clear it will be for the average Python user on Windows that a C++ runtime is missing.

On 25May2016 1229, Chris Barker wrote:
There are two different versions of this DLL for each architecture (same name).
Unfortunately, it won't "just work". More often than not, it will break in weird and very difficult to diagnose ways, as the version of msvcp140.dll changes on a regular basis (there are at least four different version of it "in the wild" since VS 2015 released, not counting the preview releases before mid-last year). Importantly, it will break forward compatibility. We are already on the hook to keep shipping vcruntime140.dll for all 3.5 (and probably 3.6) releases, and if binary packages are built with later versions of VS then they'll need to include a (hypothetical) vcruntime150.dll (which distutils will already do, because I added that functionality). Other than that, every dependency of Python 3.5+ is forwards-compatible with new operating systems and compilers. There's also a slippery slope argument that would legitimately justify shipping all sorts of "common" dependencies with the core product. I chose to draw the line at "needed by the core Python product", rather than "created by Microsoft" or "used by many packages". (Hence vcruntime140.dll is included, despite not having forwards-compatibility, and I tried real hard to avoid including that one too.) Finally, unless we demand users are administrators for every install, we will quickly have many Python installs using versions of msvcp140.dll (or any other dependency) with security vulnerabilities, with no way to update them other than a full Python release. Installing the regular runtime (which is patched automatically) or as part of a package (which can be independently updated) is far more feasible. I really don't want to be doing security updates for 15 years just to keep an unused DLL secure. Hopefully that helps explain the position somewhat. I'm happy to go into more detail on any of these issues if anyone would like (potentially just linking to where I've previously discussed it either here, on bugs.p.o or on my blog). Cheers, Steve

On Wed, May 25, 2016 at 12:29 PM, Chris Barker <chris.barker@noaa.gov> wrote:
The other challenge with this proposal is that Python 3.5.0 and 3.5.1 have already shipped without this .dll. So the most we could hope for is for it to be included in 3.5.2, and then wxPython and Matplotlib would either have to continue shipping it anyway, or else accept that their wheels actually require 3.5.2+ and will be broken if installed onto 3.5.0 or 3.5.1. And unfortunately there's no way in wheel metadata to express this kind of requirement -- wheels assume that all 3.5.x are both forwards and backwards compatible. So it would just be a weird unexplained breakage that some users would see. An alternative approach would be to stick MSVCP140.DLL into a tiny shim wheel and upload that to PyPI, and then wxPython and matplotlib's windows wheels could declare a dependency on this msvcp410 wheel. Basically this is the idea of my pynativelib proposal, though in this case you would only be using a small piece of the full proposal: https://mail.python.org/pipermail/wheel-builders/2016-April/000090.html (I really need to get back to that... if anyone wants to help sprint on this at PyCon let me know :-)) -n -- Nathaniel J. Smith -- https://vorpus.org

Wouldn't downloading the Microsoft C++ Runtime 2015 also work? Many recent computers already have it pre-installed. -- Ryan [ERROR]: Your autotools build scripts are 200 lines longer than your program. Something’s wrong. http://kirbyfan64.github.io/ On May 25, 2016 2:31 PM, "Chris Barker" <chris.barker@noaa.gov> wrote:

On Wed, May 25, 2016 at 12:37 PM, Ryan Gonzalez <rymg19@gmail.com> wrote:
Wouldn't downloading the Microsoft C++ Runtime 2015 also work?
I'm sure it would -- I know that installing the entire MSVC2015 Community Edition does... but the point here is that end users should be able to: pip install something and if there is a binary wheel for something, it should work without them having to install something else. (why MS doesn't ship ALL their runtimes with eh OS is beyond me...) As it stands now, there are two options: 1) users need to install something else themselves: the runtime, that particular dll, the compiler... 2) every package maintainer that uses C++ needs to ship that dll with the binary wheels. If we put the dll into the python binary, then it would "just work" -CHB
-- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov

On Thu, May 26, 2016 at 6:26 AM, Chris Barker <chris.barker@noaa.gov> wrote:
Agreed with that, but meh.
Counting your conclusion as option 3, you're offering the status quo and two competing solutions to it. I agree that the status quo is unideal; a binary wheel should either include or be able to fetch everything necessary for a supported platform. But why should CPython package a runtime that it doesn't use? Which is more common - someone uses two C++ modules, or someone uses none of them? I don't know how hard it is for the wheels to ship the DLL ("hard" here including any licensing or versioning issues, as well as the actual effort involved), but from a purely logical standpoint, it seems the most sane option. So I'm in favour of your option 2. ChrisA

On Wed, May 25, 2016 at 1:40 PM, Chris Angelico <rosuav@gmail.com> wrote:
But why should CPython package a runtime that it doesn't use?
Because it IS part of the "standard runtime environment" that cPython is providing. The cPython Windows builds are built with a particular compiler that expects a particular runtime. C++ is part of that runtime, even if the cPython code itself doesn't use that part. I'm sure there are features in the C runtime DLL that cPython doesn't happen to use, either. The fact that it's a separate dll is kind of an implementation detail.
Which is more common - someone uses two C++ modules, or someone uses none of them?
Irrelevant -- which is the bigger problem for users? a lot of folks getting an extra dll as part of a large install that they aren't using, or folks getting cryptic error messages and/or every package builder of C++ packages having to identify and solve the problem? BUT -- Steve Dower seems to have identified that the wonders of dll hell never cease, and this isn't possible anyway. Oh well. I don't know how hard it is for the wheels to ship the DLL ("hard"
here including any licensing or versioning issues, as well as the actual effort involved),
It's not all that hard. but one more step that has to be done. I wonder if setuptools / wheel could automate this somehow? -- hmm. On Wed, May 25, 2016 at 2:05 PM, Bernardo Sulzbach < mafagafogigante@gmail.com> wrote:
I don't know how clear it will be for the average Python user on Windows that a C++ runtime is missing.
That's the key problem -- the error message is NOT helpful -- it makes it look like the extension dll is missing, when it's actually that the extension dll can't link to the runtime dll -- but the message doesn't say what dll it's trying to link to. sigh. You need to be a pretty sophisticated user to figure it out -- or use Google :-) On Wed, May 25, 2016 at 2:52 PM, Nathaniel Smith <njs@pobox.com> wrote:
The other challenge with this proposal is that Python 3.5.0 and 3.5.1 have already shipped without this .dll.
yeah, maybe only 3.6 then, if possible at all.
(unless the user has that dll ;-) ) And to be clear, this is going to effect any package that used C++ -- those two are two I happen to be familiar with. There can't be a small number of such packages. (hmm -- GDAL, anything Boost-based, anything else that uses AGG, .....)
Yes -- if pynativelib becomes standard practice, then yes, that would be a good way to handle this issue -- jsut ike any other lib multiple packages might need. (I really need to get back to that... if anyone wants to help sprint
on this at PyCon let me know :-))
I might -- haven't decided what to sprint on yet -- let's talk. (let's talk anyway) On Wed, May 25, 2016 at 2:20 PM, Steve Dower <steve.dower@python.org> wrote:
got to love Windows! Does the OS do the right thing if they are both installed? ouch!
oh God. Well, that will pretty much kill it. So essentially every single app or lib written in C++ has to ship this dll? And IIUC, folks that have the compiler installed have SOME version on the system paths, and who knows what version other arbitrary stuff is linked to? How can this possibly be considered "Enterprise" software? Wow! Importantly, it will break forward compatibility. We are already on the
wow. So could that be extended to do the same thing with MSVCP140.DLL? then each package would be shipping it -- but it would happen automagically, rather than everyone having to figure out that they have to do it, and how to do it.
yeah, a line has to be drawn, and "created by Microsoft" wouldn't make any sense either. but "standard runtime expected by the compiler" would -- but, as you say above, you have may have to ship a new version with your extension anyway -- so it seems adding the feature for distutils to do that for C++ extensions would both solve the problem and not ship anything extra with the core install.
Though we have to do that for msvcp140.dll anyway, yes? god, what a pain! (and why MS should ship the damn things themselves!) Hopefully that helps explain the position somewhat. I'm happy to go into
Thanks Steve. Will you be at PyCon? if Nathaniel and I look at this during the sprints, maybe you could coach us a bit. Anyway, proposal withdrawn. Replaced with possible proposal to have distutils include the C++ runtime dll like it does now for the C one. -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov

On Fri, May 27, 2016 at 2:42 AM, Chris Barker <chris.barker@noaa.gov> wrote:
I'm not sure that it is an implementation detail; it's indicative that the compiler/runtime designers intended for the C runtime to be able to be distributed without the C++ runtime. You wouldn't expect to have to ship a Fortran runtime and a Cobol runtime, even if MSVC happened to package up compilers for all of the above, would you?
I agree with you that end users getting cryptic error messages is a problem. But the choice of solutions is: 1) CPython ships the C++ runtime that it doesn't use, in case someone pip-installs something that needs it 2) Package authors who choose to use C++ must ship the C++ runtime. Even apart from the DLL Hell that Steve mentioned, the second option appears more sane. Package authors get the freedom to choose any runtime dependencies they want, and then they package them or make it possible to obtain them.
BUT -- Steve Dower seems to have identified that the wonders of dll hell never cease, and this isn't possible anyway. Oh well.
I'm not entirely grasping what's happening here. There are multiple versions of msvcp140.dll floating around out there; what happens if one app requires one of them and another requires another? Can you simply say "take the newer one"? Is there a way to instantly recognize that there's a newer one available, and ignore the older one? Versioning is a *pain*. ChrisA

On Thu, May 26, 2016 at 9:53 AM, Chris Angelico <rosuav@gmail.com> wrote:
me neither :-(
That's pretty much the dll hell that has plagued Windows FOREVER! why they don't do some versioning is totally beyond me, though 8.3 filenames have not been a restriction for a long time.
Can you simply say "take the newer one"?
I *think* that standard Windows policy/practice is that newer versions of a dll with the same name should always be backward compatible -- so something built against an older version should work with the newer one -- so you can ship a dll with a security patch, and it will all work. I presume that MS itself is careful about this with their runtimes.
Is there a way to instantly recognize that there's a newer one available, and ignore the older one?
I don't think so -- the linker will pick the first one it finds on the search path :-( -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov

Chris Angelico <rosuav@gmail.com> wrote:
Windows executables can and should ask for specific versions of the DLLs in the manifest (an XML file embedded in the EXE). Details here: https://en.wikipedia.org/wiki/Side-by-side_assembly Sebastian

On 26May2016 1453, Sebastian Krause wrote:
This has been deprecated. It sounded like a great idea at the time (~8 years ago) but caused more problems than it solved. Python 2.7 and 3.2 (IIRC) were the last versions to use this feature. The way to get the newest version is to run the installer, which will upgrade if it has a newer version or leave the old one alone. Cheers, Steve

On 5/26/2016 3:18 PM, Steve Dower wrote:
This has been deprecated. It sounded like a great idea at the time (~8 years ago) but caused more problems than it solved.
Somehow I missed the announcement of the deprecation. When I first heard of Windows manifests, though, my first reaction was that it sounded like a solution that was more complex than the problem it was trying to solve. Glad to hear they've been deprecated.

On 26May2016 1601, Glenn Linderman wrote:
The feature itself probably hasn't been officially deprecated, but using it for the C and C++ runtime libraries has not been required for a long time now. (I need to be more careful that I don't accidentally announce things that may not be true :) )
Indeed. Cheers, Steve

On 26/05/2016 19:02, Brett Cannon wrote:
Steve writes such an excellent blog that when and if he has time I'm certain that he'll put something together. With people such as him following on from Martin Loewis, Tim Golden, Mark "I've forgotten more about Python on Windows than you'll ever know" Hammond and Andy "ditto" Robinson I can't say that I'll lose too much sleep over it. As I suffer insomnia on top of chronic fatigue syndrome, that is quite a statement for me to make. -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence

On 26May2016 0942, Chris Barker wrote:
This seems like a good solution. I won't be around for sprints, but we can chat about it before then.
If they're installed properly into System32/SysWOW64 (using the official installer), then yes. If you simply drop them into your Python install directory, then no, unless you drop the right one - it has to match the python.exe architecture.
To make it worse, you can only load it once per process. So the first extension to load their private copy wins. The DLL is designed to be installed using the official installer, statically linked, or embedded as part of an entire application. It's not meant to be injected into someone else's application (which is essentially what Python extension modules are doing).
It would be possible. It might be simpler to update the default compiler settings to statically link the C++ runtime while still dynamically linking the C runtime (and any project is already capable of doing this on their own - I'm 99% sure it works).
Maybe, but it's also the same reason why MS doesn't want to ship it themselves either. In fact, all the reasons why we don't want to include it in Python equally apply to Windows - until it's a direct dependency of the product itself, it has a really high bar to be included (SQLite is included in Windows because it is a dependency, not just as a convenience to developers).
The big difference here is that msvcp140.dll contains significant functionality while vcruntime140.dll is a very small amount of shared state and initialization. Changes/additions/fixes to msvcp140.dll are far more likely as a result. I don't recall whether pip extensions are a thing yet, but if they are this would be a great candidate for that (trigger the runtime installer if necessary). Otherwise, having a package that only includes this DLL would probably be fine too, though it really only defers the problem to the next DLL you depend on. Cheers, Steve

Another reason why we don't want to install this DLL with Python :) Per-user install is properly supported in 3.5 for I believe the first time ever. But yes, installing them properly requires admin privileges. An unfortunately messy situation, one that I'm happy to not have to resolve for Python's incredibly diverse user base. Top-posted from my Windows Phone -----Original Message----- From: "Chris Angelico" <rosuav@gmail.com> Sent: 5/26/2016 17:04 Cc: "python-dev" <python-dev@python.org> Subject: Re: [Python-Dev] runtime dlls on Windows On Fri, May 27, 2016 at 5:13 AM, Steve Dower <steve.dower@python.org> wrote:
What if the Python installation isn't being run as admin? Or is that deprecated too? ChrisA _______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/steve.dower%40python.org

On 05/25/2016 04:37 PM, Ryan Gonzalez wrote:
Wouldn't downloading the Microsoft C++ Runtime 2015 also work? Many recent computers already have it pre-installed.
Even though the download seems to be only 14 MB (I don't have a Windows machine, so I cannot assure that just the file on MS website is enough), it is an inconvenience. Additionally, I don't know how clear it will be for the average Python user on Windows that a C++ runtime is missing.

On 25May2016 1229, Chris Barker wrote:
There are two different versions of this DLL for each architecture (same name).
Unfortunately, it won't "just work". More often than not, it will break in weird and very difficult to diagnose ways, as the version of msvcp140.dll changes on a regular basis (there are at least four different version of it "in the wild" since VS 2015 released, not counting the preview releases before mid-last year). Importantly, it will break forward compatibility. We are already on the hook to keep shipping vcruntime140.dll for all 3.5 (and probably 3.6) releases, and if binary packages are built with later versions of VS then they'll need to include a (hypothetical) vcruntime150.dll (which distutils will already do, because I added that functionality). Other than that, every dependency of Python 3.5+ is forwards-compatible with new operating systems and compilers. There's also a slippery slope argument that would legitimately justify shipping all sorts of "common" dependencies with the core product. I chose to draw the line at "needed by the core Python product", rather than "created by Microsoft" or "used by many packages". (Hence vcruntime140.dll is included, despite not having forwards-compatibility, and I tried real hard to avoid including that one too.) Finally, unless we demand users are administrators for every install, we will quickly have many Python installs using versions of msvcp140.dll (or any other dependency) with security vulnerabilities, with no way to update them other than a full Python release. Installing the regular runtime (which is patched automatically) or as part of a package (which can be independently updated) is far more feasible. I really don't want to be doing security updates for 15 years just to keep an unused DLL secure. Hopefully that helps explain the position somewhat. I'm happy to go into more detail on any of these issues if anyone would like (potentially just linking to where I've previously discussed it either here, on bugs.p.o or on my blog). Cheers, Steve

On Wed, May 25, 2016 at 12:29 PM, Chris Barker <chris.barker@noaa.gov> wrote:
The other challenge with this proposal is that Python 3.5.0 and 3.5.1 have already shipped without this .dll. So the most we could hope for is for it to be included in 3.5.2, and then wxPython and Matplotlib would either have to continue shipping it anyway, or else accept that their wheels actually require 3.5.2+ and will be broken if installed onto 3.5.0 or 3.5.1. And unfortunately there's no way in wheel metadata to express this kind of requirement -- wheels assume that all 3.5.x are both forwards and backwards compatible. So it would just be a weird unexplained breakage that some users would see. An alternative approach would be to stick MSVCP140.DLL into a tiny shim wheel and upload that to PyPI, and then wxPython and matplotlib's windows wheels could declare a dependency on this msvcp410 wheel. Basically this is the idea of my pynativelib proposal, though in this case you would only be using a small piece of the full proposal: https://mail.python.org/pipermail/wheel-builders/2016-April/000090.html (I really need to get back to that... if anyone wants to help sprint on this at PyCon let me know :-)) -n -- Nathaniel J. Smith -- https://vorpus.org
participants (10)
-
Bernardo Sulzbach
-
Brett Cannon
-
Chris Angelico
-
Chris Barker
-
Glenn Linderman
-
Mark Lawrence
-
Nathaniel Smith
-
Ryan Gonzalez
-
Sebastian Krause
-
Steve Dower