Re: [Python-ideas] Allow to compile debug extension against releasePython in Windows
On 08.01.2018 0:11, Steve Dower wrote:
Distutils' designers seem to have thought differently. Whether the extension is linked against pythonxx_d.lib is governed by the --debug switch to the `build' command rather than the type of the running Python. Compiler optimization flags are inserted according to it, too. As a consequence, * I cannot install an extension into debug Python at all 'cuz `bdist_*' and `install' commands don't support --debug and invoke `debug' internally without it. * Neither can I compile an extension for release Python without optimizations. I'm at a loss here. Either I'm missing something, or with the current build system, it's impossible to debug extensions!
-- Regards, Ivan
I not a user of distutils or setuptools but some googling seems to say that the build command has a --debug to do what you want. If that does not work it would seem like you could ask the setuptools maintainers how to do the reason thing of a debug build. Barry
On 09.01.2018 23:31, Barry Scott wrote:
I just wrote, in https://mail.python.org/pipermail/python-ideas/2018-January/048579.html , that --debug is not sufficient, and that the problematic logic is in distutils, not setuptools. -- Regards, Ivan
Sorry, I mis-read that. I thought it was not a known option. It is certainly hard to find docs for. This does sound like something the setup tools people can answer for you. I think they hang out on the python distutils-sig mailing list. Barry
You don’t have to use distutils to build your extension, and if you want anything more complex than a release build you probably should build it yourself. All you need is the right include and lib directories when compiling/linking, set the output extension to pyd instead of dll, and use /MD when targeting python.exe and /MDd when targeting python_d.exe. You can use whatever /Ox options you like regardless of your target. There’s nothing really special about what the build tools do, other than being readily available on most Python distros. Since you're on Windows, why not have a look at Visual Studio 2017? The Python workload has a native development option that includes a template for building/debugging an extension module with these options already configured (https://docs.microsoft.com/en-us/visualstudio/python/cpp-and-python), and the debugger supports stepping through both Python and C code simultaneously (https://docs.microsoft.com/en-us/visualstudio/python/debugging-mixed-mode). It might make your life a little easier. Top-posted from my Windows phone From: Ivan Pozdeev Sent: Wednesday, January 10, 2018 5:47 To: Steve Dower; python-ideas@python.org Subject: Re: [Python-ideas] Allow to compile debug extension againstreleasePython in Windows On 09.01.2018 21:35, Ivan Pozdeev via Python-ideas wrote: On 08.01.2018 0:11, Steve Dower wrote: It’s not a good idea. You end up with two different C runtimes in memory that cannot communicate, and many things will not work properly. If you compile your debug build extension with the non-debug CRT (/MD rather than /MDd) you will lose asserts, but otherwise it will work fine and the quoted code picks the release lib. Distutils' designers seem to have thought differently. Whether the extension is linked against pythonxx_d.lib is governed by the --debug switch to the `build' command rather than the type of the running Python. Compiler optimization flags and /MD(d) are inserted according to it, too. As a consequence, * I cannot install an extension into debug Python at all 'cuz `bdist_*' and `install' commands don't support --debug and invoke `debug' internally without it. I meant "invoke `build' internally without it." , sorry. This kafkaesque "you cannot do this because you cannot do this" is taking its toll on me... * Neither can I compile an extension for release Python without optimizations. I'm at a loss here. Either I'm missing something, or with the current build system, it's impossible to debug extensions! Or if you like, when you install Python 3.5 or later there are advanced options to install debug symbols and binaries. You can use a proper debug build against the debug binaries (python_d.exe). Cheers, Steve Top-posted from my Windows phone From: Ivan Pozdeev via Python-ideas Sent: Saturday, December 30, 2017 13:01 To: python-ideas@python.org Subject: [Python-ideas] Allow to compile debug extension against releasePython in Windows The Windows version of pyconfig.h has the following construct: if defined(_DEBUG) pragma comment(lib,"python37_d.lib") elif defined(Py_LIMITED_API) pragma comment(lib,"python3.lib") else pragma comment(lib,"python37.lib") endif /* _DEBUG */ which fails the compilation of a debug version of an extension. Making debugging it... difficult. Perhaps we could define some other constant? I'm not sure whether such compilation is a good idea in general, so asking here at first. -- Regards, Ivan _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/ -- Regards, Ivan _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/ -- Regards, Ivan
I not a user of distutils or setuptools but some googling seems to say that the build command has a --debug to do what you want. If that does not work it would seem like you could ask the setuptools maintainers how to do the reason thing of a debug build. Barry
On 09.01.2018 23:31, Barry Scott wrote:
I just wrote, in https://mail.python.org/pipermail/python-ideas/2018-January/048579.html , that --debug is not sufficient, and that the problematic logic is in distutils, not setuptools. -- Regards, Ivan
Sorry, I mis-read that. I thought it was not a known option. It is certainly hard to find docs for. This does sound like something the setup tools people can answer for you. I think they hang out on the python distutils-sig mailing list. Barry
You don’t have to use distutils to build your extension, and if you want anything more complex than a release build you probably should build it yourself. All you need is the right include and lib directories when compiling/linking, set the output extension to pyd instead of dll, and use /MD when targeting python.exe and /MDd when targeting python_d.exe. You can use whatever /Ox options you like regardless of your target. There’s nothing really special about what the build tools do, other than being readily available on most Python distros. Since you're on Windows, why not have a look at Visual Studio 2017? The Python workload has a native development option that includes a template for building/debugging an extension module with these options already configured (https://docs.microsoft.com/en-us/visualstudio/python/cpp-and-python), and the debugger supports stepping through both Python and C code simultaneously (https://docs.microsoft.com/en-us/visualstudio/python/debugging-mixed-mode). It might make your life a little easier. Top-posted from my Windows phone From: Ivan Pozdeev Sent: Wednesday, January 10, 2018 5:47 To: Steve Dower; python-ideas@python.org Subject: Re: [Python-ideas] Allow to compile debug extension againstreleasePython in Windows On 09.01.2018 21:35, Ivan Pozdeev via Python-ideas wrote: On 08.01.2018 0:11, Steve Dower wrote: It’s not a good idea. You end up with two different C runtimes in memory that cannot communicate, and many things will not work properly. If you compile your debug build extension with the non-debug CRT (/MD rather than /MDd) you will lose asserts, but otherwise it will work fine and the quoted code picks the release lib. Distutils' designers seem to have thought differently. Whether the extension is linked against pythonxx_d.lib is governed by the --debug switch to the `build' command rather than the type of the running Python. Compiler optimization flags and /MD(d) are inserted according to it, too. As a consequence, * I cannot install an extension into debug Python at all 'cuz `bdist_*' and `install' commands don't support --debug and invoke `debug' internally without it. I meant "invoke `build' internally without it." , sorry. This kafkaesque "you cannot do this because you cannot do this" is taking its toll on me... * Neither can I compile an extension for release Python without optimizations. I'm at a loss here. Either I'm missing something, or with the current build system, it's impossible to debug extensions! Or if you like, when you install Python 3.5 or later there are advanced options to install debug symbols and binaries. You can use a proper debug build against the debug binaries (python_d.exe). Cheers, Steve Top-posted from my Windows phone From: Ivan Pozdeev via Python-ideas Sent: Saturday, December 30, 2017 13:01 To: python-ideas@python.org Subject: [Python-ideas] Allow to compile debug extension against releasePython in Windows The Windows version of pyconfig.h has the following construct: if defined(_DEBUG) pragma comment(lib,"python37_d.lib") elif defined(Py_LIMITED_API) pragma comment(lib,"python3.lib") else pragma comment(lib,"python37.lib") endif /* _DEBUG */ which fails the compilation of a debug version of an extension. Making debugging it... difficult. Perhaps we could define some other constant? I'm not sure whether such compilation is a good idea in general, so asking here at first. -- Regards, Ivan _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/ -- Regards, Ivan _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/ -- Regards, Ivan
participants (3)
-
Barry Scott -
Ivan Pozdeev -
Steve Dower