MSToolkit and --debug-on-release for MSVCCompiler
The MS Toolkit compiler recipe I put together for building Python 2.4 extensions currently requires manually diff-ing the source of msvccompiler.py, as does my recipe for building a single extension with debug information so that developers can track down bugs in their code (without needing to rebuild dozens of other modules and core python in debug mode). The changes are pretty trivial in both cases, so I'm looking for feedback on whether I should go with sub-classing as separate compiler modules, or whether the changes should be integrated into the core compiler code-base. The debug-on-release recipe consists entirely of altering the compile_options and ldflags_shared properties of the compiler object. Problem is, the compiler API assumes that debug is a boolean flag, and AFAICT, has no access to the options the user passed in to whatever command is calling the compiler. As a result, it looks like it would be necessary to provide a compiler sub-class, one where specifying debug causes a debug-on-release set of options to be used. That seems silly, as it's just a feature of the MSVC compilers. Still, don't see any other elegant way to approach the problem. That said, this is a comparatively minor feature (which I'm told isn't really a *great* idea, no matter how practical/useful), so no big deal to keep having people do it with diffs. The more important issue, is the MSToolkit compiler. Here the logic of creating a sub-class of msvccompiler seems obvious, but doing so doesn't seem necessary from a technical standpoint... Overriding load_macros and get_msvc_paths is no problem, but the changes to get_msvc_paths should, if I read correctly, actually fix one of the long-standing annoyances (having to run the GUI at least once to get the registry keys, even though the vcvars.bat variables are already specifying everything necessary). This is coded in such a way that it simply replaces the "you should run the GUI" error message with an attempt to retrieve the variables from the environment (i.e. the only case where you run it is where you would have failed with an error before). The changes to load_macros shouldn't interfere with MSVC operation in the least (they are done as a try:except: which is only used when the MSVC operation fails). MSToolkit howto: http://www.vrplumber.com/programming/mstoolkit/ debug-on-release howto: http://groups.google.ca/groups?selm=mailman.26.1065949224.2192.python-list%4... Enjoy yourselves, Mike ________________________________________________ Mike C. Fletcher Designer, VR Plumber, Coder http://www.vrplumber.com http://blog.vrplumber.com
Mike C. Fletcher wrote:
The MS Toolkit compiler recipe I put together for building Python 2.4 extensions currently requires manually diff-ing the source of msvccompiler.py, as does my recipe for building a single extension with debug information so that developers can track down bugs in their code (without needing to rebuild dozens of other modules and core python in debug mode).
The changes are pretty trivial in both cases, so I'm looking for feedback on whether I should go with sub-classing as separate compiler modules, or whether the changes should be integrated into the core compiler code-base.
The debug-on-release recipe consists entirely of altering the compile_options and ldflags_shared properties of the compiler object. Problem is, the compiler API assumes that debug is a boolean flag, and AFAICT, has no access to the options the user passed in to whatever command is calling the compiler. As a result, it looks like it would be necessary to provide a compiler sub-class, one where specifying debug causes a debug-on-release set of options to be used. That seems silly, as it's just a feature of the MSVC compilers. Still, don't see any other elegant way to approach the problem. That said, this is a comparatively minor feature (which I'm told isn't really a *great* idea, no matter how practical/useful), so no big deal to keep having people do it with diffs.
Perhaps I'm missing something, but I don't really see the advantage here. If I call python setup.py build --debug I do get proper MSVC debug builds (provided I have the Python debug libs installed) of distutils based extensions. I'm not even sure whether it's a good idea to mix debug and non-debug builds of libs due to the different ways they do memory allocation (but this might be a red herring).
The more important issue, is the MSToolkit compiler. Here the logic of creating a sub-class of msvccompiler seems obvious, but doing so doesn't seem necessary from a technical standpoint... Overriding load_macros and get_msvc_paths is no problem, but the changes to get_msvc_paths should, if I read correctly, actually fix one of the long-standing annoyances (having to run the GUI at least once to get the registry keys, even though the vcvars.bat variables are already specifying everything necessary). This is coded in such a way that it simply replaces the "you should run the GUI" error message with an attempt to retrieve the variables from the environment (i.e. the only case where you run it is where you would have failed with an error before).
The changes to load_macros shouldn't interfere with MSVC operation in the least (they are done as a try:except: which is only used when the MSVC operation fails).
MSToolkit howto: http://www.vrplumber.com/programming/mstoolkit/
debug-on-release howto: http://groups.google.ca/groups?selm=mailman.26.1065949224.2192.python-list%4...
Enjoy yourselves, Mike
________________________________________________ Mike C. Fletcher Designer, VR Plumber, Coder http://www.vrplumber.com http://blog.vrplumber.com
_______________________________________________ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
-- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Oct 19 2004)
Python/Zope Consulting and Support ... http://www.egenix.com/ mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/
::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::
M.-A. Lemburg wrote: ...
The debug-on-release recipe consists entirely of altering the compile_options and ldflags_shared properties of the compiler object. Problem is, the compiler API assumes that debug is a boolean flag, and AFAICT, has no access to the options the user passed in to whatever command is calling the compiler. As a result, it looks like it would be necessary to provide a compiler sub-class, one where specifying debug causes a debug-on-release set of options to be used. That seems silly, as it's just a feature of the MSVC compilers. Still, don't see any other elegant way to approach the problem. That said, this is a comparatively minor feature (which I'm told isn't really a *great* idea, no matter how practical/useful), so no big deal to keep having people do it with diffs.
Perhaps I'm missing something, but I don't really see the advantage here. If I call
python setup.py build --debug
I do get proper MSVC debug builds (provided I have the Python debug libs installed) of distutils based extensions.
Sure, but if you have a project with, for instance, all of wxPython, PyOpenGL, PyPgSQL, Numpy, mxBase, and a few other sundry extensions and all you want to do is track down a shallow error in your application's tiny little extension, then that "provided" becomes a serious PITA. Many dependencies require all sorts of setup to provide resources before they can be built (jpeg and png libraries, tcl/tk installations, postgresql client libraries, etceteras). Recursively building *everything* debug through the entire dependency tree can take days (and you're pretty much certain at *some* point you're going to be linking to a non-debug library anyway, after all, AFAIK there *is* no debug version of, for instance, OpenGL or GLU on Windows).
I'm not even sure whether it's a good idea to mix debug and non-debug builds of libs due to the different ways they do memory allocation (but this might be a red herring).
Yes, that's what people keep telling me :) . As mentioned, though, I'm neither too worried about getting *this* recipe in, nor particularly concerned with the propriety of it (since it's proved useful and practical on any number of occasions for me and I can keep patching the msvccompiler manually when I need it). I just figured there may be others out there in the same boat who'd like the functionality available w/out the patching fuss. BTW, I've never actually seen a new failure appear after switching to the debug-on-release versions of a project, but that could just be a fluke. Have fun, Mike ________________________________________________ Mike C. Fletcher Designer, VR Plumber, Coder http://www.vrplumber.com http://blog.vrplumber.com
"Mike C. Fletcher"
The debug-on-release recipe consists entirely of altering the compile_options and ldflags_shared properties of the compiler object. Problem is, the compiler API assumes that debug is a boolean flag, and AFAICT, has no access to the options the user passed in to whatever command is calling the compiler. As a result, it looks like it would be necessary to provide a compiler sub-class, one where specifying debug causes a debug-on-release set of options to be used. That seems silly, as it's just a feature of the MSVC compilers. Still, don't see any other elegant way to approach the problem. That said, this is a comparatively minor feature (which I'm told isn't really a *great* idea, no matter how practical/useful), so no big deal to keep having people do it with diffs.
I think you could make a subclass of build_ext, and override the build_extensions() method. You can then replace it's self.compiler.ldflags_shared and self.compiler.compile_options, maybe depending on a new 'fake-debug' command line flag or something like that. Thomas
Mike C. Fletcher wrote:
M.-A. Lemburg wrote: ...
The debug-on-release recipe consists entirely of altering the compile_options and ldflags_shared properties of the compiler object. Problem is, the compiler API assumes that debug is a boolean flag, and AFAICT, has no access to the options the user passed in to whatever command is calling the compiler. As a result, it looks like it would be necessary to provide a compiler sub-class, one where specifying debug causes a debug-on-release set of options to be used. That seems silly, as it's just a feature of the MSVC compilers. Still, don't see any other elegant way to approach the problem. That said, this is a comparatively minor feature (which I'm told isn't really a *great* idea, no matter how practical/useful), so no big deal to keep having people do it with diffs.
Perhaps I'm missing something, but I don't really see the advantage here. If I call
python setup.py build --debug
I do get proper MSVC debug builds (provided I have the Python debug libs installed) of distutils based extensions.
Sure, but if you have a project with, for instance, all of wxPython, PyOpenGL, PyPgSQL, Numpy, mxBase, and a few other sundry extensions and all you want to do is track down a shallow error in your application's tiny little extension, then that "provided" becomes a serious PITA.
Many dependencies require all sorts of setup to provide resources before they can be built (jpeg and png libraries, tcl/tk installations, postgresql client libraries, etceteras). Recursively building *everything* debug through the entire dependency tree can take days (and you're pretty much certain at *some* point you're going to be linking to a non-debug library anyway, after all, AFAIK there *is* no debug version of, for instance, OpenGL or GLU on Windows).
Ok, point taken. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Oct 20 2004)
Python/Zope Consulting and Support ... http://www.egenix.com/ mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/
::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::
participants (3)
-
M.-A. Lemburg -
Mike C. Fletcher -
Thomas Heller