Add a developer mode to Python: -X dev command line option
Hi, When I develop on CPython, I'm always building Python in debug mode using ./configure --with-pydebug. This mode enables a *lot* of extra checks which helps me to detect bugs earlier. The debug mode makes Python much slower and so is not the default. "python3" in Linux distributions is a binary compiled in release mode. When they also provide a binary compiled in debug mode, you will probably have issues to use your existing C extensions, since they all of them are compiled in release mode which is not compatible (you must recompile C extensions in debug mode). I propose to add a "development mode" to Python, to get a few checks to detect bugs earlier: a new -X dev command line option. Example: python3.6 -X dev script.py Checks enabled by this flag must: * Not flood the stdout/stderr (ex: don't write one message per second) * Have a low overhead in term of CPU and memory (ex: max 10%) I propose to enable: * Show DeprecationWarning and ResourceWarning warnings: python -Wd * Show BytesWarning warning: python -b * Enable Python assertions (assert) and set __debug__ to True: remove (or just ignore) -O or -OO command line arguments * faulthandler to get a Python traceback on segfault and fatal errors: python -X faulthandler * Debug hooks on Python memory allocators: PYTHONMALLOC=debug For example, I consider that enabling tracemalloc is too expensive (CPU & memory) and must not be enabled in -X dev. I wrote a proof-of-concept: if -X dev is used, executable Python once again with more parameters. Basically, replace: python3.6 -X dev ... with PYTHONMALLOC=debug python3.6 -Wd -b -X faulthandler ... The list of checks can be extended later. For example, we may enable the debug mode of asyncio: PYTHONASYNCIODEBUG=1. The scope of the "developer mode" is unclear to me. Many modules (ex: asyncio) already have a debug mode. Would it be a bad idea to enable *all* debug modes of *all* modules? For example, http.client has a debug level: do you expect debug traces of the HTTP client when you develop your application? IMHO the scope must be well defined: don't modify modules of the stdlib, only enable more debug flags in the Python core (warnings, unicode, memory allocators, etc.). Maybe we even a need -X dev=strict which would be stricter: * use -Werror instead of -Wd: raise an exception when a warning is emitted * use -bb instead of -b: get BytesWarning exceptions * Replace "inconsistent use of tabs and spaces in indentation" warning with an error in the Python parser * etc. Again, this list can be extended later. Or maybe we need multiple level to control the quantity of debug traces, warnings, ... written into stdout/stderr? In my experience, the problem with converting warnings to errors is that you don't control the code of your whole application. It's common that third party modules raise DeprecationWarning, ResourceWarning, etc. Even some modules of the Python stdlib raise DeprecatingWarning! For example, I recall that distutils raises a DeprecationWarning on Python 3.4 when importing the imp module. "Similar" idea in other programming languages: * Perl: http://perldoc.perl.org/strict.html * PHP: https://secure.php.net/manual/fr/function.error-reporting.php Victor
2016-03-30 9:49 GMT+02:00 Victor Stinner <victor.stinner@gmail.com>:
I posted my patch on the bug tracker if you would like to try it: https://bugs.python.org/issue26670 Victor
On 30.03.2016 09:49, Victor Stinner wrote:
I'm not sure whether this would make things easier for the majority of developers, e.g. someone not writing C extensions would likely not be interested in debugging memory allocations or segfaults, someone spending more time on numerics wouldn't bother with bytes warnings, etc. I usually use wrappers or custom Python builds for debugging purposes, which are then specialized for the specific task I need them for. Those are quick to write and more explicit in what they enable or not. I wouldn't want to rely on a specific set of flags which may change from Python release to Python release. So overall, I guess adding a debug howto of some sort showing and explaining the purpose of the various options would be more helpful to developers than a meta-option which triggers some subset of other available options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Experts
::::: Try our mxODBC.Connect Python Database Interface for free ! :::::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/
2016-03-30 9:49 GMT+02:00 Victor Stinner <victor.stinner@gmail.com>:
I posted my patch on the bug tracker if you would like to try it: https://bugs.python.org/issue26670 Victor
On 30.03.2016 09:49, Victor Stinner wrote:
I'm not sure whether this would make things easier for the majority of developers, e.g. someone not writing C extensions would likely not be interested in debugging memory allocations or segfaults, someone spending more time on numerics wouldn't bother with bytes warnings, etc. I usually use wrappers or custom Python builds for debugging purposes, which are then specialized for the specific task I need them for. Those are quick to write and more explicit in what they enable or not. I wouldn't want to rely on a specific set of flags which may change from Python release to Python release. So overall, I guess adding a debug howto of some sort showing and explaining the purpose of the various options would be more helpful to developers than a meta-option which triggers some subset of other available options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Experts
::::: Try our mxODBC.Connect Python Database Interface for free ! :::::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/
participants (3)
-
Ethan Furman -
M.-A. Lemburg -
Victor Stinner