[issue11393] Integrate faulthandler module into Python 3.3

[STINNER Victor]

New submission from STINNER Victor <victor.stinner at haypocalc.com>:

<< Fault handler for SIGSEGV, SIGFPE, SIGBUS and SIGILL signals: display the
Python backtrace and restore the previous handler. Allocate an alternate stack
for this handler, if sigaltstack() is available, to be able to allocate memory
on the stack, even on stack overflow (not available on Windows).

Import the module and call faulthandler.enable() to enable the fault handler.

The fault handler is called on catastrophic cases and so it can only use
signal-safe functions (eg. it doesn't allocate memory on the heap). That's why
the backtrace is limited: it only supports ASCII encoding (use the
backslashreplace error handler for non-ASCII characters), doesn't print
the source code in the backtrace (only the filename, the function name and the
line number), is limited to 100 frames and 100 threads.

By default, the Python backtrace is written to the standard error stream. Start
your graphical applications in a terminal and run your server in foreground to
see the backtrace, or pass a file to faulthandler.enable().

faulthandler is implemented in C using signal handlers to be able to dump a
backtrace on a crash or when Python is blocked (eg. deadlock). >>

https://github.com/haypo/faulthandler

faulthandler works on Python 2.5 .. 3.3.

--

faulthandler is an alternative to gdb7 + python-gdb.py to dump the backtrace. faulthandler already helped me to debug a PyQt crash on Windows: I don't know how to install gdb on Windows, especially with faulthandler!

But I didn't get a lot of feedbacks.

--

I don't know if the whole module should be included or not.

TODO list:

 * add something to enable faulthandler on the command line: an option on the command line (eg. python -x faulthandler script.py) or an environment variable (PYTHONFAULTHANDLER=y python script.py)?
 * decide what to do with child processes: disable the faulthandler after a fork?
 * use something else than Py_AtExit() to unload the module?
 * dump the Python backtrace on a fatal error (and/or maybe catch SIGABRT)
 * use maybe something else than alarm()+SIGARLM for dumpbacktrace_later() because it interrupts the current system call

--

History of this module.

I first proposed a segfault handler using setjmp/longjmp to raise a classic Python exception. So it was  possible to execute Python code after a segfault (including stack overflow). But the idea was rejected because the Python internal state may be corrupted if the segfault was an invalid memory write (buffer overflow?), and anyway we cannot warranty that the internal state is still consistent after a long jump.

=> http://bugs.python.org/issue3999 (sept. 2009)

After that, I proposed to just display the Python backtrace on segfault. The signal handler was enabled by default. Because some people was concerned about possible bugs or security vulnerabilities introduced by the signal handler, I proposed options to disable it. Finally, I proposed to disable it by default and add options to enable it, but it was too late for an inclusion into Python 3.2. Anyways, the project (a patch) was young and the API not correctly defined.

=> http://bugs.python.org/issue8863 (may 2010)

I converted my patch into a module. During this conversion, I realized that a module API is more natural than options using environment variables or functions in the sys module (I proposed sys.setfaulthandlerenabled(), an horrible name :-)). And it becomes more natural to enable the faulthandler by importing a module and calling a function (faulthandler.enable()).

Later I added functions to display the Python backtrace not only on a crash, but after a certain amount of time, or when an user signal is raised. You can also call the function directly to dump the backtrace. And finally, I added many option to configure how the backtrace is displayed: current thread of all threads, sys.stderr or another file, ...

--

I hope that this module will help to understand multiprocessing issues on the buildbots!

----------
components: Library (Lib)
messages: 130013
nosy: haypo
priority: normal
severity: normal
status: open
title: Integrate faulthandler module into Python 3.3
versions: Python 3.3

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue11393>
_______________________________________