Is it safe to call PyEval_EvalFrameEx() with an exception set?
Hi, I'm working on the issue #18408 (fix issues found by my pyfailmalloc tool). To analyze some bugs, I have to add debug traces in various functions to find which function returned NULL without setting an error, or the opposite: returned a valid object, but with an exception set. I would like to add assertions in Python/ceval.c to detect such bugs earlier. The problem is that some functions rely on the ability to call PyEval_EvalFrameEx() with an exception set. Is it expected? Should it be avoided? The current exception can be replaced with a new exception. Example 1: module = PyImport_Import(module_name); if (module == NULL) { PyErr_Format(PicklingError, "Can't pickle %R: import of module %R failed", obj, module_name); goto error; } Formatting the obj argument calls indirectly PyEval_EvalFrameEx(), whereas an ImportError was raised. The original ImportError is replaced with a new PickleError exception. To not call PyEval_EvalFrameEx() with an exception set, PyErr_Format() can call PyErr_Clear() before calling PyUnicode_FromFormatV(), maybe only in debug mode. Example 2: test_sqlite uses sqlite.connection.create_aggregate() to create a Python callback called by the C sqlite3 library. The "callback" is a class with 3 methods: constructor, step() and finalize(). The test AggregateTests.CheckAggrNoStep() uses a class with no step() method: it ensures that an AttributeError is raised. The problem is that the finalize() method is called even if the class has no step method, and so there is a current AttributeError exception. I don't know if finalize() should be called in this case. It may be a bug in the sqlite module. More generally, Victor
Le Tue, 16 Jul 2013 02:34:49 +0200, Victor Stinner <victor.stinner@gmail.com> a écrit :
Well, if your frame is a generator frame, you must be able to throw() into it, i.e. resume it with an exception set. The difficulty here is that there are different meanings for "an exception is set": - PyErr_Occurred() is true - the f_exc_type field and friends are set - possibly another one that I'm forgetting about Regards Antoine.
2013/7/16 Antoine Pitrou <solipsis@pitrou.net>:
Ah yes, a generator can call PyEval_EvalFrameEx() with an exception set (PyErr_Occurred() is non-NULL), but it sets throwflag to 1. My question is when an exception is set (PyErr_Occurred() is non-NULL) at the beginning of the bytecode evaluation loop. Just after the "goto error;" in this extract of ceval.c: if (throwflag) /* support for generator.throw() */ goto error; for (;;) { ... } Victor
2013/7/16 Victor Stinner <victor.stinner@gmail.com>:
Another example. Py_ReprLeave() function can be called while an exception is set. I'm not sure that it is "safe" to call new code while an exception is set. The function can raise a new exception. For example, PyList_SetSlice() can fail because of a MemoryError. I modified Py_ReprLeave() to save and then restore the current exception: http://hg.python.org/cpython/rev/28ff7ac91477 If Py_ReprLeave() raises a new exception, it is clears (replaced by the previous exception context). Victor
Le Tue, 16 Jul 2013 02:34:49 +0200, Victor Stinner <victor.stinner@gmail.com> a écrit :
Well, if your frame is a generator frame, you must be able to throw() into it, i.e. resume it with an exception set. The difficulty here is that there are different meanings for "an exception is set": - PyErr_Occurred() is true - the f_exc_type field and friends are set - possibly another one that I'm forgetting about Regards Antoine.
2013/7/16 Antoine Pitrou <solipsis@pitrou.net>:
Ah yes, a generator can call PyEval_EvalFrameEx() with an exception set (PyErr_Occurred() is non-NULL), but it sets throwflag to 1. My question is when an exception is set (PyErr_Occurred() is non-NULL) at the beginning of the bytecode evaluation loop. Just after the "goto error;" in this extract of ceval.c: if (throwflag) /* support for generator.throw() */ goto error; for (;;) { ... } Victor
2013/7/16 Victor Stinner <victor.stinner@gmail.com>:
Another example. Py_ReprLeave() function can be called while an exception is set. I'm not sure that it is "safe" to call new code while an exception is set. The function can raise a new exception. For example, PyList_SetSlice() can fail because of a MemoryError. I modified Py_ReprLeave() to save and then restore the current exception: http://hg.python.org/cpython/rev/28ff7ac91477 If Py_ReprLeave() raises a new exception, it is clears (replaced by the previous exception context). Victor
participants (2)
-
Antoine Pitrou -
Victor Stinner