[Python-Dev] Py_CLEAR to avoid crashes

[Amaury Forgeot d'Arc]

Hello,

I think I found a prolific vein of potential crashes throughout the
python interpreter.
The idea is inspired from the discussion Issue1020188 and is quite
simple: find a Py_DECREF(xxx->yyy), where xxx represents any kind of
Python object, and craft a __del__ method so that the same function is
recursively called with the same object.

I recently submitted corrections for 3 problems found this way. Here
are two more examples of this method:

#====================================
class Special:
    def __del__(self):
        print a.args

class MyException(BaseException):
    def __init__(self):
        global a
        a = self
        BaseException.__init__(self, Special(), 0)
        BaseException.__init__(self, "other", 1)

MyException() # segfault
#====================================
import sys
class Special2(str):
    def __del__(self):
        f.__init__("@temp", "w")

f = file(Special2("@temp"), "w")
f.__init__("@temp", "w")  # segfault
#====================================

Issue1020188 (which is still open btw) deals with member reset to
NULL; but any modification of the pointer is potentially concerned.

And the correction is always the same: use Py_CLEAR, or a similar
construction that detach the value from the structure before
defcref'ing it.

I think it would help a lot of code if we had a safe standard way to
set struct members from C. A macro like the following:
    Py_SETREF(lvalue, newobj)
(whatever its name) would perform the assignment and expand to code
equivalent to:
    PyObject* oldobj = lvalue;
    Py_INCREF(newobj);
    lvalue = newobj;
    Py_DECREF(oldobj);

I am currently searching for all places that could benefit of this,
but I am not sure to find a test case for every potential crash.
Most of the time, it is very unlikely that "normal" python code can
fall in these traps (who calls f.__init__ in a __del__ method?), with
the exception of the one corrected by r60871.

Should we however intensively search and correct all of them?
Is there a clever way to prevent these problems globally, for example
by delaying finalizers "just a little"?

-- 
Amaury Forgeot d'Arc