[Python-Dev] Py_CLEAR to avoid crashes

[Brett Cannon]

On Feb 16, 2008 3:12 PM, Amaury Forgeot d'Arc <amauryfa at gmail.com> wrote:
> Hello,
>
> I think I found a prolific vein of potential crashes throughout the
> python interpreter.
> The idea is inspired from the discussion Issue1020188 and is quite
> simple: find a Py_DECREF(xxx->yyy), where xxx represents any kind of
> Python object, and craft a __del__ method so that the same function is
> recursively called with the same object.
>
> I recently submitted corrections for 3 problems found this way. Here
> are two more examples of this method:
>
> #====================================
> class Special:
>     def __del__(self):
>         print a.args
>
> class MyException(BaseException):
>     def __init__(self):
>         global a
>         a = self
>         BaseException.__init__(self, Special(), 0)
>         BaseException.__init__(self, "other", 1)
>
> MyException() # segfault
> #====================================
> import sys
> class Special2(str):
>     def __del__(self):
>         f.__init__("@temp", "w")
>
> f = file(Special2("@temp"), "w")
> f.__init__("@temp", "w")  # segfault
> #====================================
>
> Issue1020188 (which is still open btw) deals with member reset to
> NULL; but any modification of the pointer is potentially concerned.
>
> And the correction is always the same: use Py_CLEAR, or a similar
> construction that detach the value from the structure before
> defcref'ing it.
>
> I think it would help a lot of code if we had a safe standard way to
> set struct members from C. A macro like the following:
>     Py_SETREF(lvalue, newobj)
> (whatever its name) would perform the assignment and expand to code
> equivalent to:
>     PyObject* oldobj = lvalue;
>     Py_INCREF(newobj);
>     lvalue = newobj;
>     Py_DECREF(oldobj);
>
> I am currently searching for all places that could benefit of this,
> but I am not sure to find a test case for every potential crash.
> Most of the time, it is very unlikely that "normal" python code can
> fall in these traps (who calls f.__init__ in a __del__ method?), with
> the exception of the one corrected by r60871.
>

Testing C code like this can be tough. Usually the crasher is used as
the test in hopes that any fix is general enough that if it fails that
same crasher will rear its ugly head again.

> Should we however intensively search and correct all of them?

I think it's fine to.

> Is there a clever way to prevent these problems globally, for example
> by delaying finalizers "just a little"?

Beats me.

-Brett