[Python-Dev] safety of Py_CLEAR and self

[Jeremy Hylton]

I was wondering today how I could convince myself that a sequence of
Py_CLEAR() calls in a tp_clear function was safe.  Take for example a
really trivial sequence of code on frame_clear():

	Py_CLEAR(f->f_exc_type);
	Py_CLEAR(f->f_exc_value);
	Py_CLEAR(f->f_exc_traceback);
	Py_CLEAR(f->f_trace);

We use Py_CLEAR() so that multiple calls to frame_clear() are safe.
The first time this runs it sets exc_type to NULL before calling
DECREF.  This guarantees that a subsequent frame_clear() or
frame_dealloc() won't attempt to DECREF it a second time.  I think I
understand why that's desireable and why it works.  The primary risk
is that via DECREF we may execute an arbitrary amount of Python code
via weakref callbacks, finalizers, and code in other threads that gets
resumed while the callbacks and finalizers are running.

My question, specifically, then: Why it is safe to assume that f
doesn't point to trash after a particular call to Py_CLEAR()?  Any
particular call to Py_CLEAR() could break the cycle that the object is
involved in an lead to a call to frame_dealloc().  The frame could get
returned to an obmalloc pool, returned to the OS, or just re-used by
another object before we get back to Py_CLEAR().  It seems like such
behavior would be exceedingly unlikely, but I can't convince myself
that it is impossible.  Which is it: improbable or impossible?  If it
is only improbable, should we figure out how to write code that is
safe against such an improbable event?

Jeremy