[Python-Dev] More fun with Python shutdown

Thomas Heller theller at python.net
Tue Nov 11 16:05:31 EST 2003


Thomas Heller <theller at python.net> writes:

> "Tim Peters" <tim.one at comcast.net> writes:
>
>> Jim (Fulton) refactored oodles of Zope3 to make heavier use of weak
>> references.  Now Zope3 dies with a segfault when it's shut down, which makes
>> its adoption of Python 2.3.2 a bit less attractive <wink>.
>>
>> The problem isn't really understood.  I hope that once it is, there will be
>> a simple way to avoid it under 2.3.2.  Jim filed a bug report with a fix to
>> the symptom here:
>>
>>     http://www.python.org/sf/839548
>
> Is the problem I currently have the same, I also use weakrefs (although
> Jim's patch doesn't seem to help)?
>
> It is triggered when I have set the gc threshold to small values in a
> 2.3.2 debug build under Windows.  When some containers in my program are
> destroyed Python crashes with an access violation in
> _Py_ForgetReference() because op->_ob_next and
> _op->_ob_prev are both NULL:
>
>   void
>   _Py_ForgetReference(register PyObject *op)
>   {
>   #ifdef SLOW_UNREF_CHECK
>           register PyObject *p;
>   #endif
>         if (op->ob_refcnt < 0)
>                 Py_FatalError("UNREF negative refcnt");
>         if (op == &refchain ||
>             op->_ob_prev->_ob_next != op || op->_ob_next->_ob_prev != op)
>                 Py_FatalError("UNREF invalid object");

Here is the smallest program I can currently come up with that triggers
this bug.  Most of the code is extracted from Patrick O'Brian's
dispatcher module on activestate's cookbook site, it creates weak
references to bound methods by dissecting them into im_self and im_func.

This program only prints "A" before crashing, so it does occur *before*
interpreter shutdown.

Thomas

-----
import weakref
import gc

gc.set_threshold(1)

connections = {}
_boundMethods = weakref.WeakKeyDictionary()

def safeRef(object):
    selfkey = object.im_self
    funckey = object.im_func
    if not _boundMethods.has_key(selfkey):
        _boundMethods[selfkey] = weakref.WeakKeyDictionary()
    if not _boundMethods[selfkey].has_key(funckey):
        _boundMethods[selfkey][funckey] = \
        BoundMethodWeakref(boundMethod=object)
    return _boundMethods[selfkey][funckey]

class BoundMethodWeakref:
    def __init__(self, boundMethod):
        def remove(object, self=self):
            _removeReceiver(receiver=self)
        self.weakSelf = weakref.ref(boundMethod.im_self, remove)
        self.weakFunc = weakref.ref(boundMethod.im_func, remove)

def _removeReceiver(receiver):
    for senderkey in connections.keys():
        for signal in connections[senderkey].keys():
            receivers = connections[senderkey][signal]
            try: receivers.remove(receiver)
            except: pass
            _cleanupConnections(senderkey, signal)

################

class X(object):
    def test(self):
        pass

def test():
    print "A"
    safeRef(X().test)
    print "B"

if __name__ == "__main__":
    test()
-----




More information about the Python-Dev mailing list