[ python-Bugs-840829 ] weakref callbacks and gc corrupt memory

[SourceForge.net]

Bugs item #840829, was opened at 2003-11-12 18:05
Message generated for change (Comment added) made by theller
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=840829&group_id=5470

Category: Python Interpreter Core
Group: Python 2.3
Status: Open
Resolution: None
Priority: 9
Submitted By: Thomas Heller (theller)
>Assigned to: Tim Peters (tim_one)
Summary: weakref callbacks and gc corrupt memory

Initial Comment:
This program crashes the Python interpreter in a debug
build, and probably corrupts memory in release build
(on Windows, at least):

"""
import weakref
import gc

_boundMethods = weakref.WeakKeyDictionary()

def safeRef(object):
    selfkey = object.im_self
    funckey = object.im_func
    _boundMethods[selfkey] = weakref.WeakKeyDictionary()
    _boundMethods[selfkey][funckey] =
BoundMethodWeakref(object)

class BoundMethodWeakref:
    def __init__(self, boundMethod):
        def remove(object):
            gc.collect()
        self.weakSelf =
weakref.ref(boundMethod.im_self, remove)

class X(object):
    def test(self):
        pass

def test():
    print "A"
    safeRef(X().test)
    print "B"

if __name__ == "__main__":
    test()
"""

See also these messages:

http://mail.python.org/pipermail/python-dev/2003-November/040189.html

http://mail.python.org/pipermail/python-dev/2003-November/040191.html


----------------------------------------------------------------------

>Comment By: Thomas Heller (theller)
Date: 2003-11-13 08:49

Message:
Logged In: YES 
user_id=11105

Thanks for the fast response, Tim, but this doesn't seem to
be the (complete) solution.

The debug build crashes far less than before, but it still
occurs, and in the same way.  I attach a complete MSVC6
stack dump.  The function names you do not know come from
the ctypes extension.

Back to you again :-(

Oh, what helped for me was to disable gc in the weakref
callback functions.  Don't know if that is an option for the
C code...

----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2003-11-12 21:47

Message:
Logged In: YES 
user_id=31435

Back to you, Thomas!  Please check your real app against 
current CVS Python.  I checked in a putative fix as

Lib/test/test_weakref.py; new revision: 1.29
Misc/NEWS; new revision: 1.890
Objects/typeobject.c; new revision: 2.251

The new test case in test_weakref.py is very much simpler 
than what you've been running, so I want confirmation that 
the real problem is fixed too before closing this.

Please record what happens here, then assign back to me.

----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2003-11-12 20:54

Message:
Logged In: YES 
user_id=31435

I agree this is a critical bug; assigned to me.

----------------------------------------------------------------------

Comment By: Thomas Heller (theller)
Date: 2003-11-12 18:08

Message:
Logged In: YES 
user_id=11105

Attached the script triggering the error.

----------------------------------------------------------------------

Comment By: Thomas Heller (theller)
Date: 2003-11-12 18:07

Message:
Logged In: YES 
user_id=11105

IMO this is a serious problem because it renders weakref
callbacks basically unusable - gc can occur at any time.
So I raised the priority to 9.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=840829&group_id=5470