Confusion with weakref, __del__ and threading

[George Sakkis]

On Jun 11, 1:40 am, Rhamphoryncus <rha... at gmail.com> wrote:

> On Jun 10, 8:15 pm, George Sakkis <george.sak... at gmail.com> wrote:
>
>
>
> > I'm baffled with a situation that involves:
> > 1) an instance of some class that defines __del__,
> > 2) a thread which is created, started and referenced by that instance,
> > and
> > 3) a weakref proxy to the instance that is passed to the thread
> > instead of 'self', to prevent a cyclic reference.
>
> > This probably sounds like gibberish so here's a simplified example:
>
> > ==========================================
>
> > import time
> > import weakref
> > import threading
>
> > num_main = num_other = 0
> > main_thread = threading.currentThread()
>
> > class Mystery(object):
>
> >     def __init__(self):
> >         proxy = weakref.proxy(self)
> >         self._thread = threading.Thread(target=target, args=(proxy,))
> >         self._thread.start()
>
> >     def __del__(self):
> >         global num_main, num_other
> >         if threading.currentThread() is main_thread:
> >             num_main += 1
> >         else:
> >             num_other += 1
>
> >     def sleep(self, t):
> >         time.sleep(t)
>
> > def target(proxy):
> >     try: proxy.sleep(0.01)
> >     except weakref.ReferenceError: pass
>
> > if __name__ == '__main__':
> >     for i in xrange(1000):
> >         Mystery()
> >     time.sleep(0.1)
> >     print '%d __del__ from main thread' % num_main
> >     print '%d __del__ from other threads' % num_other
>
> > ==========================================
>
> > When I run it, I get around 950 __del__ from the main thread and the
> > rest from non-main threads. I discovered this accidentally when I
> > noticed some ignored AssertionErrors caused by a __del__ that was
> > doing "self._thread.join()", assuming that the current thread is not
> > self._thread, but as it turns out that's not always the case.
>
> > So what is happening here for these ~50 minority cases ? Is __del__
> > invoked through the proxy ?
>
> The trick here is that calling proxy.sleep(0.01) first gets a strong
> reference to the Mystery instance, then holds that strong reference
> until it returns.

Ah, that was the missing part; I thought that anything accessed
through a proxy didn't create a strong reference. The good thing is
that it seems you can get a proxy to a bounded method and then call it
without creating a strong reference to 'self':

num_main = num_other = 0
main_thread = threading.currentThread()

class MysterySolved(object):

    def __init__(self):
        sleep = weakref.proxy(self.sleep)
        self._thread = threading.Thread(target=target, args=(sleep,))
        self._thread.start()

    def __del__(self):
        global num_main, num_other
        if threading.currentThread() is main_thread:
            num_main += 1
        else:
            num_other += 1

    def sleep(self, t):
        time.sleep(t)


def target(sleep):
    try: sleep(0.01)
    except weakref.ReferenceError: pass


if __name__ == '__main__':
    for i in xrange(1000):
        MysterySolved()
    time.sleep(.1)
    print '%d __del__ from main thread' % num_main
    print '%d __del__ from other threads' % num_other

==========================================
Output:
1000 __del__ from main thread
0 __del__ from other threads


Thanks a lot, I learned something new :)

George