WeakValueDict and threadsafety

Duncan Booth duncan.booth at invalid.invalid
Sat Dec 10 14:09:50 EST 2011


Darren Dale <dsdale24 at gmail.com> wrote:

> On Dec 10, 11:19 am, Duncan Booth <duncan.bo... at invalid.invalid>
> wrote:
>> Darren Dale <dsdal... at gmail.com> wrote:
> def get_data(oid):
>     with reglock:
>         data = registry.get(oid, None)
>         if data is None:
>             data = make_data(oid)
>             registry[oid] = data
>     return data
> 
> Does that look better? I am actually working on the h5py project
> (bindings to hdf5), and the oid is an hdf5 object identifier.
> make_data(oid) creates a proxy object that stores a strong reference
> to oid.

Yes, that looks better.

> 
> Now that I am using this _Registry class instead of
> WeakValueDictionary, my test scripts and my actual program are no
> longer producing segfaults.
> 
I think that so far as multi-thread race conditions are concerned Python 
usually tries to guarantee that you won't get seg faults. So if you were 
getting seg faults my guess would be that either you've found a bug in the 
WeakValueDictionary implementation or you've got a bug in some of your code 
outside Python.

For example if your proxy object has a __del__ method to clean up the 
object it is proxying then you could be creating a new object with the same 
oid as one that is in the process of being destroyed (the object disappears 
from the WeakValueDictionary before the __del__ method is actually called). 

Without knowing anything about HDF5 I don't know if that's a problem but I 
could imagine you could end up creating a new proxy object that references 
something in the HDF5 library which you then destroy as part of cleaning up 
a previous incarnation of the object but continue to access through the new 
proxy.

-- 
Duncan Booth http://kupuguy.blogspot.com



More information about the Python-list mailing list