[Python-Dev] Weak dict iterators are fragile

Antoine Pitrou solipsis at pitrou.net
Sun Oct 11 17:50:54 CEST 2009


In py3k, the weak dict methods keys(), values() and items() have been 
changed to return iterators (they returned lists in 2.x).
However, it turns out that it makes these methods quite fragile, because 
a GC collection can occur whenever during iterating, destroy one of the 
weakref'ed objects, and trigger a resizing of the underlying dict, which 
in turn raises an exception ("RuntimeError: dictionary changed size 
during iteration").

This has just triggered (assuming the diagnosis is correct) a hang in 
test_multiprocessing: http://bugs.python.org/issue7060

I would like to propose some possible solutions against this:

1. Add the safe methods listkeys(), listitems(), listvalues() which would 
behave as the keys(), etc. methods from 2.x

2. Make it so that keys(), items(), values() atomically build a list of 
items internally, which makes them more costly for large weak dicts, but 

What do you think?



More information about the Python-Dev mailing list