[Python-Dev] Re: test_bsddb blocks testing popitem - reason
Alex Martelli
aleaxit at yahoo.com
Mon Oct 27 05:25:16 EST 2003
On Monday 27 October 2003 10:40 am, Gregory P. Smith wrote:
...
> The big difference i see between 2.3cvs and 2.4cvs that could "explain"
> it is that Lib/bsddb/__init__.py has been updated to use a private
> (in memory, single process only) DBEnv with locking and thread support
> enabled. That explains why db->del() would be doing locking. But not
> why it would deadlock.
*AH*! I wasn't looking in the right place, silly me. Good job!!! Yes,
now that you've pointed it out, the change from 2.3's
d = db.DB()
to 2.4's
e = _openDBEnv()
d = db.DB(e)
must be the culprit.
I still don't quite see how the lock ends up being "held", but, don't mind
me -- the intricacy of mixins and wrappings and generators and delegations
in those modules is making my head spin anyway, so it's definitely not
surprising that I can't quite see what's going on.
> How do python dictionaries deal with modifications to the dictionary
> intermixed with iteration?
In general, Python doesn't deal well with modifications to any
iterable in the course of a loop using an iterator on that iterable.
The one kind of "modification during the loop" that does work is:
for k in somedict:
somedict[k] = ...whatever...
i.e. one can change the values corresponding to keys, but not
change the set of keys in any way -- any changes to the set of
keys can cause unending loops or other such misbehavior (not
deadlocks nor crashes, though...).
However, on a real Python dict,
k, v = thedict.iteritems().next()
doesn't constitute "a loop" -- the iterator object returned by
the iteritems call is dropped since there are no outstanding
references to it right after this statement. So, following up
with
del thedict[k]
is quite all right -- the dictionary isn't being "looped on" at
that time.
Given that in bsddb's case that iteritems() first [and only]
next() boils down to a self.first() which in turn does a
self.dbc.first() I _still_ don't see exactly what's holding the
lock. But the simplest fix would appear to be in __delitem__,
i.e., if we have a cursor we should delete through it:
def __delitem__(self, key):
self._checkOpen()
if self.dbc is not None:
self.dbc.set(key)
self.dbc.delete()
else:
del self.db[key]
...but this doesn't in fact remove the deadlock on the
unit-test for popitem, which just confirms I don't really
grasp what's going on, yet!-)
Alex
More information about the Python-Dev
mailing list