[Python-Dev] bsddb & popitems

Gregory P. Smith greg at electricrain.com
Mon Oct 6 18:01:16 EDT 2003


On Mon, Oct 06, 2003 at 03:59:51PM -0500, Skip Montanaro wrote:
> 
>     Martin> Skip Montanaro <skip at pobox.com> writes:
>     >> I think it would be real nice if we hammered hard on the bsddb3
>     >> problems.  Whatever it is, it seems to affect a broad cross-section
>     >> of the community.
> 
>     Martin> But is there a single report that cannot be attributed to
>     Martin> multi-threading, or multi-processes?
> 
> I don't know.  But the fact that we have so far been unable to answer even
> this question reliably means we have some work to do.  I have been assuming
> that the problems have all been related to access from multiple threads or
> processes, but others haven't seemed so sure.  What about the popitem()
> hangs?

The popitem() stack trace on win98 that was just posted still looks
like a BerkeleyDB issue.  Its stuck in a lock.  (bsddb always opens its
database and environment with DB_INIT_LOCK and DB_THREAD flags because
it can't tell if it will be used by multiple threads)

I agree with your assumption and still believe that it is BerkeleyDB /
OS locking issues causing the hangs on various platforms.


also, not related to bsddb the problem but since i noticed it...:

Looking at the code for popitem it looks like bsddb uses
UserDict.DictMixin's implementation which does not look thread safe if
two threads were removing things from the "dict" with only one of them
using popitem.  Am I missing something?

A race condition exists in iteritems between finding that k exists in
the dictionary and looking up self[k].

    def iteritems(self):
	for k in self:
	    return (k, self[k])


Greg



More information about the Python-Dev mailing list