[New-bugs-announce] [issue7205] bz2file deadlock

Robert Collins report at bugs.python.org
Sun Oct 25 22:55:54 CET 2009

New submission from Robert Collins <robertc at robertcollins.net>:

There is a systemic bug in BZ2File where the GIL is released to perform
compression work, and any other thread calling into BZ2File will
deadlock. We noticed in the write method, but inspection of the code
makes it clear that its systemic.

The problem is pretty simple. Say you have two threads and one bz2file
object. One calls write(), the other calls (say) seek(), but it could be
write() or other methods too. Now, its pretty clear that the question
'should these two threads get serialised' could be contentious. So lets
put that aside by saying 'raising an exception or serialising in
arbitrary order would be ok'.

What happens today is:
   bz2compression starts
   bz2file.lock.acquire(wait=1)  <- this thread is stuck now, and has
the GIL
t1:bz2compression finishes
   gil.acquire <- this thread is stuck now, waiting for the GIL

If any owner of the bz2file object lock will release the GIL, *all*
routines that attempt to lock the bz2file object have to release the GIL
if they can't get the lock - blocking won't work. I'm not familiar
enough with the python threading API to know whether its safe to call
without the GIL. If its not then clearly it can't be used to work with
getting the GIL, and lower layer locks should be used.

components: Extension Modules
files: bz2fail.py
messages: 94462
nosy: barry, rbcollins, statik
severity: normal
status: open
title: bz2file deadlock
type: crash
Added file: http://bugs.python.org/file15196/bz2fail.py

Python tracker <report at bugs.python.org>

More information about the New-bugs-announce mailing list