[Python-Dev] Software Transactional Memory for Python

[Antoine Pitrou]

On Sat, 27 Aug 2011 15:08:36 +0200
Armin Rigo <arigo at tunes.org> wrote:
> Hi Nick,
> 
> On Sat, Aug 27, 2011 at 2:40 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> > 1. How does the patch interact with C code that explicitly releases
> > the GIL? (e.g. IO commands inside a "with atomic:" block)
> 
> As implemented, any code in a "with atomic" is prevented from
> explicitly releasing and reacquiring the GIL: the GIL remain acquired
> until the end of the "with" block.  In other words
> Py_BEGIN_ALLOW_THREADS has no effect in a "with" block.

You then risk deadlocks. Say:
- thread A is inside a "with atomic" and calls a library function which
  tries to take lock L
- thread B has already taken lock L and is currently executing an I/O
  function with GIL released
- thread B then waits for the GIL (and hence depends on thread A going
  forward), while thread A waits for lock L (and hence depends on
  thread B going forward)

Lock L could simply be the lock used by the file object  (a
Buffered{Reader,Writer,Random}) which thread B is reading or writing
from.

Regards

Antoine.