[Python-Dev] PyFAQ: thread-safe interpreter operations

[Armin Rigo]

Hi Martin,

On Mon, Nov 20, 2006 at 11:55:42PM +0100, "Martin v. L?wis" wrote:
> In general, += isn't atomic: it may invoke __add__ or __iadd__ on the
> left-hand side, or __radd__ on the right-hand side.

> If you only look at the actual operation, the these aren't atomic:
> 
> x.field = y # may invoke __setattr__, may also be a property
> D[x] = y    # may invoke x.__hash__, and x.__eq__

I think this list of examples isn't meant to be read that way.  Half of
them can invoke custom methods, not just the two you mention here.  I
think the idea is that provided only "built-in enough" objects are
involved, the core operation described by each line works atomically, in
the sense e.g. that if two threads do 'L.append(x)' you really add two
items to the list (only the order is unspecified), and if two threads
perform  x.field = y  roughly at the same time, and the type of  x
doesn't override the default __setattr__ logic, then you know that the
object  x  will end up with a 'field' that is present and has exactly
one of the two values that the threads tried to put in.

Python programs rely on these kind of properties, and they are probably
a good thing - at least, much better IMHO than having to put locks
everywhere.  I would even say that the distinction between "preventing
the interpreter from crashing" and "getting sane results" is not really
relevant.  If your program doesn't crash the interpreter, but loose some
append()s or produce similar nonsense if you forget a lock, then we get
the drawbacks of the GIL without its benefits...

In practice, the list of operations that is atomic should (ideally) be
documented more precisely -- one way to do that is to specify it at the
level of built-in methods instead of syntax, e.g. saying that the method
list.append() works atomically, and so does dict.setdefault() as long as
all keys are "built-in enough" objects.


A bientot,

Armin