Are python objects thread-safe?

[Duncan Booth]

RajNewbie <raj.indian.08 at gmail.com> wrote:

> Say, I have two threads, updating the same dictionary object - but for
> different parameters:
> Please find an example below:
> a = {file1Data : '',
>        file2Data : ''}
> 
> Now, I send it to two different threads, both of which are looping
> infinitely:
> In thread1:
> a['file1Data'] = open(filename1).read
>           and
> in thread2:
> a['file2Data'] = open(filename2).read
> 
> My question is  - is this object threadsafe? - since we are working on
> two different parameters in the object. Or should I have to block the
> whole object?
> 
It depends exactly what you mean by 'threadsafe'. The GIL will guarantee 
that you can't screw up Python's internal data structures: so your 
dictionary always remains a valid dictionary rather than a pile of bits.

However, when you dig a bit deeper, it makes very few guarantees at the 
Python level. Individual bytecode instructions are not guaranteed 
atomic: for example, any assignment (including setting a new value into 
the dictionary) could overwrite an existing value and the value which is 
overwritten may have a destructor written in Python. If that happens you 
can get context switches within the assignment.

Other nasty things can happen if you use dictionaries from multiple 
threads. You cannot add or remove a dictionary key while iterating over 
a dictionary. This isn't normally a big issue, but as soon as you try to 
share the dictionary between threads you'll have to be careful never to 
iterate through it.

You will probably find it less error prone in the long run if you get 
your threads to write (key,value) tuples into a queue which the 
consuming thread can read and use to update the dictionary.

-- 
Duncan Booth http://kupuguy.blogspot.com