Is python is fully thread safe

Dave Brueck dave at pythonapocrypha.com
Mon Jul 26 11:25:24 EDT 2004


Donn Cave wrote:

> Quoth Franois Pinard <pinard at iro.umontreal.ca>:
> ...
> | Well, bugs occurring out of lack of thread-safety might, in many
> | circumstances, hide for long before popping out, and hide again for long
> | afterwards.  They are not easy to reproduce.  "Testing reveals presence
> | of bugs, not their absence", as a proverb, is especially true in the
> | context of thread-safety.
> |
> | Unless I'm missing a point (which is very possible), it does not seem
> | to me that Python documentation is especially collaborative at teaching
> | where threads are safe or not to use, so I'm left with the unpleasant
> | feeling that my threaded Python applications should (ideally) never be
> | allowed to go in real production.  It's OK if I'm the only one to suffer
> | from unexpected problems.
> |
> | I would much like if Python documentation had more definite and
> | definitive answers about thread-safety of Python, everywhere.
> 
> Well, I too am skeptical about multithreaded programs in production -
> written in Python or anything else.  That is not to say it can't work,
> but concurrency is just fraught with potential for trouble.  I don't
> imagine that's going to stop people from using them, but they really
> need to be able to think somewhat analytically about the issues.
> I think you understand that it's a little more complex than "Hot dog,
> the documentation says Python is thread safe!"
> 
> Ironically, Python is actually a little ahead here because it doesn't
> strictly speaking support concurrency.  That makes Python internals
> effectively the same as thread safe, so thread safety issues would be
> confined to application level issues and external functions that run
> after releasing the interpreter lock.  I've had no particular trouble
> in my programs using a multithreaded API where others reportedly did
> have trouble, and for all I know it may be just because I was using
> Python and they weren't.

This is quite true - automatic cleanup of resources on thread 
termination combined with the fact that multithreaded Python code won't 
corrupt the interpreter internals makes Python a fairly safe platform 
for multi-threading - you certainly end up with a bigger safety net 
than, say, C++.

As an aside, I think the documentation wrt to multi-threaded safety 
could be improved, but I'm not sure how exactly to improve it because 
"thread safety" doesn't always have a clear definition. For example, two 
threads can safely access and even modify the same Python object (such 
as a dictionary) without explicit locking and the interpreter will not 
crash, so at that low-level you could say it is thread safe. But at the 
semantic level of your particular application, a function that modifies 
the same dictionary by multiple threads may or may not need explicit 
locking around the dictionary modification (depending on what is being 
done), so at that level the access may not be considered thread safe.

As to what the Python documentation needs to say (and, in the end, 
perhaps it already does this well enough), I'd think it needs to make a 
few things clear:

1) Accessing the same Python object from multiple threads from Python 
code won't corrupt the internals of the interpreter.

2) Modules/functions that are just plain not thread-safe should be 
marked as such (e.g. the fpectl module).

Beyond that, can you think of anything else? I doubt that the work 
required to document the low-level thread safety of each and every 
function is really worth it (not to mention the effort required to keep 
it up to date over time) - partly because of #1, above, partly because 
in decent multi-threaded programs there aren't too many shared objects 
and the points at which they are shared are fairly well-defined, and 
partly because when in doubt, it's fairly simple to add a mutex around a 
  resource.

-Dave



More information about the Python-list mailing list