[Python-ideas] A concurrency survey of sorts

[Antoine Pitrou]

On Wed, 2 Nov 2011 12:36:26 -0700
Mike Meyer <mwm at mired.org> wrote:
> In order to get a better idea of where things stand, I'd like to get
> answers to a few questions. This isn't a traditional broadbased
> survey, but an attempt to get answers from a few people who might know
> or have good ideas. This is probably where I should have started, but
> better late than never.
> 
> 1) How much of the Python standard library is known to be thread safe?

It depends what the thread safety assumptions are.
I'd say not much of it, but not much of it *needs* to either.
For example, if you mutate the same XML tree concurrently,
my opinion is that the problem is with your code, not the stdlib :-)
(on the other hand, if mutating *different* XML trees concurrently
produces errors, then it's a stdlib bug)

Buffered binary file objects are known to be thread-safe.
Text file objects are not, except perhaps for writing (I think we did
the latter, because of print() and logging; I'm not sure it's well
tested, though).
Raw file objects are not (they are "raw" after all: they simply expose
the OS' behaviour).

As a separate issue, binary file objects forbid reentrant accesses from
signal handlers. Therefore I would advocate against using print() in a
signal handler.
(see http://bugs.python.org/issue10478)

>  b) Add a high-level, high-performance shared object facility to the
> multiprocess package.

It will be difficult (IMHO: very difficult) to devise such a thing.
multiprocessing already has shared memory facilities, though - but they
are very low-level.

Regards

Antoine.