[Python-ideas] A concurrency survey of sorts

Mike Meyer mwm at mired.org
Wed Nov 2 20:36:26 CET 2011


In order to get a better idea of where things stand, I'd like to get
answers to a few questions. This isn't a traditional broadbased
survey, but an attempt to get answers from a few people who might know
or have good ideas. This is probably where I should have started, but
better late than never.

1) How much of the Python standard library is known to be thread safe?

2) How many packages in PyPI are known to be thread safe?

3) Can you suggest another approach to getting safe high-performance
shared data in concurrent operation? I've already considered:

 a) I proposed making actions that mutate data require locked objects,
because I've seen that work in other languages. I recognize that
doesn't mean it will work in Python, but it's more than I can say
about the alternatives I knew about then.,

  b) Bertrand Meyer's SCOOPS system, designed for Eiffel. It has two
major strikes against it: 1) it is based on type attributes on
*variables*, andI could figure out how to translate that to a language
where variables aren't typed. 2) I don't know that there's a working
implementation.

4) Can you suggest a minor change that would move things toward safer
concurrent code with high-performance shared data? I can see two
possibilities:

 a) Audit any parts of the standard library that aren't already known
to be thread safe, and flag those that aren't.  Fixing them may want
to wait on a better mechanism than posix locks.

 b) Add a high-level, high-performance shared object facility to the
multiprocess package.

    Thanks,
    <mike



More information about the Python-ideas mailing list