[Python-ideas] A concurrency survey of sorts

[Jim Jewett]

On Wed, Nov 2, 2011 at 3:36 PM, Mike Meyer <mwm at mired.org> wrote:

> 1) How much of the Python standard library is known to be thread safe?

None.  Though our confidence in the threading library is fairly high
(except when the underlying C library is broken).

Not so long ago, there were a series of changes to the regression
tests that boiled down getting rid of spurious failures caused by
tests running serially, but in an unusual order.  If that level of
separation was still new, then finer-grained parallelism can't really
be expected to work either.

That said, test cases relied far more on global state than a typical
module itself does, so problems are far more likely to occur in user
code than in the library.


>  a) I proposed making actions that mutate data require locked objects,
> because I've seen that work in other languages. I recognize that
> doesn't mean it will work in Python, but it's more than I can say
> about the alternatives I knew about then.,

If you really want to do this, you should probably make the changes at
the level of "object" (or "type") and inherit them everywhere.  And it
may simplify things to also change the memory allocation.

There are a few projects for remote objects that already use a
different memory model to enforce locking; you could start there.

>  b) Bertrand Meyer's SCOOPS system, designed for Eiffel. It has two
> major strikes against it: 1) it is based on type attributes on
> *variables*, andI could figure out how to translate that to a language
> where variables aren't typed.

Actually, that isn't so bad.  Common Lisp doesn't normally type
variables at the source code level, but
    (a)  You can explicitly add typing information if you want to, and
    (b)  The compiler can often infer types

If you want this to mesh with python, the constraints are similar; not
only does the locking and safety marking have to be unobtrusive, it
probably has to be optional.  And there is existing (if largely
superseded by Pypy) work on type inference for variables.

-jJ