[Python-Dev] Pythonic concurrency
Phillip J. Eby
pje at telecommunity.com
Thu Sep 29 19:06:29 CEST 2005
At 10:48 AM 9/29/2005 -0600, Bruce Eckel wrote:
>I haven't spent the weekends on the paper yet (but it looks like that
>is what it would take), but I had the impression that they were
>talking about the lock-free techniques such as the ones used in Java
>5. Basically, you start a write operation "in the background" without
>locking the data structure, so reads can continue while the
>calculation is taking place but before the result is committed. When
>the result is ready, an atomic "test and write" operation is used to
>determine whether any other task has modified the value in the
>meantime, and if not to commit the new value. If another task did
>modify the value, then the calculation begins anew.
>That was my take, but I haven't studied everything about STM yet, so
>I'm probably missing something.
No, that's certainly the general idea. The issue for an imperative
language like Python is that side-effects are the norm, rather than an
exception. The Haskell implementation of the idea effectively relies on
the fact that in Haskell you have to jump through monadic hoops to get side
effects, so if you have a low-level transactional memory facility, you can
create monadic combinators that restrict those side effects to occuring in
the way you want them to. This is so utterly alien to Python's execution
model - or indeed most imperative languages' execution models - that I
don't see how it can be modelled in Python in a way that retains the
significant benefits of the approach.
Now, in PyPy, I suppose it might be possible to create a custom
object-space that works that way, and I hadn't thought of that
before. Maybe you should run that paper past one of the PyPy wizards and
see if you can get them interested in the idea, then stand back and see
what happens. :)
>The one thing about this paper is that it seems to be an orthogonal
>perspective to anything about concurrency that *I* have seen before.
Yes - it's dramatically different, and seems like the One True Way to do
imperative concurrency with side-effects. Certainly transactions work
nicely for databases, anyway. :)
>So with concurrency, I would like to know when I do something wrong,
>but if I am told at runtime that's OK with me as long as I'm told.
Yeah, but in current Python implementations - with the possible exception
of PyPY - detecting the problem at all is virtually impossible. PyPy is an
extensible VM, though, so in principle you could create some kind of
transactional object space that could keep track of things and tell you if
you mess up.
More information about the Python-Dev