[Python-Dev] Pythonic concurrency

[Michael Hudson]

"Phillip J. Eby" <pje at telecommunity.com> writes:

> At 09:30 AM 9/29/2005 -0600, Bruce Eckel wrote:
>>This paper looks very interesting and promises some good ideas. It
>>also looks like it will require time and effort to digest.
>>
>>I've only read the first few pages, but one thing that does leap out
>>is at the beginning of section 3, they say:
>>
>>"... a purely-declarative language is a perfect setting for
>>transactional memory."
>>
>>What's not clear to me from this is whether STM will work in a
>>non-declarative language like Python.
>
> I spent a few weekends studying that paper earlier this year in order to 
> see if anything could be stolen for Python; my general impression was "not 
> easily" at the least.  One notable feature of the presented concept was 
> that when code would otherwise block, they *rolled it back* to the last 
> nonblocking execution point.  In a sense, they ran the code backwards, 
> albeit by undoing its effects.  They then suspend execution until there's a 
> change to at least one of the variables read during the forward execution, 
> to avoid repeated retries.
>
> It was a really fascinating idea, because it was basically a way to write 
> nonblocking code without explicit yielding constructs.  But, it depends 
> utterly on having all changes to data being transactional, and on being 
> able to guarantee it in order to prevent bugs that would be just as bad as 
> the ones you get from threading.

Oh yes, if implemented this really has be baked thoroughly into the
implementation.  It's not something that can be implemented as a
library (as far as I can see, anyway).

> Oddly enough, this paper actually demonstrates a situation where having 
> static type checking is in fact a solution to a non-trivial problem!  It 
> uses static type checking of monads to ensure that you can't touch 
> untransacted things inside a transaction.

Well, this is an extension of the way Haskell deals with side-effects
in general... but yes, it's interesting to be sure.

Cheers,
mwh

-- 
  People think I'm a nice guy, and the fact is that I'm a scheming,
  conniving bastard who doesn't care for any hurt feelings or lost
  hours of work if it just results in what I consider to be a better
  system.                                            -- Linus Torvalds