[pypy-dev] PyPy STM

[Andrew Francis]

Hi Ronny:



________________________________
 From: Ronny Pfannschmidt <Ronny.Pfannschmidt at gmx.de>
To: Armin Rigo <arigo at tunes.org> 
Cc: Sarah Mount <s.mount at wlv.ac.uk>; PyPy_Developers <pypy-dev at python.org> 
Sent: Sunday, September 30, 2012 10:43 AM
Subject: Re: [pypy-dev] PyPy STM
 
>the reason why i start with generators instead of green-lets is simply 
>cause they cannot ever be nested.

I would hold the reverse view since to the best of my knowledge, generators
don't lend themselves to composability. I suspect greenlets (or continuelets?)
are easier to work with. 

>i think for most purposes simple sequential communicating programs
>are way more easy to reason about than anything else.

That is why I use Stackless Python.... And I would love to try to incorporate STM into Stackless.

I recently read a book by John Reppy on Concurrent ML. In it, Reppy gives the theoretical reasons 
about the virtues of synchronous channels with buffers. 

You may want to look at John Reppy's paper "Parallel Concurrent ML." ( http://people.cs.uchicago.edu/~jhr/papers/2009/icfp-parallel-cml.pdf). Under the hood, Reppy and company use optimistic locking to implement efficient message passing on a multicore system. Another paper you should check out is "Scalable Join Patterns" by Russo and Turon ( I exchanged e-mails with these guys). 

I'll admit, I don't understand all the ins-and-outs of the papers. However my takeaway is that STM and STMish mechanisms are used in a very small place: the message passing system's implementation. Given that message passing systems typically share information through messages :-), this ought to create a very small footprint for transaction conflicts. 

>the transaction module approach already seems to require to chunk up 
>programs in semi-small transactions,
>that may cause other transactions to be scheduled

>which seems more and more like twisted's defereds.
>to my eyes twisted style code is a kind of spaghetti
>that is very hard to reason about.

>which is why i want to experiment executing multiple longer sequential 
>programs in chunks that may be interleaved and/or parallel

As I understand it, a properly functioning STM implementation is going to make transactions
appear that they are serialised. As for interleaving. That is happening below the hood and is an 
efficiency and implementation  issue. One may use the scheduler to directly enforce serialisation.
(I have to review my notes for how STM aware is the Haskell runtime scheduler). 

Cheers,
Andrew
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20121001/0f58e5f6/attachment.html>