On 10/01/2012 08:32 PM, Andrew Francis wrote:
Hi Ronny:
------------------------------------------------------------------------ *From:* Ronny Pfannschmidt <Ronny.Pfannschmidt@gmx.de> *To:* Armin Rigo <arigo@tunes.org> *Cc:* Sarah Mount <s.mount@wlv.ac.uk>; PyPy_Developers <pypy-dev@python.org> *Sent:* Sunday, September 30, 2012 10:43 AM *Subject:* Re: [pypy-dev] PyPy STM
the reason why i start with generators instead of green-lets is simply cause they cannot ever be nested.
I would hold the reverse view since to the best of my knowledge, generators don't lend themselves to composability. I suspect greenlets (or continuelets?) are easier to work with.
practically there is no semantic difference between yielding a request for completion of "another process" and calling into a function its practically equivalent, the "calls" just look different
i think for most purposes simple sequential communicating programs are way more easy to reason about than anything else.
That is why I use Stackless Python.... And I would love to try to incorporate STM into Stackless.
no point in pushing that into old stackless once continulet-jit-3 works as we hope to, "continuations" will have distinct stacks (Armin please correct if i missed )
I recently read a book by John Reppy on Concurrent ML. In it, Reppy gives the theoretical reasons about the virtues of synchronous channels with buffers.
channels with buffers will be one of my primitives
You may want to look at John Reppy's paper "Parallel Concurrent ML." ( http://people.cs.uchicago.edu/~jhr/papers/2009/icfp-parallel-cml.pdf). Under the hood, Reppy and company use optimistic locking to implement efficient message passing on a multicore system. Another paper you should check out is "Scalable Join Patterns" by Russo and Turon ( I exchanged e-mails with these guys).
I'll admit, I don't understand all the ins-and-outs of the papers. However my takeaway is that STM and STMish mechanisms are used in a very small place: the message passing system's implementation. Given that message passing systems typically share information through messages :-), this ought to create a very small footprint for transaction conflicts.
stm will be everywhere in pypy because we need it everywhere to have it appear as if we still have the gil
the transaction module approach already seems to require to chunk up programs in semi-small transactions, that may cause other transactions to be scheduled
which seems more and more like twisted's defereds. to my eyes twisted style code is a kind of spaghetti that is very hard to reason about.
which is why i want to experiment executing multiple longer sequential programs in chunks that may be interleaved and/or parallel
As I understand it, a properly functioning STM implementation is going to make transactions appear that they are serialised. As for interleaving. That is happening below the hood and is an efficiency and implementation issue. One may use the scheduler to directly enforce serialisation. (I have to review my notes for how STM aware is the Haskell runtime scheduler).
see Armin's latest blog entry on stm its no longer completely under the hood, but the transaction module will be a convenient interface to it for some kinds of program the base primitives will be thread.atomic/thread.exclusive_atomic my experiment will touch it from a different angle -- Ronny