[pypy-dev] STM

[Andrew Francis]

Hi Armin:



________________________________
 From: Armin Rigo <arigo at tunes.org>
To: Andrew Francis <andrewfr_ice at yahoo.com> 
Cc: PyPy Developer Mailing List <pypy-dev at python.org> 
Sent: Monday, January 9, 2012 8:03 AM
Subject: Re: [pypy-dev] STM
 
Hi Andrew,

On Sun, Jan 8, 2012 at 21:24, Andrew Francis <andrewfr_ice at yahoo.com> wrote:

AF> A silly question? What is the paper you are discussing?

>The Problem with Threads, Edward A. Lee, EECS 2006.

Thanks!

AF> Perhaps this is off-topic, in  non-trivial stackless programme using
AF> cooperative scheduling, unless you explicitly impose a logical ordering, it is difficult to predict
AF> when a particular tasklet will  run. The round-robin doesn't mean much.

>Ok, so basically no real Stackless program relies on a particular order anyway.  This is in 
>line with what I expected.

Yes.

Even taking into consideration stuff like channel preferences, I think it is difficult to second guess
the scheduler based in even a moderately complex application. Where you really see the philosophy of
scheduler as an opaque entity is in the Bell Labs family of languages (Go included).  The reasons I
mention Go is that they are tackling the multiple CPU issue (albeit differently from what is proposed
here) and it and Stackless share family resemblances. 

Now what I believe is a bigger issue is rationalising the relationship between scheduler(s), 
tasklets, and threads. Again, maybe  STM solutions can be tested with an inexpensive 
prototype.

AF> If I understand what you are proposing, the problem I see with yielding in
AF> the coarse-grained tasklet-as-transaction, is that you probably need more
AF> machinery to ensure ACID properties, since you are now getting thread
AF> interweaving and potential race conditions.

>Yes, and this machinery is exactly what STM libraries offer.  Given
>that you have multiple transactions that can *mostly* occur in
>parallel, STM ensures that in *all* cases, it will look like the
>transactions have been run serially, while actually performing them in
>parallel under the hood.

A rich STM library like rstm offers a variety of techniques for making this happen. 
On the other hand, the Transactional specification has very specific (and maybe more
conservative) ways of achieving serialisation and race-free programmes. My gut feeling 
is that it is difficult to get a one size fits all solution. Maybe an interim solution is to get
to a stage where one has a prototype that allows for pluggable STM solutions? A "try before you
buy"strategy.

I am also not clear on whether a STM interface exposed to a Python programmer will be 
the same as STM (and other techniques, say lock-free data structures) used internally to 
support multiple processors.

I think this is an exciting time for stuff like PyPy/Stackless!

Cheers,
Andrew
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120109/fd8aee56/attachment.html>