[pypy-dev] The Work Plan Re: STM proposal funding

[Andrew Francis]

Hi Armin:



________________________________
From: Armin Rigo <arigo at tunes.org>
To: Andrew Francis <andrewfr_ice at yahoo.com> 
Cc: PyPy Developer Mailing List <pypy-dev at python.org>; "stackless at stackless.com" <stackless at stackless.com> 
Sent: Wednesday, April 11, 2012 8:29 AM
Subject: Re: The Work Plan Re: [pypy-dev] STM proposal funding


On Sun, Apr 8, 2012 at 01:25, Andrew Francis <andrewfr_ice at yahoo.com> wrote:
AF> Question: without specific transaction_start() and transaction_commit()
AF> calls, how does rstm know what the start and finish of transactions are?

>Please take a different point of view: if the proper adaptation is
>done, the users of the "stackless" module don't need any change.  

Yes. Suggestion: perhaps in a future position paper, you can state as a design principle that the programmer
does not explicitly state a commit point?

So returning to the example,  a programme more in the spirit of what you are doing is:

def transferTasklet(ch)
      def transfer(toAccount, fromAccount, amount):
 .          fromAccount.withdraw(amount)
            toAccount.deposit(amount)

      while someFlag:
               toAcount, fromAccount, amount = ch.receive()
               transaction.add(transfer, toAccount, fromAccount, amount)

if __name__ == "__main__":
      ch = stackless.channel()
      task = stackless.tasklet(transferTasklet)(ch)
      transaction.add(task)
     ....
     """
     let us assume that the stackless scheduler and transaction manager are somehow integrated 
     """
     stackless.run()    


and we assume that the underlying system "magically" takes of stuff. However the programmer does have to throw the
transaction manager a bone (hence transaction.add())

>They will continue to work as they are, with the same semantic as today ---
>with the exception of the ordering among tasklets, which will become
>truly random.  

Noted. I think issues of ordering (serialisation)  is a consequence of a correctly implemented transaction manager.

If I understand your strategy, the approach is to give a Python developer a race free programme with minimum effort. 

However I think a major concern would be implementations that minimises conflicts/contention. As I stated
in previous posts, I believe in the case of Stackless, the message passing system is natural way to give the application
programmer more control in regard to minimising conflicts/contention (or whatever term the literature uses).

>This can be achieved by hacking at a different level.

I am interested in what this hacking looks like under the hood. Again, I am assuming as a design principle guideline
(and to focus folk's efforts), one should be providing the rstm module just enough information to work but make no
assumptions about how it works. So one ought not depend on whether  strategies like eager/lazy evaluation (right now, 
I think your system depends on lazy evaluation since I see stuff like redo logs), are used. 

Still I am interested in stuff like: would rstm function correctly if  the underlying implementation tracked tasklets rather than threads?
I am looking at your implementation and the AME.c/h file in Duhton. I want to get into a position to do some hacking. So I want 
to think about:

What do we know about the relationship between transactions, tasklets and threads?

Under what conditions in a Stackless application, would conflicts occur? What could be done by 1) The programmer 2) The STM
implemenation to avoid this?

Some side notes: I was reading about the Haskell implementation in the "Software Transactional Memory 2nd" book (chapter
4.6.2 - conditionl synchronization). It seems that Haskell has a user space "thread" library that is somewhat integrated with the STM.
I am going to look at this more since this is something that Stackless could take advantage of. 

Salut,
Andrew
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120415/d2cd4fa2/attachment.html>