[pypy-dev] Would the following shared memory model be possible?

Andrew Francis andrewfr_ice at yahoo.com
Thu Jul 29 22:39:16 CEST 2010


Hi Michael:

--- On Thu, 7/29/10, Michael Sparks <ms at cerenity.org> wrote:

> It's a pity we didn't get a chance to chat at the
> conference. (I was the one videoing everything for upload after >transcoding :)

Yes I noticed. I gave the talk "Prototyping Go's Select with stackless.py for Stackless Python." Much of that talk dealt with rendezvous semantics courtesy via synchronous channels.

I will post the Original slides and the Revised version (mistakes corrected ) in a day or two. 
 
> > >http://www.kamaelia.org/MiniAxon.html
> > 

> I'm biassed towards Kamaelia (naturally :-), but I agree.
> MiniAxon is just a  toy/tutorial. Early in kamaelia's history we >considered using stackless, but rejected it simply because we wanted to >work mainly with mainline python, rather than a specialised version.

Fair enough. Currently Stackless Python is being integrated with Psyco and will be available as a module. 

> Other things in Stackless's favour (IIRC) - include the
> fact that you can  pickle generators, and send them across network >connection, unpickle them and let them continue running. I don't know 
>if you do the same with tasklets, but I wouldn't be surprised if you do :)

As long as you do not have a C Frame involved, you can pickle a tasklet.
That was the subject of my "Silly Stackless Python Trick" lighting talk.
I was going to demonstrate a version of the Sieve of Eratosthenes that could be pickled and resumed on another machine. However my HP Netbook had a non-standard VGA output connection and I needed to install Stackless
on a loaner ThinkPad that died as I hooked it up. However you saw all
that :-(

> That means you have potential for process migration.

Yep. Gabriel Lavoie does a lot of work in that area with PyPy (thanks Gabriel !)

> Doing that sensibly though IMO would require better understanding
> in the system of what the user is trying to achieve and what they're >sending. (It's easy to think of examples where this causes more pain than >it's worth after all)

You have to understand what can be pickled. Occasionally you are in for
a surprise (i.e. functools).

>You could argue in that case that the biggest _real_ difference is 
>that we try to use a unified API for different concurrency
>methods. 

Well I would argue that Stackless has a simple elegant model. The addition of select just adds more power.Stackless channels can also serve as generators (they are iterable). I recently took a stab at writing the Sleeping Barber's problem. I think in Stackless, the basic solution was about 30 lines. Very little clutter.

> One **highly subjective** other thing in our favour, is
> that generators are limited to a single level of control flow 
>(ie non-nestable without a trampoline). This doesn't sounds like
> an advantage, but it tends to lead to simpler components which are 
>in turn reusable. (and that I view as useful :)

Okay. I attended Ray's Hettinger's talk on Monocle. In the past
I have encountered situations where I bumped up with the nesting problem.
If I recall, the problem involved request handlers that had a RPC style AND made additional Twisted deferred calls:

class MyRequestHandler(...):
   
    @defer.inlineCallbacks
    def process(self):
        try:
            result = yield
client.getPage("http://www.google.com")
        except Exception, err:
            log.err(err, "process getPage call
failed")
        else:
            # do some processing with the result 
            return result

looks reasonable but Python will balk. Nested generators. Only way around it is that you had to hope that the Twisted protocol was properly written and chain deferreds.

> Have fun,

I do :-)

Cheers,
Andrew



      




More information about the Pypy-dev mailing list