[pypy-dev] Stacklets

[Andrew Francis]

Hi Armin and Folks:




________________________________
From: Armin Rigo <arigo at tunes.org>
To: Александр Седов <elec.lomy.ru at gmail.com>
Cc: pypy-dev at python.org
Sent: Friday, September 23, 2011 4:38 PM
Subject: Re: [pypy-dev] Stacklets

>Thanks!  Work in this direction is already well advanced.  More
>precisely, the directory pypy/module/_stackless is obsolete and gone,
>and the pure Python module lib_pypy/stackless.py has been ported to
>use _continuation.  (I wonder somehow why we had all this code in
>pypy/module/_stackless that seems not needed any more.)

I downloaded the latest build  looked at stackless.py and wrote a simple test. When I have
time, I will try incorporate and test my new (and very unofficial) stackless features. I also
 should
have enough code to run weird examples that ought to stress the system.

That said, I'm really impressed that continulets was ported so fast. Kudos to Rodrigo! A 
suggestion. Perhaps it would be good to keep the test for whether CPython is the interpreter and greenlets
ought to be used? In this fashion, someone that does want to use pypy-c can still play with stackless. And
one authoritative copy of stackless.py can be kept (as opposed to hacking a version for greenlets).

> But it is not multi-thread-safe so far, which is probably an easy fix, using a
>thread-local instead of all these global variables initialized in
>_init() in stackless.py.

In the stackless mailing list, there is a conversation about some gotchas concerning threads and
tasklets that one may want to read. Although the conversation revolves around C Stackless Python
based internals, the 50000ft view is about threads dying with tasklets binded to them.

>So far you can pickle continulets, greenlets, and coroutines, but
 not
>tasklets.  It looks messy because of early-optimization issues from
>stackless.py .

All I know about pickling is that one cannot pickle a tasklet with cstate. Or a blocked tasklet.
I don't know how that translates into the pypy world. 

>--- e.g. it would be much more natural for it to switch
>to the main tasklet every time it needs to do the scheduling and
>choose the next tasklet to switch to, instead of being clever and
>switching directly to the target tasklet; this "unwanted cleverness"
>prevents pickling from working at all, because it sees too much
>unrelated stuff in a suspended tasklet

This makes the scheduler sound like a generator trampoline. Also 
adds an additional context switch, if I understand things correctly.

>All in all what I would be most happy with, at this point, is if
>someone would step up and finish porting and maintaining stackless.py.
>Ideally it would be someone that needs this code for his own
>projects, too.

Seems that Rodrigo did a pretty good job. What is left to be done?

Cheers,
Andrew
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20110924/e2469cbc/attachment-0001.html>