[pypy-dev] Question about stm_descriptor_init(), tasklets and OS threads
arigo at tunes.org
Thu Apr 19 21:37:15 CEST 2012
On Thu, Apr 19, 2012 at 19:43, Andrew Francis <andrewfr_ice at yahoo.com> wrote:
> I am trying to understand enough to get into a position to attempt an
I believe you are trying to approach the problem from the bottom-most
level up --- which is a fine way to approach problems; but in this
case, you are missing that there are still a few levels between were
you got so far and the destination, which is the stackless.py module
in pure Python.
Let us try to approach it top-down instead, because there are far less
levels to dig through. The plan is to make any existing
stackless-using program work on multiple cores. Take a random
existing stackless example, and start to work by editing the pure
Python lib_pypy/stackless.py (or, at first, writing a new version from
scratch, copying parts of the original). The idea is to use the
"transaction" module. The goal would be to not use the _squeue, which
is a deque of pending tasklets, but instead to add pending tasklets
with transaction.add(callback). Note how the notion of tasklets in
the _squeue (which offers some specific order) is replaced by the
notion of which callbacks have been added. See below for what is in
The transaction.run() would occur in the main program, directly called
by stackless.schedule(). So the callbacks are all invoked in the main
program too; in fact all the "transaction" dispatching is done in the
main program, and only scheduling new tasklets can occur anywhere.
The callbacks would just contain a switch to the tasklet. When the
tasklet comes back, if it is not finished, re-add() it. This is all.
You have to make sure that all tasklet.switch()es internally go back
to the main program, and not directly to another tasklet. This should
ensure that the duration of every transaction is exactly the time in a
tasklet between two calls to switch().
Of course, this can all be written and tested against
"transaction.py", the pure Python emulator. Once it is nicely
working, you are done. You just have to wait for a
continuation-capable version of pypy-stm; running the same program on
it, you'll get multi-core usage.
More information about the pypy-dev