[Python-Dev] Experimenting with STM on CPython

Armin Rigo arigo at tunes.org
Wed Apr 11 17:08:09 CEST 2012

Hi Antoine, hi Stefan,

On Wed, Apr 11, 2012 at 16:33, Antoine Pitrou <solipsis at pitrou.net> wrote:
> I think Armin's plan is not to work at the bytecode level, but make
> transactions explicit (at least in framework code - e.g. Twisted or
> Stackless -, perhaps not in user code). Perhaps he can elaborate on
> that.

Yes, precisely.  It should be explained in the proposal.  The
references in "http://pypy.org/tmdonate.html#why-hasn-t-the-idea-been-implemented-for-cpython-already"
don't change CPython (or only minimally).  They use Hardware TM, but
(the most important thing imho) they target bytecode-level
transactions --- i.e. the programmer is still stuck with the
"threading" module.

About using it explicitly in user code: I found out that there are use
cases to do so directly.  If you have a CPU-intensive program that

    for x in some_random_order_iterator:

Then if the things you do are "not too dependent on each other", you
can win by replacing it with:

    for x in some_random_order_iterator:
        transaction.add(do_stuff_for, x)

and no other change.  It has exactly the same semantics, and in this
case you don't really need a framework in which to hide the
transaction.add().  Compare it with the situation of spawning threads:
you need to carefully add locks *everywhere* or your program is buggy
--- both in today's CPython or in a GIL-less,
bytecode-level-transaction CPython.

By the way, that's why I said that transactions are arbitrarily long:
one transaction will be, in this case, everything that do_stuff_for(x)

A bientôt,


More information about the Python-Dev mailing list