[pypy-dev] STM "version 2"

[Armin Rigo]

Hi all,

To keep you informed of the progress on STM:

In the middle of August I found a potentially better approach to STM,
which uses copies of objects more extensively (something which is
neither natural nor easy to do in C/C++, which is probably why it was
not researched before).

The main change is that all globally visible objects are and stay
read-only.  A write that occurs in theory on one such object is ---
like before --- done inside a local copy of the object.  The
difference with before is what occurs on commit.  Before, the local
copy would have its content copied back over the global object.  But
now, the local copy "becomes" the next version of the global object.
It makes the reads simpler and cheaper, because we don't have to worry
about other threads committing changes to the object in parallel.  We
end up with the old version of the global object that says, in its
header, "I'm outdated, here's a pointer to some newer version".
Obviously, this approach relies on good GC support to eventually
collect the old copies.

I ended up documenting it extensively there, in a very terse form so far:

* https://bitbucket.org/pypy/extradoc/raw/extradoc/talk/stm2012/stmimpl.rst

I adapted the high-level testing framework I wrote last year to check
that it is, or appears to be, correct (and found of course a couple of
subtle bugs):

* https://bitbucket.org/arigo/arigo/raw/default/hack/stm/python/c2.py
* https://bitbucket.org/arigo/arigo/raw/default/hack/stm/python/test_c2.py

And I rewrote and tested it in C:

* https://bitbucket.org/arigo/arigo/raw/default/hack/stm/c2

In the next days I will start to adapt PyPy to it.  This approach has
the advantage over the previous one to not require complex support for
the reads, which should also make the JIT backend work more directly.
Also, unlike the previous one, this approach works on non-Intel CPUs
(the C code has been tested on POWER64).


A bientôt,

Armin.