[Python-Dev] Experimenting with STM on CPython

[Antoine Pitrou]

On Wed, 11 Apr 2012 15:31:09 +0200
Stefan Behnel <stefan_ml at behnel.de> wrote:
> 
> Ok. I guess once the code is there, the hardware will eventually catch up.
> 
> However, I'm not sure what you consider "large". A lot of manipulation
> operations for the builtin types are not all that involved, at least in the
> "normal" cases (read: fast paths) that involve no memory reallocation etc.,
> and anything that can be called by and doesn't call into the interpreter
> would be a complete and independent transaction all by itself, as the GIL
> is allowed to be released between any two ticks.

I think Armin's plan is not to work at the bytecode level, but make
transactions explicit (at least in framework code - e.g. Twisted or
Stackless -, perhaps not in user code). Perhaps he can elaborate on
that.

> Do you know if hybrid TM is possible at this level? I.e. short transactions
> run in hardware, long ones in software? (Assuming we know what's "long" and
> "short", I guess...)

There are other issues than the size of transactions. For example, one
issue is that not all operations may be allowed in a transaction:

“In addition, there are a number of instructions that may cause an
abort on specific implementations. These instructions include x87 and
MMX, mixed access to XMM and YMM registers, updates to non-status parts
of EFLAGs, updating segment, debug or control registers, ring
transitions, cache and TLB control instructions, any non-writeback
memory type accesses, processor state save, interrupts, I/O,
virtualization (VMX), trusted execution (SMX) and several miscellaneous
types.”

http://realworldtech.com/page.cfm?ArticleID=RWT021512050738

So, realistically, a (S)TM implementation in CPython (and probably also
in PyPy) would have to stand on its own merits, rather than betting on
pie-in-the-sky improvements of HTM implementations.

Regards

Antoine.