[pypy-dev] llvm.gcroot

[Armin Rigo]

Hi Eric, hi all,

A field report about the 'llvmgcroot' branch...

I've been playing with ways to remove the "shadow stack" of GC roots.
More precisely, with our own framework GCs, the problem is that in C
there is no portable way to walk the current stack and locate the local
variables containing pointers to GC-managed objects (let alone modify
these pointers safely, in case of a moving GC).  So what we do at the
moment is to push/pop all such pointers in a custom shadow stack around
each call, everywhere.

In LLVM there is a very recently implemented primitive called
llvm.gcroot() which allows us to declare local variables which contain
pointers to GC-managed objects.  Then with the help of a few pages of
custom C++ code to dynamically link with LLVM's native code backend
'llc', we can put extra data into the assembler that describes where
these local variables are and how big each function's stack frame is.
This is not platform-specific so far, in the sense that the same C++
code works on any platform.  There is still a bit of platform-specific
code, which is how exactly to interpret this data to walk the stack from
our GC's collect() methods.

For more info:
  C++ plugin for llc (see README.txt):
    http://codespeak.net/svn/user/arigo/hack/pypy-hack/stackrootwalker
  PyPy branch:
    http://codespeak.net/svn/pypy/branch/llvmgcroot
  with the platform-specific stack walking code:
    http://codespeak.net/svn/pypy/branch/llvmgcroot/pypy/rpython/memory/gctransform/llvmgcroot.py

This gives some modest speed-up over a plain llvm translation (which is
slower than a C translation, which is itself slower than a --llvm-via-c,
but the latter doesn't support llvm.gcroot(), obviously, as it goes
through C as well).  To activate this in the branch, use translate.py
--llvmgcroot.

Then I started playing with something similar for genc by parsing the .s
files produced by gcc.  Completely platform-specific, obviously.  To
active this in the branch, use translate.py --llvmgcroot --backend=c
(sorry if it's a bit confusing).  This uses the special 'asm' keyword of
gcc to generate markers that are found by
http://codespeak.net/svn/pypy/branch/llvmgcroot/pypy/translator/c/trackgcroot.py
and we get data added to the .s file in the same format as the custom
C++ GC plugin for LLVM.

Definitely an obscure hack, though it's kind of safe in the sense that
if gcc does too advanced optimizations it will not give obscure
segfaults, but clean crashes in trackgcroot.py itself.  It gives the
same modest speed-ups for pypy-c.

In both the C and the LLVM case, the presence of the gcroot markers
currently prevent some low-level optimizations by, respecitvely, llc and
gcc.  The reason is that the local variables that are marked as GC roots
cannot be simply "fast" variables that the compiler is free to put in
any register and move around as needed.  I'm not sure how to fix this
for llc, but for gcc I have the project to try to extend trackgcroot.py.

Finally, if we're talking about maximum performance, it might be
possible (given enough time and obscure hacking) to use trackgcroot.py
on the assembler files produced by gcc during a --llvm-via-c
translation...  by convincing LLVM to put in the C code things that look
like calls, but that are really macros that expand to the asm()
markers...


A bientot,

Armin.