[Python-Dev] A new JIT compiler for a faster CPython?

[Stefan Behnel]

Alex Gaynor, 18.07.2012 03:24:
> Victor Stinner writes:
>> Example:
>> ----
>> a = GETLOCAL(0); # "a"
>> if (a == NULL) /* error */
>> b = GETLOCAL(1); # "b"
>> if (b == NULL) /* error */
>> return PyNumber_Add(a, b);
>> ----
>>
>> I don't expect to run a program 10x faster, but I would be happy if I
>> can run arbitrary Python code 25% faster.
>>
>> --
>>
>> Specialization / tracing JIT can be seen as another project, or at
>> least added later.
> 
> This is almost exactly what Unladen Swallow originally did.  First, LLVM will 
> not
> do all of the optimizations you are expecting it to do out of the box.  It will
> still have all the stack accesses, and it will have all of the ref counting
> operations.  You can get a small speed boost from removing the interpretation
> dispatch overhead, but you also explode your memory usage, and the speedups are
> tiny.

My experience with Cython tells me that even if you move the entire
interpretation overhead out of the way, you'd only get some 5-20% speedup
for real code, rarely more if you have some really tight loops. Adding a
full-blown JIT compiler to the dependencies just for that is usually not
worth it, and Unladen Swallow succeeded in showing that pretty clearly.
It's when you start specialising and optimising code patterns that it
becomes really interesting, but you can do that statically at build time or
compile time in most cases (at least in the more interesting ones) and
Cython is one way to do it. Again, no need to add a JIT compiler.

The nice thing about JIT compilers is that you can give them your code and
they'll try to optimise it for you without further interaction. That
doesn't mean you get the fastest code ever, it just means that they do all
the profiling for you and try to figure it out all by themselves. That may
or may not work out, but it usually works quite ok (and you'll love JIT
compilers for it) and only rarely gets seriously in the way (and that's
when you'll hate JIT compilers).

However, it requires that the JIT compiler knows about a lot of
optimisations. PyPy's JIT is full of those. It's not the fact that it has a
JIT compiler at all that makes it fast and not the fact that they compile
Python to machine code, it's the fact that they came up with a huge bunch
of specialisations that makes lots of code patterns fast once it detected
them. LLVM (or any other low-level JIT compiler) won't help at all with that.

Stefan