Hi, I really like the work you do and try to support by trying to make all my python project runing under PyPy, being PyPy proponent, convince others to PyPy, putting some founding, talk about PyPy. I try to learn PyPy architecture and someday contribute (if I got more free time). I know that PyPy bind a lot of optimizations. My question is how it looks from bytecode interpretation (so without JITing)? I learned about AST optimizers for CPython (like https://bitbucket.org/haypo/astoptimizer, hotpy). Does PyPy bytcode interpreter implements some of them on bytecode level? I know that PyPy produce similar bytecode as CPython (http://doc.pypy.org/en/latest/architecture.html#the-python-interpreter), so I guess PyPy doesn't introduce such optymalizations. But maybe on some other interpreter level? PyPy fail on some astoptimizer tests. It is pure Python, but manipulates AST. I make a little introspection and i think, the reason is due to small bytecode differences? The AST (resulted and expected) looks similar, by the PyPy one is simpler.
Hi, 2013/1/19 Robert Zaremba <robert.zaremba@zoho.com>
Thank you, and good luck!
IIRC, PyPy used an AST compiler before CPython did. Most (all?) code optimizations are already performed at the AST level, see https://bitbucket.org/pypy/pypy/src/default/pypy/interpreter/astcompiler/opt... Cpython on the other hand optimizes the bytecode itself (in Python/peephole.c) PyPy fail on some astoptimizer tests. It is pure Python, but manipulates
astoptimizer is similar to the one built in PyPy, but much more aggressive, and can sometimes change semantics. For example, it assume that builtin functions are not shadowed. It can be interesting in many cases, but will produce different behavior in some cases. Most of the failures in astoptimize tests are shallow: When compiling from an ast tree, PyPy will optimize it and modify it in-place (CPython doesn't) I modified a bit these tests to add a call to compile() on the ast used for comparison, and most tests pass now. They found a crash too, around the empty set(); I fixed it in my workspace, will try to submit it tonight. -- Amaury Forgeot d'Arc
Hi, We should also mention that all, or almost all, of these optimizations have limited benefits in PyPy: as soon as the JIT kicks in, they are entirely pointless. They have a limited benefit anyway in the sense that they make the interpreter part marginally faster. A bientôt, Armin.
Thanks for all responses! I will try to compare both imlementations. I'm aware about JIT (its own optimizations in conjunction with gcc ones), so my question was only about interpreter stage (non JITed path). Cheers.
Hi, 2013/1/19 Robert Zaremba <robert.zaremba@zoho.com>
Thank you, and good luck!
IIRC, PyPy used an AST compiler before CPython did. Most (all?) code optimizations are already performed at the AST level, see https://bitbucket.org/pypy/pypy/src/default/pypy/interpreter/astcompiler/opt... Cpython on the other hand optimizes the bytecode itself (in Python/peephole.c) PyPy fail on some astoptimizer tests. It is pure Python, but manipulates
astoptimizer is similar to the one built in PyPy, but much more aggressive, and can sometimes change semantics. For example, it assume that builtin functions are not shadowed. It can be interesting in many cases, but will produce different behavior in some cases. Most of the failures in astoptimize tests are shallow: When compiling from an ast tree, PyPy will optimize it and modify it in-place (CPython doesn't) I modified a bit these tests to add a call to compile() on the ast used for comparison, and most tests pass now. They found a crash too, around the empty set(); I fixed it in my workspace, will try to submit it tonight. -- Amaury Forgeot d'Arc
Hi, We should also mention that all, or almost all, of these optimizations have limited benefits in PyPy: as soon as the JIT kicks in, they are entirely pointless. They have a limited benefit anyway in the sense that they make the interpreter part marginally faster. A bientôt, Armin.
Thanks for all responses! I will try to compare both imlementations. I'm aware about JIT (its own optimizations in conjunction with gcc ones), so my question was only about interpreter stage (non JITed path). Cheers.
participants (4)
-
Amaury Forgeot d'Arc -
Armin Rigo -
Maciej Fijalkowski -
Robert Zaremba