[pypy-svn] r63047 - pypy/branch/pyjitpl5/pypy/doc/jit

arigo at codespeak.net arigo at codespeak.net
Wed Mar 18 17:44:21 CET 2009 

Author: arigo
Date: Wed Mar 18 17:44:19 2009
New Revision: 63047

Modified:
   pypy/branch/pyjitpl5/pypy/doc/jit/index.txt
   pypy/branch/pyjitpl5/pypy/doc/jit/overview.txt
   pypy/branch/pyjitpl5/pypy/doc/jit/pyjitpl5.txt
Log:
Update (a little bit) to docs about the JIT.


Modified: pypy/branch/pyjitpl5/pypy/doc/jit/index.txt
==============================================================================
--- pypy/branch/pyjitpl5/pypy/doc/jit/index.txt	(original)
+++ pypy/branch/pyjitpl5/pypy/doc/jit/index.txt	Wed Mar 18 17:44:19 2009
@@ -19,6 +19,13 @@
 
 - Overview_: motivating our approach
 
+- `Draft notes`_ about the current work in PyPy (Feb 08).
+
+Older stuff
+------------------------------------------------------------
+
+*The following documentation is no longer up-to-date!*
+
 - Status_: using a pypy-c with a JIT
 
 - `How-to`_: make your own JIT compilers, with examples for tiny languages
@@ -29,14 +36,6 @@
 
 - Machine code Backends_
 
-- Current work and prototype in Prolog (!): see `this blog post`__.
-
-- `Draft notes`_ about the current work in PyPy (Feb 08).
-
-*(work and documentation in progress!)*
-
-.. __: http://morepypy.blogspot.com/2008/06/hi-all-some-news-from-jit-front.html
-
 ------------------------------------------------------------
 
 .. _Overview: overview.html

Modified: pypy/branch/pyjitpl5/pypy/doc/jit/overview.txt
==============================================================================
--- pypy/branch/pyjitpl5/pypy/doc/jit/overview.txt	(original)
+++ pypy/branch/pyjitpl5/pypy/doc/jit/overview.txt	Wed Mar 18 17:44:19 2009
@@ -23,9 +23,9 @@
 
 But coding in a high-level language has other benefits beyond the
 obvious ones.  Perhaps most importantly, it allows the language
-interpreter to be analyzed and turned into a compiler.  This is
-precisely what our JIT compiler generator does.  Based on partial
-evaluation techniques, **it can turn the interpreter of an arbitrary
+interpreter to be analyzed when turned into a compiler.  This is
+precisely what our JIT compiler generator does.  Based on tracing
+JIT techniques, **it can turn the interpreter of an arbitrary
 dynamic language into a just-in-time compiler for the same language.**
 It works mostly automatically and only needs guidance by the language
 implementor in the form of a small number of hints in the source code of
@@ -34,10 +34,11 @@
 machine code for the user program while aggressively optimizing it,
 leading to a big performance boost.
 
-Partial evaluation vs. manually-written JIT compilers
------------------------------------------------------
+The path we followed
+--------------------
 
-Partial evaluation is a well-known and much-researched topic, considered
+Our previous incarnations of PyPy's JIT generator were based on partial
+evaluation. This is a well-known and much-researched topic, considered
 to be very promising. There have been many attempts to use it to
 automatically transform an interpreter into a compiler. However, none of
 them have lead to substantial speedups for real-world languages. We
@@ -46,16 +47,22 @@
 compilers.  If this turns out to be correct, the practical speed of
 dynamic languages could be vastly improved.
 
-By comparison, getting a JIT compiler today involves either manually
-writing one, which is a huge amount of effort, or reusing an existing
-well-tuned JIT (like that of the JVM). However, the latter is hard
-because of concept mismatches between the implementor's language and the
-host VM language: the former needs to be compiled to the target
-environment in such a way that the JIT is able to speed it up
-significantly - an approach which essentially has failed in Python so
-far: even though CPython is a simple interpreter, its Java and .NET
-re-implementations are not faster (and cannot run all existing Python
-applications).
+Today (beginning 2009), our prototype is no longer using partial
+evaluation -- at least not in a way that would convince paper reviewers.
+It is instead based on the notion of *tracing JIT,* recently studied for
+Java and JavaScript.  When compared to all existing tracing JITs so far,
+however, partial evaluation gives us some extra techniques that we
+already had in our previous JIT generators, notably how to optimize
+structures by removing allocations.
+
+The closest comparison to our current JIT is Tamarin's TraceMonkey.
+However, this JIT compiler is written manually, which is quite some
+effort.  In PyPy, we write a JIT generator at the level of RPython,
+which means that our final JIT does not have to -- indeed, cannot -- be
+written to encode all the details of the full Python language.  These
+details are automatically supplied by the fact that we have an
+interpreter for full Python.
+
 
 Practical results
 -----------------
@@ -82,8 +89,10 @@
 
 * Fast enough: we think that we can get some rather good performance out
   of the generated JIT compilers.  That's the whole point, of course.
-  Previous experience shows that it should be possible (our generated
-  JIT compilers are similar to the hand-written Psyco).
+  Previous experience shows that it should be possible.  Our previous
+  generated JIT compilers were similar to the hand-written Psyco; due
+  to limits in automating the way Psyco works, our current generated
+  JIT compilers are instead similar to tracing JITs like TraceMonkey.
 
 
 Alternative approaches to improve speed
@@ -131,7 +140,7 @@
 encoding of language semantics - knowledge about Python behavior needs to be
 encoded by hand and kept up-to-date.  At least, Psyco works correctly even when
 encountering one of the numerous Python constructs it does not support, by
-falling back to CPython.  The PyPy JIT can be seen as a metaprogrammatic,
+falling back to CPython.  The PyPy JIT started out as a metaprogrammatic,
 non-language-specific equivalent of Psyco.
 
 A different kind of prior art are self-hosting JIT compilers such as Jikes.
@@ -142,6 +151,14 @@
 parts of the JIT compiler would need retargetting in order to run in a
 different environment than the intended low-level one.
 
+Simply reusing an existing well-tuned JIT like that of the JVM does not
+really work, because of concept mismatches between the implementor's
+language and the host VM language: the former needs to be compiled to
+the target environment in such a way that the JIT is able to speed it up
+significantly - an approach which essentially has failed in Python so
+far: even though CPython is a simple interpreter, its Java and .NET
+re-implementations are not significantly faster.
+
 More recently, several larger projects have started in the JIT area.  For
 instance, Sun Microsystems is investing in JRuby, which aims to use the Java
 Hotspot JIT to improve the performance of Ruby. However, this requires a lot of
@@ -158,16 +175,16 @@
 is contributing insights from our JIT research, in ways that will also
 benefit PyPy.
 
+Finally, tracing JITs are now emerging for dynamic languages like
+JavaScript with TraceMonkey.  The code generated by PyPy is very similar
+(but not hand-written) to the concepts of tracing JITs.
+
 
 Further reading
 ========================================================================
 
-The basic ideas have been shown to work in our prototype JIT-enabled
-``pypy-c``, as described in Status_.  Further technical documentation is
-in progress; see the Index_ page.  You can also read more about the
-`Theory of partial evaluation`_.
-
-.. _Index: index.html
-.. _Status: status.html
-.. _`Theory of partial evaluation`: theory.html#terminology
+The description of the current PyPy JIT generator is given in PyJitPl5_
+(draft).
+
 .. _`JSR 292`: http://jcp.org/en/jsr/detail?id=292
+.. _PyJitPl5: pyjitpl5.html

Modified: pypy/branch/pyjitpl5/pypy/doc/jit/pyjitpl5.txt
==============================================================================
--- pypy/branch/pyjitpl5/pypy/doc/jit/pyjitpl5.txt	(original)
+++ pypy/branch/pyjitpl5/pypy/doc/jit/pyjitpl5.txt	Wed Mar 18 17:44:19 2009
@@ -2,3 +2,10 @@
                               PyJitPl5
 ========================================================================
 
+Current information is available on blog posts (oldest first):
+
+* http://morepypy.blogspot.com/2008/06/hi-all-some-news-from-jit-front.html
+* http://morepypy.blogspot.com/2008/10/prolog-jit-masters-thesis-finished.html
+* http://morepypy.blogspot.com/2009/03/applying-tracing-jit-to-interpreter.html
+* http://morepypy.blogspot.com/2009/03/jit-bit-of-look-inside.html
+* http://morepypy.blogspot.com/2009/03/good-news-everyone.html

More information about the Pypy-commit mailing list