[pypy-svn] r51876 - pypy/extradoc/proposal

[antocuni at codespeak.net]

Author: antocuni
Date: Tue Feb 26 20:30:48 2008
New Revision: 51876

Modified:
   pypy/extradoc/proposal/openjdk-challenge.txt
Log:
this completes the first draft of the proposal



Modified: pypy/extradoc/proposal/openjdk-challenge.txt
==============================================================================
--- pypy/extradoc/proposal/openjdk-challenge.txt	(original)
+++ pypy/extradoc/proposal/openjdk-challenge.txt	Tue Feb 26 20:30:48 2008
@@ -44,15 +44,140 @@
 
 The goal of this proposal is to extend the PyPy JIT to work in
 conjunction with the JVM backend.  After the work has been completed,
-it will be possibile to translate the interpreter into a Python
+it will be possible to translate the interpreter into a Python
 implementation that runs on top of the JVM and contains a JIT; the JIT
 will dynamically translate part of Python programs into JVM bytecode,
 which will then be executed by the underlying virtual machine.
 
+XXX: should we explain in more detail what we are doing?
 
 
+Porting the JIT to the MLVM
+---------------------------
 
-.. _PyPy: http://codespeak.net/pypy
-.. _RPython: http://codespeak.net/pypy/dist/pypy/doc/coding-guide.html#rpython 
+As stated above, PyPy JIT for JVM would work by dynamically emitting
+and loading JVM bytecode at runtime.  Even if this approach has been
+tried in a couple of projects (XXX: link to JRuby for example), it has
+to been said that the JVM was not originally designed for such
+applications; for example, the process of loading a single method is
+very expensive, since it involves the creation and loading of a
+sorrounding class.
+
+The new Da Vinci Machine contains a lot of interesting features that
+could be effectively exploited by the PyPy JIT to produce an even more
+efficient implementation of the Python language, as `John Rose said`_
+after the talk with PyPy people.
+
+Features of the MLVM that could be exploited by PyPy JIT include but
+are not limited to: dynamic invocation, lightweight bytecode loading,
+tail calls, etc.
+
+Implementation wise, the JIT backends for the plain JVM and for the
+MLVM could share most of the code, with the latter making use of the
+special features when needed.
+
+Moreover, the experience of this project will help the MLVM team to
+understand which features are really useful to implement dynamic
+languages on top of the JVM and which one we still lack.
+
+Deliverables
+------------
+
+Due to the its strict dependency on PyPy, it will not possible to
+release the result of the work as a separate and independent project.
+In particular, to reach the goals of the proposal it will be necessary
+to extensively modify parts of PyPy that are already there, as well as
+write completely new code.
+
+If the project goes to completion, the code developed will be
+integrated into the PyPy codebase; if Sun requires to release the code
+under the SCA (thus sharing the copyright between the original author
+and Sun itself), we will send to Sun a document in unified diff format
+that extensively shows all and sole lines of code on which Sun will
+have the copyright.
+
+
+Project completion
+------------------
+
+PyPy JIT is still under heavy development; potentially, the resulting
+JIT compiler will be able to optimize a large number of Python
+programs, but at the moment it gives the best results only with
+computational intensive functions that use only operations between
+integers.
+
+We expect to get a pypy-jvm executable that can execute a function
+with those characteristics at roughly the same speed as its equivalent
+written in Java, excluding the costs of the JIT compilation itself,
+which have not been optimized yet.
+
+For an example of a function with is highly optimized by the PyPy JIT,
+look at the `function f1`_: when executed by a pypy-c compiled with
+JIT support, it runs roughly at the same speed as its C equivalent
+compiled with `gcc -O1`. (XXX: is there a link somewhere?)
+
+We also expect to find benchmarks in which the JIT that targets the
+MLVM will perform better thant the JIT that targets the plain JVM,
+though it is hard to speficy a precise commitment here without knowing
+which features of the MLVM will be possible to use.
+
+
+Relevance to the community
+--------------------------
+
+Recently the community showed a lot of interests in dynamic languages
+on top of the JVM.  Even if right now Jython is the only usable
+implementation of Python for the JVM, PyPy can potentially become the
+reference implementation in the future.  
+
+To have a working JIT for the JVM is an important step to make PyPy
+the fastest Python for the JVM ever.  Moreover, due to the innovative
+ideas implemented by PyPy, there are chances that Python could become
+the fastest dynamic language that run on the top of the JVM.
+
+Finally, PyPy is not limited to Python: it is entirely possible to
+write interpreters for languages other than Python and translate them
+with the TT; as a proof of concept, PyPy already contains partial
+implementations of Prolog, JavaScript, Scheme and Smalltalk.
+
+Since the JIT generator is independent of the Python languages, it
+will be possible to automatically add a JIT compiler to every language
+written using the PyPy TT; thus, PyPy could become a very attractive
+platform to develop dynamic languages for the JVM.
+
+
+Dependencies on Sun
+-------------------
+
+There are no dependencies on Sun regarding the implementation of a JIT
+compiler that targets the plain JVM.  However, in order to implement a
+JIT compiler that targets the new MLVM, we need the new features we
+want to exploit to be implemented.
+
+Related work
+------------
+
+Dynamic generation of bytecode for object oriented virtual machine is
+a hot topic:
+
+  - `this paper`_ shows how this technique is exploited to write an
+    efficient implementation of EcmaScript which runs on top of the JVM;
+
+  - JRuby also come with a JIT compiler that dynamically translates
+    ruby code into JVM bytecode; the difference with the PyPy approach
+    is that JRuby doesn't exploit type information that are known only
+    at runtime to produce specialized, efficient versions of the
+    function; moreover, while the JRuby JIT is hand-written, the whole
+    goal of the PyPy JIT generator is to generate it automatically
+    from the intepreter;
+
+  - in the .NET world, IronPython also emit code dynamically to
+    optimize hot spots, but not in a pervasive way as JRuby or PyPy.
 
 
+
+.. _PyPy: http://codespeak.net/pypy
+.. _RPython: http://codespeak.net/pypy/dist/pypy/doc/coding-guide.html#rpython 
+.. _`John Rose said`: http://blogs.sun.com/jrose/entry/a_day_with_pypy
+.. _`function f1`: http://codespeak.net/svn/pypy/dist/demo/jit/f1.py
+.. _`this paper`: http://www.ics.uci.edu/~franz/Site/pubs-pdf/ICS-TR-07-10.pdf