[pypy-dev] PyPy JVM Backend

Antonio Cuni anto.cuni at gmail.com
Sat Apr 28 14:02:36 CEST 2007


Hi Niko, hi Paul!
Here are some remarks/suggestions about your discussion.
I'm also CC-ing pypy-dev, so we can also catch someone else's
suggestions, if any :-).

> First, there are those things are absolutely necessary to do to get PyPy 
> to translate:
> 
> 1. weak refs
> 2. I don't know what else goes here --- try and see I guess

Apart from weak refs, I think the other big feature missing is support
for external functions (mostly I/O functions).
Unfortunately this part is a bit of a mess: since in the old days pypy's
only targets were low-level platforms the I/O model of RPython is
modeled after the unix file descriptors. This means that at the moment
high level backends need to emulate the fd interface and forward the
real work to the native I/O functions: to see how gencli does, look at
the ll_os class in translator/cli/src/ll_os.cs. I guess this would be
the easiest way also for genjvm, and probably for now we should pick
this solution.

I know, this it's both ugly and hackish; the good news is that I and
Carl did some work towards a better solution: at the moment all the I/O
inside the standard interpreter is done using the (interp-level)
rlib/streamio.py library, which in turns uses the low-level file
descriptor interface; the long-term solution would be to provide an
alternative implementation of streamio.py that uses .NET/Java streams
and let the backends using that instead of the current one.

Finally, let me add another task that I think it's very important:

3. make most of existing tests passing with genjvm

Look at cli/test directory: you will find a lot of files that import
tests from somewhere else (mostly from rpython/test). genjvm do the
same, but a lot of tests are still missing; once you get all those tests
passing you can be quite confident (but still not sure, of course) that
pypy would translate.

Some of the tests in cli/tests are cli-specific or very old and not
conforming to the new test framework; here is a list of the tests you
should port these tests to jvm:

   - test_range.py
   - test_constant.py
   - test_tuple.py
   - test_float.py
   - test_builtin.py
   - test_int.py
   - test_exception.py

Moreover, there is test_runtest, which test a bit of the gencli test
framework itself; if the genjvm test framework is similar to the
gencli's one, it could make sense to port this also this one. Niko, what
do you think?


> New features:
> 
> 1. allow Java code to be invoked (need to look at the CLI work here)
> 2. JSR 223

I have to admit that I don't know very much about JSR-223: Paul, could
you send me the specification, please? I don't feel like to register to
sun's website :-).

If I understand correctly the level of integration provided by JSR-223
is simpler/less powerful than the one offered by Jython, right? If this
is the case, it could make sense to implement JSR-223 first (also
because it's part of Paul's SoC proposal), then try to provide the same
level of integration as Jython (or IronPython for the .NET side); for
the latter task, I think that some code could be shared between gencli
and genjvm.

> Internal Changes That "Would Be Nice":
> 
> 1. Estimate the maximum size of the stack, rather than giving some 
> arbitrary "high" number as we do know.

this would also be useful for gencli :-)

> 2. Deal with JVM size limits for very large methods and classes. (never 
> hit those yet but probably will)

probably this falls into the "try to translate pypy-jvm and see" task :-).

> 3. Find a better way to handle exceptions.  One suggestion was not to 
> map rpython's "Object" to the class Object but rather to an interface.

Indeed. Also useful for gencli.

> 4. Use protected access modifiers rather than public.  Pedronis seems to 
> think this would be important.

I don't see how it could be so important at this point. Am I missing
something?

> 5. Speed and performance optimizations, similar to those being done for C#

let me add another point:

6. once we get pypy-jvm, run Python's regression tests and try to pass
as much of them as possible.

I personally think that point 6 is more important than point 5 at the
beginning.

> Of these ideas, I am not sure which are best for a "first project".  I 
> would be happy to handle weakrefs, since it seems like they may be quite 
> easy for me but time consuming for you, but if you wanted to look into 
> them, that's fine with me too.

Adding support for weakrefs to gencli was a very easy task: just
wrap/unwrap the objects into the System.WeakReference when requested.

> Antonio, you said you tried to translate PyPy as a whole with the JVM 
> backend?  It seems that is the best way to find the list of things that 
> need doing, but I haven't done it myself yet.

Yes, I tried: after a bit of hacking on the interpreter code I could get
rid of weakrefs, but the translations failed on a KeyError inside
jvm/database.py, then I gave up.

To sum up, I think that priority wise the most important thing to do is 
to port the remaining tests to genjvm and make them passing: weakrefs 
and external functions are subtasks of this; then try-fail-fix-try until 
we get a running pypy-jvm.

ciao Anto



More information about the Pypy-dev mailing list