Compiling Python

[Edward K. Ream]

Jerzy Karczmarczuk wrote:

> > My question is this; I realize that Python compiles to bytecode automatically
> > as part of the interpreters tokenization run, but is there a way to permanently
> > compile python code outputting to an executable binary?
> 
> Once upon a time Guido Van Rossum thought "this seems very difficult,
> perhaps impossible".
> 
> He changed his mind a little bit now...
> 
> See:
> 
> http://www.onlamp.com/pub/a/python/2002/06/04/guido.html
> 
> and the references cited by The Master:
> 
> http://www.cosc.canterbury.ac.nz/~greg/python/Pyrex/
> http://homepages.ulb.ac.be/~arigo/psyco/

Thanks for these links.

Earlier this year I looked into this question in some depth and
concluded:

a) compiling Python to, say, C or C++ is in fact doable much of the
time,
b) the hard issues are more related to library compatibility (what is a
compiled object?)
c) such a project probably would be reasonable only if part of
"official" Python.

As far as a) goes, it should be possible to "propagate" type information
from constants and arguments and return values from known library
routines, so that in fact a lot more type information may be present
than one might think.  I saw this first this way: there is an
isomorphism between the C++ and Python versions of Leo, so it ought to
be possible to back translate from leo.py to leo.cpp.  The more I looked
at this, the more plausible it became.  However, because of b), it is
not clear what kind of performance improvements can be expected.  Point
b) also implies a close connection between compiler, interpreter and
libraries, which is why I say c).

I offer the following existence proof for the possibility of compiling
Python.  One could simply "unwrap" the Python interpreter.  That is, we
could replace byte codes with the equivalent portions of the
interpreter's main loop.  Obviously, nothing practical is gained by
this.  However, it does provide a mental picture of what is going on,
and it hints at what might need to be optimized in order to make a
compiler practical.

I found this preliminary design work to be great fun, and I am excited
that Guido has shown some interest in this area.  I wonder whether there
would be any interest in a sig related to this subject?  I have lots of
experience and ideas on this subject and I would love to toss ideas
around.

BTW, a compiled version of a Python program would be valuable
commercially, as it would allow companies to protect their source code,
as well as (possibly) improve performance.

If anyone knows of any reasons why a Python to C++ compiler clearly can
not be made to work, I would be very interested in hearing them.

Edward
--------------------------------------------------------------------
Edward K. Ream   email:  edream at tds.net
Leo: Literate Editor with Outlines
Leo: http://personalpages.tds.net/~edream/front.html
--------------------------------------------------------------------