[Python-Dev] Re: native code compiler? (or, OCaml vs. Python)

Greg Ewing greg@cosc.canterbury.ac.nz
Fri, 31 Jan 2003 14:37:16 +1300 (NZDT)


Graham Guttocks:

> Even on this list, whose users are presumably well clued-in 
> about Python, I've heard:
> 
>   ``Nobody cared enough to do it.''

Almost always, only a small part of any program is computationally
intensive. Many people have had great success in such cases by coding
that small part in C, making an extension module out of it, and using
Python for all the rest.

This technique has proved suffiently successful that, so far, nobody
has had both enough motivation and enough ability to attempt the
creation of an optimising compiler for pure Python. And yes, it *is* a
very difficult problem, due to the complete absence of explicit type
information, and the extremely dynamic nature of Python's semantics.

In summary, ``Nobody cared enough to do it.'' But there are good
*reasons* for that.

>  ``you're welcome to volunteer.''

I suppose that may have sounded like a brush-off, but it wasn't mean
that way. If you have the necessary knowledge and ability to attempt
anything in this area, any contribution you could make really would be
welcome.

If not, well, you'll just have to take our word for it that what
you're asking for is very non-trivial. If it weren't, someone would
probably have done it by now.

> However, head-way (Psycho) has been made in a very short
> period of time, so it hardly seems impossible given enough time
> and effort.

As I understand it, Psyco isn't a compiler in the conventional sense
of something which statically analyses the program and generates code
ahead of time. Rather, it notices at run time that certain functions
tend to be called with certain types as parameters, and compiles
specialised versions of the function for those types. A sort of
"empirical type analysis".

Psyco is still in the early stages. If it proves successful enough,
maybe it or something like it will become a part of the standard
Python implementation. Or maybe not -- the current Python
implementation is exceptionally straightforward and easy to
understand, and Guido may want to keep it that way.

By the way, you might also like to take a look at Pyrex:

   http://www.cosc.canterbury.ac.nz/~greg/python/Pyrex/

which is my attempt at making it easier to bridge the gap between
Python and other languages. As a side effect, it can be used to
"compile" Python code, although the speed gain that way is small (only
about 2 or 3 times at best).  It's main purpose is for interfacing
with C code, which is where the real speed gains are to be had.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+