[Python-Dev] Re: native code compiler? (or, OCaml vs. Python)

Greg Ward gward@python.net
Tue, 4 Feb 2003 08:58:05 -0500


On 31 January 2003, Graham Guttocks said:
> It would be nice to use Python for more serious projects, but it isn't
> fast enough currently.

The usual solution is to take the bits that are too slow in Python and
recode them in C.  This may not be as far-sighted as a spiffy
native-code Python compiler, but it's a hell of a lot less work for each
individual project.

A good middle-ground is to figure out where Python is slow and make it
faster.  Eg. I recently discovered that certain operations in ZODB --
specifically, converting back and forth between 8-byte strings and
64-bit integers -- are a significant bottleneck.  Those operations are
currently implemented as tiny Python functions that use the struct
module to do most of their work.  Through discussion on the
zodb-dev@zope.org list, I learned that the struct module has never been
heavily optimized (aka "passed through the timbot").  I chose to
reimplement those ZODB operations in C, because I wanted maximum bang
for my (employer's) buck.  But a more far-sighted approach --
ie. something that would benefit more projects for a longer time --
would be to spend some time tightening up the struct module.

In all that time, it never once occurred to me that compiling ZODB to
machine code was the answer.  Dynamic typing is boatloads of fun, but it
can really kill performance to do all that work at runtime.  Doesn't
matter if your runtime is Python bytecode interpreted by a C program or
just plain machine code.  Unless, of course, you have a
super-sophisticated compiler that does ML-style type inference ... but
without ML's bondage and discipline.

I suspect that compiling Python properly (ie. with real performance
gains) would be roughly as difficult as compiling ML -- and, because
Python always gives you a way to sneak around whatever informal type
safety mechanisms you might use, you wouldn't always benefit from type
inference.

        Greg
-- 
Greg Ward <gward@python.net>                         http://www.gerg.ca/
No animals were harmed in transmitting this message.