[Tutor] Compiling Python to Native Machine Code

[Marc Barry]

Thanks for the info Danny.

First off, I thought that I should download Psyco and install it to see what 
kind of performance gains could be realised. After running a number of 
simulations, it looks like I can save about 10% on the execution time. Well 
this is substantial, I had hoped for more.

I think that I may have to recode some of the performance critical code in 
another language as you mentioned.

The type of algorithm I am using is the well known Dijkstra shortest path. I 
am computing paths in a graph many thousands or even millions of times. To 
make thinks worse, some of my graphs are very large with 74 - 100 vertices 
and 230 - 400 bi-directional edges. At each point that a shortest path in 
the graph is found, the edge weights must be updated (thus changing the 
graph) and therefore I cannot take any advantage of storing paths that have 
already been calculated. I have implemented the algorithm as efficient as I 
know how.

I may have to look at implementing the Dijkstra algorithm portion of the 
simulation in C. Although, it would be a terrible waste of time if I didn't 
do any better than the 10% reduction I got using Psyco. I thought that there 
would be more improvement since basically I run the Dijkstra algorithm many 
times over and over thus making it a prime candidate for compiling to native 
machine code.

Thank you for the help.

Cheers,

Marc





From: Danny Yoo <dyoo at hkn.eecs.berkeley.edu>
To: Marc Barry <marc_barry at hotmail.com>
CC: tutor at python.org
Subject: Re: [Tutor] Compiling Python to Native Machine Code
Date: Tue, 2 Sep 2003 03:36:50 -0700 (PDT)



On Tue, 2 Sep 2003, Marc Barry wrote:

 > Is it possible to compile Python directly to native machine code? The
 > reason I ask is because I am using Python to do some pretty intensive
 > simulations that take anywhere between 10 minutes and a few hours to
 > run. I assume that if I could compile it directly to machine code that
 > this may speed up my simulation times.

Hi Mark,

At the moment, no.  You may be able to get some substantial performance by
using Psyco, though:

     http://psyco.sourceforge.net/

If your code is particularly algorithmic in nature, Psyco might do wonders
for it.


If you can identify which parts of code are performance hogs, we have a
few options.  For example, it might be possible to recode those
performance-critical pieces in another language --- like C or OCaml ---
and bind those recoded pieces to Python.

If you want to take this route, there's good prior work that's gone into
this.  *grin* People have written several popular tools to help do this
"extending"; you may want to look into Scientific Python's "weave" module:

    http://www.scipy.org/

as well as SWIG:

     http://www.swig.org/



 > Does anyone have any experience with doing this? Would there be any
 > significant gains?

It really depends on the algorithms that you are using.  Do you mind
showing us a sample of the kind of calculations you're doing?  And do you
have a "profile" of your application?  Profiles can help pinpoint busy
areas of the code.  Here's documentation to Python's profiler:

     http://www.python.org/doc/lib/profile.html


Good luck to you!

_________________________________________________________________
MSN 8 with e-mail virus protection service: 2 months FREE*  
http://join.msn.com/?page=features/virus