suitability of python

[Dave Angel]

On 11/24/2011 07:31 AM, Rudra Banerjee wrote:
> Dear friends,
> I am a newbie in python and basically i use python for postprocessing
> like plotting, data manipulation etc.
> Based on ease of programming on python I am wondering if I can consider
> it for the main development as well. My jobs (written on fortran) runs
> for weeks and quite CPU intensive. How python works on these type of
> heavy computation?
> Any comment or reference is welcome.
>
If I take your description at face value, then I'd say that stock 
CPython would be slower than Fortran.  If the CPU-intensive parts had to 
be rewritten in CPython, they'd be slower than the Fortran they replace, 
by somewhere between 10:1 and 500:1.  Further, if you've already got 
those Fortran algorithms written and debugged, why rewrite them?  And 
finally, even for new code, you might be getting ideas for your 
algorithms from journals and other resources, where the examples may 
well be done in Fortran, so productivity might be best in Fortran as well.

HOWEVER, you don't have to use stock CPython, alone.  It could be that 
some of your Fortran algorithms are written in shared libraries, and 
that you could get your CPython code to call them to do the "heavy 
lifting."  Or it could be that numpy, sage, or other 3rd party libraries 
might be usable for your particular problems, and that speed is then 
comparable to Fortran.  Or it could be that one of the alternative 
Python implementations might be fast enough.

Or it could even be that you're mistaken that the present code is even 
CPU intensive.

Or it could be that by the time you recode the problem in Python, you 
discover a more efficient algorithm, and that way gain back all the 
speed you theoretically lost.

There are tools to measure things, though I'm not the one to recommend 
specifics.  And those probably depend on your platform as well.

The last Fortran that I wrote was over 40 years ago.  I'm afraid when I 
need speed, I usually use C++.  But if I were starting a personal 
math-intensive project now, I'd try to prototype it in Python, and only 
move portions of it to Fortran or other compiled language.  Only the 
portions that measurably took too long.  And those portions might be 
rewritten in Cython, C++, or Fortran, depending on what kind of work 
they actually did.

Another alternative that might make sense is to use Python as a "macro 
language" to Fortran, where you call out to Python to automate some 
tasks within the main program.  I have no experience with doing that, 
but I assume it'd be something like how MSWord can call out to VBA 
routines.  And it'd make the most sense when the main app is already 
written, and the macro stuff is an afterthought.

I think the real point is that it doesn't have to be "all or nothing."  
I suspect that the pieces you're already doing in Python are calling out 
to pre-existing libraries as well.  So your plotting code does some 
massaging, and then calls into some plotting library, or even execs a 
plotting executable.

-- 

DaveA