Using Python for processing of large datasets (convincing managment)

Paul Rubin phr-n2002b at NOSPAMnightsong.com
Sun Jul 7 00:39:37 EDT 2002


Depending on what kind of computation you're doing on that dataset,
Python may not give you the speed you need.  It has big advantages over
C/C++ for development speed and program maintainability, but the ugly
truth is that the interpreter is pretty slow.

You may be best off with a hybrid approach, writing the control
structure of your program in Python but the inner computational loops
in C, called through the Python-to-C API or through SWIG.  These have
a bit more learning curve than programming in pure Python, but will
let you get at the performance of C code when you need it.



More information about the Python-list mailing list