Threading, multi-processors, and Numeric Python
I've written Numeric Python code (with Python 1.5.2) to analyze weather radar data. In an attempt to speed up this code, I used threads to perform some of the computations. I'm running on a dual processor Linux machine running 2.4.1 with SMP enabled. I'm using Numeric -17.3.0 with Python 1.5.2 When I run the threaded code, and monitor the system with 'top', 1 processor spends much of its time idle, and I rarely see two copies of my 'compute' thread executing. Each thread is computing its results from different arrays. However, all arrays are referenced from the same dictionary. Any ideas on how to get both threads computing at the same time? Thanks for your help! -- Joe VanAndel National Center for Atmospheric Research http://www.atd.ucar.edu/~vanandel/ Internet: vanandel@ucar.edu
OK, I guess I've just discovered the answer to my own question. [Don't you just hate that! :-) ] My C++ extensions that perform the real calculations needed to be modified to support multiple threads. By adding the Py_BEGIN_ALLOW_THREADS/Py_END_ALLOW_THREADS macros, the Python interpreter knew it was safe to allow other threads to execute during computations, just like it allows other threads to execute during I/O. To accomodate the Python global thread lock, I needed to change my code as follows: my_func() { /* python calls */ Py_BEGIN_ALLOW_THREADS /* computations that don't use python API */ Py_END_ALLOW_THREADS /* python API calls */ } Now, I can see both processors being used. -- Joe VanAndel National Center for Atmospheric Research http://www.atd.ucar.edu/~vanandel/ Internet: vanandel@ucar.edu
participants (1)
-
Joe Van Andel