Do you have the code posted someplace for this? I'd like to add it into the tests I am running On May 15, 2008, at 11:56 AM, Tom Pinckney <thomaspinckney3@gmail.com> wrote:
All the discussion recently about pyprocessing got me interested in actually benchmarking Python's multiprocessing performance to see if reality matched my expectations around what would scale up and what would not. I knew Python threads wouldn't be good for compute bound problems, but I was curious to see how well they worked for i/o bound problems. The short answer is that for i/o bound problems, python threads worked just as well as using multiple operating system processes.
I wrote two simple benchmarks, one compute bound and the other i/o bound. The compute bound one did a parallel matrix multiply and the i/o bound one read random records from a remote MySQL database. I ran each benchmark via python's thread module and via MPI (using mpi4py and openmpi and Send()/Recv() for communication). Each test was run multiple times and the numbers were consistent between test runs. I ran the tests on a dual-core Macbook Pro running OS X 10.5 and the included python 2.5.1.
1) Python threads
a) compute bound:
1 thread - 16 seconds 2 threads - 21 seconds
b) i/o bound:
1 thread -- 13 seconds 4 threads -- 10 seconds 8 threads -- 5 seconds 12 threads - 4 seconds
2) MPI
a) compute bound:
1 thread - 17 seconds 2 threads -- 11 seconds
b) i/o bound
1 thread -- 13 seconds 4 threads -- 10 seconds 8 threads -- 6 seconds 12 threads -- 4 seconds _______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/jnoller%40gmail.com