parallel processing in standard library
konrad.hinsen at laposte.net
Tue Jan 1 21:06:18 CET 2008
Emin.shopper Martinian.shopper wrote:
> Is there any hope of a parallel processing toolkit being
> incorporated into the python standard library? I've seen a wide
> variety of toolkits each with various features and limitations.
> Unfortunately, each has its own API. For coarse-grained
> parallelism, I suspect I'd be pretty happy with many of the
> existing toolkits, but if I'm going to pick one API to learn and
> program to, I'd rather pick one that I'm confident is going to be
> supported for a while.
I don't think that parallel computing is mature enough to allow the
standardization of APIs, except within a given and well specified
parallel computing model such as message passing.
The Python Wiki has an impressive list of parallel processing options
for Python (see http://wiki.python.org/moin/ParallelProcessing). With
the exception of the various MPI interfaces, I don't think that any
two of them are based on the same parallel computing model. I don't
expect this situation to change any time soon, as parallel computing
is still very much experimental. Whereas sequential computing has
well-tested software engineering techniques, reliable libraries that
can be combined into programs, and ever improving testing techniques,
none of these exist for parallel computing.
For an overview of parallel computing models and for a more detailed
description of one of them as implemented in Python, please see my
recent article in "Computing in Science and Engineering":
More information about the Python-list