Parallelization with Python: which, where, how?
fredrik at pythonware.com
Mon Dec 20 14:33:34 CET 2004
"Mathias" <no_sp at m_please.cc> wrote:
> I have a (pretty much) "emberassingly parallel" problem and look for the right toolbox to
> parallelize it over a cluster of homogenous linux workstations. I don't need automatic
> loop-parallelization or the like since I prefer to prepare the work packets "by hand".
> I simply need
> - to specify a list of clients
> - a means of sending a work packet to a free client and receiving the
> result (hopefully automatically without need to login to each one)
> - optionally a timeout mechanism if a client doesn't respond
> - optionally help for debugging of remote clients
> So far I've seen scipy's COW (cluster of workstation) package, but couldn't find documentation or
> even examples for it (and the small example in the code crashes...).
> I've noticed PYRO as well, but didn't look too far yet.
> Can someone recommend a parallelization approach? Are there examples or documentation? Has someone
> got experience with stability and efficiency?
googling for "parallel python" brings up lots of references; tools like
(see https://geodoc.uchicago.edu/climatewiki/DiscussPythonMPI for
seem to be commonly used.
More information about the Python-list