Parallelization with Python: which, where, how?
no_sp at m_please.cc
Mon Dec 20 14:03:09 CET 2004
I have a (pretty much) "emberassingly parallel" problem and look for the
right toolbox to parallelize it over a cluster of homogenous linux
workstations. I don't need automatic loop-parallelization or the like
since I prefer to prepare the work packets "by hand".
I simply need
- to specify a list of clients
- a means of sending a work packet to a free client and receiving the
result (hopefully automatically without need to login to each one)
- optionally a timeout mechanism if a client doesn't respond
- optionally help for debugging of remote clients
So far I've seen scipy's COW (cluster of workstation) package, but
couldn't find documentation or even examples for it (and the small
example in the code crashes...).
I've noticed PYRO as well, but didn't look too far yet.
Can someone recommend a parallelization approach? Are there examples or
documentation? Has someone got experience with stability and efficiency?
Thanks a lot,
More information about the Python-list