multiprocessing newbie - semaphore question

xera121 lolmcbride at googlemail.com
Sat Oct 3 01:06:39 CEST 2009


Hi,
I'm interested in this type of programming but need some fundamental
questions answered if someone could oblige:

In the docs for the mutiprocessing module it mentions right at the
beginning about an underlying semaphore system - would MPI2 fit the
bill running on a Beowulf cluster?

Is the mutiprocessing module usable for writing apps to run on a
Beowulf cluster - i.e. can I use it for an app that will share the
work amongst the nodes?

I am looking for a way to share the processing on a task which
requires intensive and repetitive work done over a large dataset in a
sqlite table and I wondered if this would be a way to share the load
over a number of pcs I have available for this task.

My other idea is a simple(ish) client/server model which I am pretty
confident I can execute whereas the setup of the cluster and the
writing of a multiprocessing app would require a lot of prep and study/
testing.

Therefore the bottom line is is it worth it? I know I haven't been too
specific of what I'm doing but I just wanted a quick feel for the task
from those who have more knowledge of the subject - Oh and btw I am
NOT an expert programmer by any means and would expect almost all my
code to be deemed as amateurish - but it keeps me happy and only
breaks every other iteration ;-)

xera121



More information about the Python-list mailing list