Parallelization in Python 2.6

[Stefan Behnel]

Robert Dailey wrote:
> I'm looking for a way to parallelize my python script without using
> typical threading primitives. For example, C++ has pthreads and TBB to
> break things into "tasks". I would like to see something like this for
> python. So, if I have a very linear script:
> 
> doStuff1()
> doStuff2()
> 
> 
> I can parallelize it easily like so:
> 
> create_task( doStuff1 )
> create_task( doStuff2 )
> 
> Both of these functions would be called from new threads, and once
> execution ends the threads would die. I realize this is a simple
> example and I could create my own classes for this functionality, but
> I do not want to bother if a solution already exists.

I think the canonical answer is to use the threading module or (preferably)
the multiprocessing module, which is new in Py2.6.

http://docs.python.org/library/threading.html
http://docs.python.org/library/multiprocessing.html

Both share a (mostly) common interface and are simple enough to use. They
are pretty close to the above interface already.

Stefan