Parallelization in Python 2.6

[Stefan Behnel]

Dennis Lee Bieber wrote:
> On Tue, 18 Aug 2009 13:45:38 -0700 (PDT), Robert Dailey wrote:
>> Really, all I'm trying to do is the most trivial type of
>> parallelization. Take two functions, execute them in parallel. This
>> type of parallelization is called "embarrassingly parallel", and is
>> the simplest form. There are no dependencies between the two
>> functions. They do requires read-only access to shared data, though.
>> And if they are being spawned as sub-processes this could cause
>> problems, unless the multiprocess module creates pipelines or other
>> means to handle this situation.

It wouldn't be much worth if it didn't, as the subprocess module handles
everything else nicely. See the Queue classes.


> 	If they are number crunchers (CPU-bound) and don't make use of
> binary extension libraries that release the GIL (for the most common
> Python implementation), they'll run faster being called in sequence
> since you won't have the overhead of task switching.

... unless, obviously, the hardware is somewhat up to date (which is not
that uncommon for number crunching environments) and can execute more than
one thing at once.

Stefan