Adding a Par construct to Python?
MRAB
google at mrabarnett.plus.com
Sun May 17 20:35:01 CEST 2009
Steven D'Aprano wrote:
> On Sun, 17 May 2009 17:19:15 +0100, MRAB wrote:
>
>>> But reduce()? I can't see how you can parallelize reduce(). By its
>>> nature, it has to run sequentially: it can't operate on the nth item
>>> until it is operated on the (n-1)th item.
>>>
>> It can calculate the items in parallel,
>
> I don't understand what calculation you are talking about. Let's take a
> simple example:
>
> reduce(operator.sub, [100, 50, 25, 5]) => 100-50-25-5 = 20
>
> What calculations do you expect to do in parallel?
>
>
>> but the final result must be
>> calculated sequence, although if the final operation is commutative then
>> some of them could be done in parallel.
>
> But reduce() can't tell whether the function being applied is commutative
> or not. I suppose it could special-case a handful of special cases (e.g.
> operator.add for int arguments -- but not floats!) or take a caller-
> supplied argument that tells it whether the function is commutative or
> not. But in general, you can't assume the function being applied is
> commutative or associative, so unless you're willing to accept undefined
> behaviour, I don't see any practical way of parallelizing reduce().
>
I meant associative not commutative.
I was thinking about calculating the sum of a list of expressions, where
the expressions could be calculated in parallel.
More information about the Python-list
mailing list