Debugging reason for python running unreasonably slow when adding numbers
Thomas Passin
list1 at tompassin.net
Wed Mar 15 10:58:13 EDT 2023
On 3/15/2023 10:24 AM, David Raymond wrote:
>> Or use the sum() builtin rather than reduce(), which was
>> *deliberately* removed from the builtins. The fact that you can get
>> sum() without importing, but have to go and reach for functools to get
>> reduce(), is a hint that you probably shouldn't use reduce when sum
>> will work.
>
> Out of curiosity I tried a couple variations and am a little confused by the results. Maybe I'm having a brain fart and am missing something obvious?
>
> Each of these was run with the same "data" and "perceptrons" values to keep that fair.
> Times are averages over 150 iterations like the original.
> The only thing changed in the trainPerceptron function was how to calculate sum_
>
>
> Original:
> sum_ = 0
> for key in input:
> v = weights[key]
> sum_ += v
> 418ms
>
> The reduce version:
> sum_ = reduce(lambda acc, key: acc + weights[key], input)
> 758ms
>
> Getting rid of the assignment to v in the original version:
> sum_ = 0
> for key in input:
> sum_ += weights[key]
> 380ms
>
> But then using sum seems to be slower
>
> sum with generator expression:
> sum_ = sum(weights[key] for key in input)
> 638ms
>
> sum with list comprehension:
> sum_ = sum([weights[key] for key in input])
> 496ms
>
> math.fsum with generator expression:
> sum_ = math.fsum(weights[key] for key in input)
> 618ms
>
> math.fsum with list comprehension:
> sum_ = math.fsum([weights[key] for key in input])
> 480ms
>
>
> I'm not quite sure why the built-in sum functions are slower than the for loop,
> or why they're slower with the generator expression than with the list comprehension.
I tried similar variations yesterday and got similar results. All the
sum() versions I tried were slower. Like you, I got the smallest times for
> for key in input:
> sum_ += weights[key]
but I didn't get as much of a difference as you did.
I surmise that in using the sum() variations, that the entire sequence
was constructed first, and then iterated over. In the non-sum()
versions, no new sequence had to be constructed first, so it would make
sense for the latter to be slower.
More information about the Python-list
mailing list