[Python-ideas] Fast sum summary [was Re: Fast sum() for non-numbers - why so much worries?]

[MRAB]

On 12/07/2013 01:52, Steven D'Aprano wrote:
> This started out as a response to Ronald, but turned into a summary
> and pseudo-code implementation, so I've changed the subject line.
>
>
> On 11/07/13 21:15, Ronald Oussoren wrote: [...]
>> That, and Paul's example about using += for something else than
>> addition, pretty much kills the idea of using += in the
>> implementation of sum() as that would break to much code.
>
> Not quite :-)
>
> After spending so much time explaining why I don't think sum *should*
> optimize the list case, I'm now going to suggest a way which (I
> think) sum *could* optimize the list case without changing
> behaviour.
>
> [puts on Devil's Advocate cap]
>
> What we have to give up is Sergey's hope to make sum fast for
> "everything". I don't think that is possible, but even if it is, we
> should not let the Perfect be the enemy of the Good. Can we make sum
> fast for lists, without changing behaviour? I think we can.
>
> We know that list.__iadd__ is a fast, inplace version of __add__. We
> can't make that assumption about every class, not even list
> subclasses, but we don't have to speed up every class. Let's not be
> greedy: incremental improvements are better than no improvements.
>
> sum already has special cases for ints and floats and strings (the
> string case is to just prohibit the use of sum). We could add a
> special case for lists: if the start parameter is an actual built-in
> list, not a subclass (because subclasses might override __iadd__)
> then we start with an accumulator list, and call __iadd__ (or extend)
> on that list to concatenate on it. As soon as we find an item which
> is not a built-in list, we drop out of the fast-code and fall back to
> the normal non-fast path. This doesn't make it "fast for everything",
> but it's still an optimization.
>
[snip]
While you have your cap on, if you're going to special-case lists, then
why not strings too (just passing them on to "".join())?