[Python-ideas] Intermediate Summary: Fast sum() for non-numbers

[Joshua Landau]

On 16 July 2013 04:22, David Mertz <mertz at gnosis.cx> wrote:
> On Mon, Jul 15, 2013 at 7:12 PM, Ron Adam <ron3200 at gmail.com> wrote:
>>
>> >      + 1   Add specialised sum_iters(),  (based on the sum()
>>
>> You mean chain.from_iterable?
>>
>> No, I mean a new function written in C, which writes (appends) the values
>> directly into a new (or start) sequence.  Chain from_iterable builds a
>> generator and yields the items out.  That's not going to be as fast, but it
>> does use much less memory in many situations.
>
>
> If a new function could *actually* be significantly faster than
> chain.from_iterable(), I think it would be reasonable to have.  However, if
> writing something new as basically an alias for 'list(chain(...))' only gets
> us, say 10% speedup, I think nothing should be included.

I'm not convinced. I have three reasons, and I have full faith in all three.

1) Ignoring speed, I don't believe there are *any* use-cases where
concat(...) *isn't worse* than list(chain.from_iterable(...)),
excluding fabricated ones that have never happened.

2) I'm not convinced this is a bottleneck for a significant number of
people; chain is much faster than most constructs we have already, so
it'd be off for the chaining to be the slowest part

3) This belongs on PyPi, not stdlib, as it is niche and we can already
do the same thing with the same asymptotic performance. Blist gives
*way* more speed advantages yet that's been rejected from stdlib so I
disagree that this can cross the barrier. I realise blist was rejected
for the additional reason of trying to replace our normal lists, but
it could've gotten into stdlib so that doesn't discount the analogy.

> But PLEASE, don't call such a new function sum_iters().  The obviously
> correct name for such a thing is 'concat()'.  This is what I've argued a
> bunch of times, but let's just call concatenation by its actual name rather
> than try to squint in just the right way to convince ourselves that
> "summation" is the same concept.

I agree, because a summation that only works on lists is really just
concat after-all.