[Python-ideas] Fast sum() for non-numbers

[Joshua Landau]

On 4 July 2013 04:35, Steven D'Aprano <steve at pearwood.info> wrote:
> On 04/07/13 08:02, Joshua Landau wrote:
>> On 3 July 2013 18:58, Steven D'Aprano <steve at pearwood.info> wrote:
>>> On 04/07/13 02:58, Joshua Landau wrote:
>>>>

>> Why? There are a lot of things it makes sense to take the sum of - a
>> lot of which have constant-ish performance on addition.
>
> A "lot" of things?
...
> I can't think of anything else that supports + and has near-constant time
> repeated addition. What are you thinking of?
...
>> I'd also point out there are a *lot* of times when iadd is faster than
>> add, not just contancation.
>
> Examples please.

I'm going to retreat from this point as my argument is weaker than I
thought it was. Mostly it comes from an internal delusion that I
overload operators more than I really do -- when I do it's normally
not for a final product. Since most of the following argument is based
around this, I put this at the top.

To be clear, I don't feel I'm wrong -- but I feel I'm less right than you.


>> I don't see why it should be constrained.
>
> I didn't say is should be constrained. But there is a big difference between
> "don't prevent people calling sum() on lists, if they insist" and "encourage
> people to use sum() on lists, in preference to list.extend".

I meant constrained in a more general term. Regardless of the OP's
intent, I don't think we need to encourage sum() on lists if the patch
goes through. I bet you a lot of people still do it though.

>>> It's unfortunate that Python uses + for concatenation,
>>
>> Is it? I'm very happy with it myself.
>
> Yes. If Python used & for concatenation, we wouldn't have to worry about
> sum(lists or strings or tuples) being quadratic, because people wouldn't
> call sum on lists, strings or tuples.

Optimising for the people who don't know what they're doing seems kind
of backwards. I like "+" to add lists.

I also think "data & [elem]" looks stupid. That's possibly just me,
but it's true for me nonetheless.

>> A fast sum() is not the one obvious way to concatenate data,
>
> The Original Poster seems to think it is.

I think he's wrong.

>>> In my opinion, the One Obvious Way to concatenate a lot of arbitrary data
>>> is list.extend, not sum.
>>
>> Is it? Maybe for lists. Often itertools.chain is better. It really
>> depends on your data. I tend to use itertools.chain a lot more than
>> list.extend, because I sum iterators more than I sum lists. Maybe I'm
>> just weird.
>
> I didn't say *the Best Way*, I said *the One Obvious Way*. The obvious way
> to have a lot of data of some arbitrary type is a list; the obvious way to
> concatenate a whole lot of lists into one list is using the extend method.
> If you have some special type or some special need, you may be able to do
> better.

I know what you said, and I raised a disagreement. The OOW to have a
lot of data of some arbitrary type is context-dependent. Sometimes
it's an iterator. Sometimes it's a list. Sometimes it's a set.
Sometimes it's deque (OK, pushing it).