a.extend(b) better than a+=b ?

[Steven D'Aprano]

On Thu, 22 Apr 2010 09:31:12 +0200, candide wrote:

> Alf P. Steinbach a écrit :
>> * candide:
>>> Suppose a and b are lists.
>>>
>>> What is more efficient in order to extend the list a by appending all
>>> the items in the list b ?
>>>
>>>
>>> I imagine a.extend(b)to be more efficient for only appendinding the
>>> items from b while a+=b creates a copy of a before appending, right ?
>> 
>> No.
>> 
>> But in general, if you're concerned about efficiency, *measure*.
>> 
>> 
> 
> But my question refers to memory management rather than to time
> execution. Imagine foo is a big size list and bar is a small one. It
> would be a waste of memory to copy list foo somewhere in memory before
> appending the items in bar.



Yes, unfortunately measuring Python's memory use from within Python is 
not easy... however the fact that += and extend take approximately the 
same time suggests that they are doing approximately the same amount of 
work, and therefore it's unlikely that one is radically more wasteful of 
memory than the other.

Keep in mind that since Python uses duck-typing, you may not necessarily 
be dealing with actual built-in lists, but some other list-like object. 
If you're programming a library, and don't control your input, any 
conclusion you draw about lists may not apply in practice. If somebody 
provides you a sequence like this, what are you going to do?

class StupidList(list):
    def extend(self, other):
        new = []
        for item in self:
            new = new[:] + item
        for item in other:
            new = new[:] + item
        return new
    def __iadd__(self, other):
        self.extend(other)
        return self

You can't defend against the caller doing something stupid, so it's not 
worth spending too much effort trying.

In general the warnings against premature optimization apply just as 
strongly against memory optimizations as against speed optimizations. 



-- 
Steven