huge slowdown on append

[Peter Otten]

ianaré wrote:

> On Dec 21, 3:29 pm, "ianaré" <ian... at gmail.com> wrote:
>> Hey all,
>>
>> I have a for loop which included the line:
>>
>> items_ren.append(join(newPath,renamedItem))
>>
>> I changed it to this:
>> items_ren.append([join(newPath,renamedItem), False])
>>
>> And processing speed is now much much slower. For 5780 items the old
>> function would take 9.5 seconds (there is other stuff going on
>> obviously), after changing that single line, speed is now 55 seconds
>> in the same conditions!
>>
>> Can anyone give me some pointers please? I would like to keep the same
>> structure if possible, as it allows the code in other places to be
>> much faster. TIA
>>
>> BTW, the 'join' function is this:
>> def join(newPath,renamedItem):
>>             return unicode(os.path.join(newPath,renamedItem))
> 
> Seems like I found a way ... Use the old way, then at the end of the
> for loop add the 'False'.
> def addStatus(x):
>     return (x,False)
> items_ren = map(addStatus, items_ren)
> 
> 9.5 seconds wOOt!
> 
> However, if anyone has a faster way, let me know!

Maybe 

import itertools as it
items_ren = zip(items_ren, it.repeat(False))

Or if you want to start earlier on

def gen_items():
    for ...
        yield newPath, renamedItem

items_ren = zip(it.imap(unicode, it.starmap(os.path.join, gen_items())),
it.repeat(False))

Probably buggy, but you should get the idea...

Peter