Optimisation Hints (dict processing and strings)

OPQ poyol at hotmail.com
Wed Mar 30 09:41:15 CEST 2005


#For the record, I'm not on premature optimisation anymore. 
#The code is working. I just want to save hours of computing, without
relying to much on C extensions.
#Nevertheless, thansk for tips, clarifications and explanations.

> >>>>longone=longone + char # where len(char)== 1
> > 
> > I known that string concatenation is time consuming, but a small test
> > on timeit seems to show that packing and creating an array for those 2
> > elements is equally time consuming
> 
> - use cStringIO instead
> - or append all chars to a list and do "".join (listvar)


Yes, but as Peter said using a list won't be useful for a single
operation.
And I have to compute the string at each step of the loop. I think
listvar won't be useful.
But I'm going to try cStringIO. 

> 
> > 
> > for (2):
> > for k in hash.keys()[:]: # Note : Their may be a lot of keys here
> >    if len(hash[k])<2:
> >       del hash[k]
> 
> - Try if it isn't faster to iterate using items instead of iterating 
> over keys

items are huge lists of numbers. keys are simple small strings. And
even if it is faster, how can I find the key back, in order to delete
it ?
for v in hashh.items():
    if len(v)<2:
           del ???????



> - use the dict.iter* methods to prevent building a list in memory. You 
> shouldn't use these values directly to delete the entry as this could 
> break the iterator:
> 
> for key in [k for (k, v) in hash.iteritems () if len (v) < 2]:
>      del hash (key)
> 

I gonna try, but think that would be overkill: a whole list has to be
computed !
Maybe whith genexps ...... for key in (k for (k,v) in hash.iteritems()
if len(v)<2)


> This of course builds a list of keys to delete, which could also be large.
> 

Yes. Which tell me to use genexps.

> - also: hash.keys()[:] is not necessary, hash.keys () is already a copy
> 

Yes I got that one, but too late !

Thanks !



More information about the Python-list mailing list