Optimisation Hints (dict processing and strings)
poyol at hotmail.com
Wed Mar 30 09:41:15 CEST 2005
#For the record, I'm not on premature optimisation anymore.
#The code is working. I just want to save hours of computing, without
relying to much on C extensions.
#Nevertheless, thansk for tips, clarifications and explanations.
> >>>>longone=longone + char # where len(char)== 1
> > I known that string concatenation is time consuming, but a small test
> > on timeit seems to show that packing and creating an array for those 2
> > elements is equally time consuming
> - use cStringIO instead
> - or append all chars to a list and do "".join (listvar)
Yes, but as Peter said using a list won't be useful for a single
And I have to compute the string at each step of the loop. I think
listvar won't be useful.
But I'm going to try cStringIO.
> > for (2):
> > for k in hash.keys()[:]: # Note : Their may be a lot of keys here
> > if len(hash[k])<2:
> > del hash[k]
> - Try if it isn't faster to iterate using items instead of iterating
> over keys
items are huge lists of numbers. keys are simple small strings. And
even if it is faster, how can I find the key back, in order to delete
for v in hashh.items():
> - use the dict.iter* methods to prevent building a list in memory. You
> shouldn't use these values directly to delete the entry as this could
> break the iterator:
> for key in [k for (k, v) in hash.iteritems () if len (v) < 2]:
> del hash (key)
I gonna try, but think that would be overkill: a whole list has to be
Maybe whith genexps ...... for key in (k for (k,v) in hash.iteritems()
> This of course builds a list of keys to delete, which could also be large.
Yes. Which tell me to use genexps.
> - also: hash.keys()[:] is not necessary, hash.keys () is already a copy
Yes I got that one, but too late !
More information about the Python-list