[Python-Dev] shrinking dicts

[M.-A. Lemburg]

Vladimir Marangozov wrote:
> 
> M.-A. Lemburg wrote:
> >
> > [me]
> > > Any other ideas on how to deal with this? Thoughts, comments?
> >
> > I think that integrating this into the C code is not really that
> > effective since the situation will not occur that often and then
> > it often better to let the programmer decide rather than integrate
> > an automatic downsize.
> 
> Agreed that the situation is rare. But if it occurs, its Python's
> responsability to manage its data structures (and system resources)
> efficiently. As a programmer, I really don't want to be bothered with
> internals -- I trust the interpreter for that. Moreover, how could
> I decide that at some point, some dict needs to be resized in my
> fairly big app, say IDLE?

You usually don't ;-) because "normal" dict only grow (well, more or
less). The downsizing thing will only become a problem if you use
dictionaries in certain algorithms and there you handle the problem
manually.

My stack implementation uses the same trick, BTW. Memory is cheap
and with an extra resize method (which the mxStack implementation
has), problems can be dealt with explicitly for everyone to see
in the code.

> > You can call dict.update({}) to force an internal
> > resize (the empty dictionary can be made global since it is not
> > manipulated in any way and thus does not cause creation overhead).
> 
> I know that I can force the resize in other ways, but this is not
> the point. I'm usually against the idea of changing the programming
> logic because of my advanced knowledge of the internals.

True, that why I mentioned...
 
> >
> > Perhaps a new method .resize(approx_size) would make this even
> > clearer. This would also have the benefit of allowing a programmer
> > to force allocation of the wanted size, e.g.
> >
> > d = {}
> > d.resize(10000)
> > # Insert 10000 items in a batch insert
> 
> This is interesting, but the two ideas are not mutually excusive.
> Python has to dowsize dicts automatically (just the same way it doubles
> the size automatically). Offering more through an API is a plus for
> hackers. ;-)

It's not really for hackers: the point is that it makes the technique
visible and understandable (as opposed to the hack above). The same
could be useful for lists too (the hack there is l = [None] * size,
which I find rather difficult to understand at first sight...).

-- 
Marc-Andre Lemburg
______________________________________________________________________
Y2000:                                                   143 days left
Business:                                      http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/