[Python-Dev] Dictionary sparseness

[Guido van Rossum]

> > Hm.  Maybe so, but it *is* a feature that there are no user controls
> > over dictionary behavior, based on the observation that for every user
> > who knows enough about the dict implementation to know how to tweak
> > it, there are at least 1000 who don't, and the latter, in their
> > ill-advised quest for more speed, will use the tweakage API to their
> > detriment.
> 
> Perhaps there should be safety-belts and kindergarten controls:
> 
>    d.pack(fat=False) --> None.  Reclaims deleted entries.
>          If optional fat argument is true, the internal size is doubled
>          resulting in potentially faster lookups at the expense of
>          slower iteration and more memory.
>   
> This ought to be both safe and simple.

And a waste of time except in the most rare circumstances.

> Raymond Hettinger
> 
> 
> P.S.  Also, I think it worthwhile to at least transform dictresize()
> into PyDict_Resize() so that C extensions will have some control.
> This would make it possible for us to add a single line making
> the builtin dictionary more sparse and providing a 75% first probe
> hit rate.

And that would give *how much* of a performance improvement of typical
applications?

Sorry, I really think that you're complexificating APIs here without
sufficient gain.  I really value the work you've done on figuring out
how to improve dicts, but I think you've come to know the code too
well to see the other side of the coin.

--Guido van Rossum (home page: http://www.python.org/~guido/)