[Python-Dev] Dictionary sparseness

[Raymond Hettinger]

> > After more dictionary sparseness experiments, I've become 
> > convinced that the ideal settings are better left up to the user 
> > who is in a better position to know:
> > 
> > * anticipated dictionary size 
> > * overall application memory issues
> > * characteristic access patterns (stores vs. reads vs. deletions
> >   vs. iteration)
> > * when the dictionary is growing, shrinking, or stablized.
> > * whether many deletions have taken place
> 
> Hm.  Maybe so, but it *is* a feature that there are no user controls
> over dictionary behavior, based on the observation that for every user
> who knows enough about the dict implementation to know how to tweak
> it, there are at least 1000 who don't, and the latter, in their
> ill-advised quest for more speed, will use the tweakage API to their
> detriment.

Perhaps there should be safety-belts and kindergarten controls:

   d.pack(fat=False) --> None.  Reclaims deleted entries.
         If optional fat argument is true, the internal size is doubled
         resulting in potentially faster lookups at the expense of
         slower iteration and more memory.
  
This ought to be both safe and simple.


Raymond Hettinger


P.S.  Also, I think it worthwhile to at least transform dictresize()
into PyDict_Resize() so that C extensions will have some control.
This would make it possible for us to add a single line making
the builtin dictionary more sparse and providing a 75% first probe
hit rate.