[Python-Dev] Dictionary tuning upto 100,000 entries

[M.-A. Lemburg]

Raymond Hettinger wrote:
>>I believe that the reason for the speedups you see is
>>that cache sizes and processor optimizations have changes
>>since the times the current resizing implementation was chosen,
>>so maybe we ought to rethink the parameters:
>>
>>* minimum table size
>>* first three resize steps
> 
> 
> I've done dozens of experiements with changing these parameters
> and changing the resize ratio (from 2/3 to 4/5, 3/5, 1/2, 3/7, and 4/7)
> but found that what helped some applications would hurt others.
> The current tuning remains fairly effective.  Changing the resize
> step from *2 to *4 was the only alteration that yielded across 
> the board improvements.

Ok, but I still fear that using *4 will cause too much memory
bloat for dicts which have more than 10-30 entries. If you instrument
Python you'll find that for typical applications, most dictionaries
will have only few entries. Tuning the implementation to those
findings is what you really want to do :-)

If you take e.g. Zope, what difference in memory consumption does
your patch make ?

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Software directly from the Source  (#1, Apr 29 2003)
 >>> Python/Zope Products & Consulting ...         http://www.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
EuroPython 2003, Charleroi, Belgium:                        56 days left