[Python-Dev] Dictionary tuning

[Delaney, Timothy C (Timothy)]

> From: Raymond Hettinger [mailto:python@rcn.com]
>=20
> [Timothy Delaney]
> > That's what I was getting at. I know that (for example) most
> > classes I create have less that 16 entries in their __dict__.
> > With this change, each class instance would take (approx) twice
> > as much memory for its __dict__. I suspect that class instance
> > __dict__ is the most common dictionary I use.
>=20
> Those dicts would also be the ones benefitting from the patch.
> Their density would be halved; resulting in fewer collisions,
> improved search times, and better cache performance.

No question that they would benefit. The question is whether the
benefit outweighs the possible penalties. Of course, we can't
evaluate that until we've got some data ...

> [Timothy Delaney]
> > Perhaps we need to add some internal profiling, so that
> > "quickly-growing" dictionaries get larger reallocations ;)
>=20
> I came up with this patch a couple of months ago and have
> since tried every tweak I could think of (apply to this size
> dict but not that one, etc) but found nothing that survived
> a battery of application benchmarks.

Note the smiley. Not at all intended seriously - the extra
complication would almost certainly eliminate any possible
performance gains.

> Have you guys tried out the patch? I'm very interested in=20
> getting results from different benchmarks, processors,=20
> cache sizes, and various operating systems.

If I can find the time I will. We're in crunch time on my
project at the moment ... I'm somewhat over-allocated :(

In case I forgot to mention it - I like the ideas in the patch,
and really like the performance improvement. But with the things
I'm doing at the moment, memory is proving more of a bottleneck
than performance ... once we start hitting virtual memory, you
can forget about a 5% performance improvement ...

Tim Delaney