[Python-ideas] Keep free list of popular iterator objects
raymond.hettinger at gmail.com
Sun Sep 15 05:28:50 CEST 2013
On Sep 14, 2013, at 8:12 PM, Kyle Fisher <anthonyfk at gmail.com> wrote:
> We tend to do a lot of iterating over dictionaries in our product in some performance critical areas. It occurred to me that allocating a new iterator object every single time seems a little wasteful, especially considering that there's probably only a handful of them alive at any time. Doing a quick test with dictiterobject and 3 free lists (one for Keys, Values and Items) showed about a 4% speedup in this (best) case:
It is surprising that you saw any performance gain at all.
Python already has a default Python freelist scheme
in the _PyObject_Malloc() function in Objects/obmalloc.c.
Another thought is that this isn't an inner-loop optimization.
The O(1) time for iterator creation is dominated by the O(n)
time to actually iterate over the dict keys, values, and items.
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the Python-ideas