[Python-Dev] Frame zombies
Eyal Lotem
eyal.lotem at gmail.com
Sun Jun 10 06:38:03 CEST 2007
The freelist currently serves as a good optimization of a special case
of a recurring recursion. If the same code object (or one of the same
size) is used for recursive calls repeatedly, the freelist will
realloc-to-same-size (which probably has no serious cost) and thus the
cost of allocating/deallocating frames was averted.
I think that in general, the complexity of a sophisticated and
efficient aging mechanism is not worth it just to optimize recursive
calls. The only question is whether it is truly a memory problem, if
using, say, up-to 50 frames per code object?
Note that _only_ recursions will have more than 1 frame attached.
How many recursions are used and then discarded?
How slow is it to constantly malloc/free frames in a recursion?
My proposal will accelerate the following example:
def f(x, y):
if 0 == x: return
f(x-1, y)
def g(x):
if 0 == x: return
g(x-1)
while True:
f(100, 100)
g(100)
The current implementation will work well with the following:
while True:
f(100, 100)
But removing freelist altogether will not work well with any type of recursion.
Eyal
On 6/10/07, "Martin v. Löwis" <martin at v.loewis.de> wrote:
> > Should I make a patch?
>
> -1. This could consume quite a lot of memory, and I doubt
> that the speed improvement would be significant. Instead,
> I would check whether performance could be improved by
> just dropping the freelist. Looking at the code, I see
> that it performs a realloc (!) of the frame object if
> the one it found is too small. That surely must be
> expensive, and should be replaced with a free/malloc pair
> instead.
>
> I'd be curious to see whether malloc on today's systems
> is still so slow as to justify a free list. If it is,
> I would propose to create multiple free lists per size
> classes, e.g. for frames with 10, 20, 30, etc. variables,
> rather than having free lists per code object.
>
> Regards,
> Martin
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/eyal.lotem%40gmail.com
>
More information about the Python-Dev
mailing list