[Python-Dev] Frame zombies

[Eyal Lotem]

The freelist currently serves as a good optimization of a special case
of a recurring recursion.  If the same code object (or one of the same
size) is used for recursive calls repeatedly, the freelist will
realloc-to-same-size (which probably has no serious cost) and thus the
cost of allocating/deallocating frames was averted.

I think that in general, the complexity of a sophisticated and
efficient aging mechanism is not worth it just to optimize recursive
calls. The only question is whether it is truly a memory problem, if
using, say, up-to 50 frames per code object?

Note that _only_ recursions will have more than 1 frame attached.

How many recursions are used and then discarded?
How slow is it to constantly malloc/free frames in a recursion?

My proposal will accelerate the following example:

def f(x, y):
  if 0 == x: return
  f(x-1, y)
def g(x):
  if 0 == x: return
  g(x-1)

while True:
  f(100, 100)
  g(100)

The current implementation will work well with the following:

while True:
  f(100, 100)

But removing freelist altogether will not work well with any type of recursion.

Eyal

On 6/10/07, "Martin v. Löwis" <martin at v.loewis.de> wrote:
> > Should I make a patch?
>
> -1. This could consume quite a lot of memory, and I doubt
> that the speed improvement would be significant. Instead,
> I would check whether performance could be improved by
> just dropping the freelist. Looking at the code, I see
> that it performs a realloc (!) of the frame object if
> the one it found is too small. That surely must be
> expensive, and should be replaced with a free/malloc pair
> instead.
>
> I'd be curious to see whether malloc on today's systems
> is still so slow as to justify a free list. If it is,
> I would propose to create multiple free lists per size
> classes, e.g. for frames with 10, 20, 30, etc. variables,
> rather than having free lists per code object.
>
> Regards,
> Martin
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/eyal.lotem%40gmail.com
>