Let's imagine that I have some code like the following in RPython: def wrapper_func(arg1, arg2): return inner_func(arg2) def inner_func(x): for y in range(x): # do something here pass return -1 bigint = 1000000 wrapper_func(list(range(bigint)), bigint) The problem here is that arg1 is going to be held onto in (in CPython at least) until inner_func returns. This means that the list created on the invocation of wrapper_func is going to stick around during the entire execution time of inner_func. This would not make much of a difference normally, but in languages with extensive use of lazy evaluation, holding onto the head of a sequence could cause out-of-memory errors. In Clojure this is fixed-up by the compiler via "locals clearing". Basically the compiler inserts "arg1 = None" before the invocation of inner_func. What's the story here in RPython? Since RPython basically compiles down to single-assignment code I'm guessing the Clojure fix won't help me. When is the GC able to go and free data held by arguments? Timothy
2012/9/6 Timothy Baldridge <tbaldridge@gmail.com>:
Let's imagine that I have some code like the following in RPython:
def wrapper_func(arg1, arg2): return inner_func(arg2)
def inner_func(x): for y in range(x): # do something here pass return -1
bigint = 1000000
wrapper_func(list(range(bigint)), bigint)
Since that's all evaluated at import time, I don't see what the problem is. -- Regards, Benjamin
On Thu, Sep 6, 2012 at 1:30 PM, Benjamin Peterson <benjamin@python.org>wrote:
2012/9/6 Timothy Baldridge <tbaldridge@gmail.com>:
Let's imagine that I have some code like the following in RPython:
def wrapper_func(arg1, arg2): return inner_func(arg2)
def inner_func(x): for y in range(x): # do something here pass return -1
bigint = 1000000
wrapper_func(list(range(bigint)), bigint)
Since that's all evaluated at import time, I don't see what the problem is.
Nice, but that completely missed the point of my question. I know this wouldn't be a problem in this exact case. The question is: when is the GC free to free data passed into a function's arguments. Will that function hold on to all data passed in through arguments until the execution of the function terminates? If so is there a way to trigger garbage collection of unneeded argument data before the end of the function's execution? Timothy
Hi Timothy, On Thu, Sep 6, 2012 at 8:34 PM, Timothy Baldridge <tbaldridge@gmail.com> wrote:
The question is: when is the GC free to free data passed into a function's arguments. Will that function hold on to all data passed in through arguments until the execution of the function terminates? If so is there a way to trigger garbage collection of unneeded argument data before the end of the function's execution?
RPython is not precisely defined for this question to have a definite answer. However, currently, with all our own GCs, keep-alive works this way: 1. First, an RPython function keeps alive a variable "for as little time as necessary". Even if, as plain Python, the variable would be kept alive until the function returns just because it's stored in a local, it is not the case in RPython. 2. Across a call, *only* the variables that need to remain alive *after* the call is done are pushed on the shadow stack. In particular, tail calls are really just goto's from the point of view of the GC (even if they are not implemented as tail calls from the point of view of the translation to C). Using these general rules to figure out what occurs in your case:
def wrapper_func(arg1, arg2): return inner_func(arg2)
def inner_func(x): for y in range(x): # do something here pass return -1
wrapper_func(list(range(bigint)), bigint)
The list is passed as 'arg1' but not passed any further. It is not saved either across the call to wrapper_func() nor the call to inner_func(). So it is dead. If inner_func() triggers a GC cycle, it will be collected. If you need to prevent that (e.g. because 'arg1' is an object with a __del__ and you don't want this __del__ to be called too early) then you need to insert a call to pypy.rlib.objectmodel.keepalive_until_here(). A bientôt, Armin.
2012/9/6 Timothy Baldridge <tbaldridge@gmail.com>:
Nice, but that completely missed the point of my question. I know this wouldn't be a problem in this exact case. The question is: when is the GC free to free data passed into a function's arguments. Will that function hold on to all data passed in through arguments until the execution of the function terminates? If so is there a way to trigger garbage collection of unneeded argument data before the end of the function's execution?
It depends on the GC. The moving gcs will not keep arguments as stack roots unless they are needed. -- Regards, Benjamin
participants (3)
-
Armin Rigo -
Benjamin Peterson -
Timothy Baldridge