[Python-Dev] recursive closures - reference cycle

"Martin v. Löwis" martin at v.loewis.de
Wed Dec 9 23:37:11 CET 2009


> Yes, and a number of different workarounds.  That's not really the
> issue.  The issue is that what looks like a perfectly safe idiom
> (calling a function recursively) happens to create a reference cycle
> if that function is a closure. This is a non-obvious "gotcha" that I
> must now educate my team about.

I'm not sure why you call that non-obvious. As Antoine(?) pointed
out, *any* recursive function calls a reference cycle, and that is
not surprising, because the function refers to itself, so the function
definition is *obviously* cyclic (i.e. from within the function body,
you can get back to the function object).

I know you dismissed the global function example as non-relevant,
because the cycle is through the globals dictionary (i.e. any global
function is in a cycle, whether it is recursive or not), and because
it is possible to break the cycle (by deleting the function from the
globals). However, that's beside the point, because
a) it is an optimization that closures are only cyclic when they
   need to be (i.e. a nested function only keeps references to those
   name bindings that are mentioned in its body, rather than referencing
   the entire closure), and
b) it is also possible to break the cycle in the closure case, by
   setting the cell target to None (say), and
c) the actual problem that you are having is not that the nested
   functions create cycles, but that you get a fresh one every time
   you call the outer function (so that you get the impression of
   a memory leak).

> Note that at least parts of the library strive to avoid reference
> cycles even though "gc" exists.  An example being
> xlm.minidom.Document.unlink()

That doesn't avoid reference cycles. Instead, it helps breaking them
(as a minidom tree *is* cyclic). minidom has this explicit API for
breaking cycles because it predated the introduction of cyclic GC
in Python.

Regards,
Martin


More information about the Python-Dev mailing list