<br><div><span class="gmail_quote">On 3/27/06, <b class="gmail_sendername">Neal Norwitz</b> <<a href="mailto:nnorwitz@gmail.com">nnorwitz@gmail.com</a>> wrote:</span><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
On 3/27/06, Thomas Wouters <<a href="mailto:thomas@python.org">thomas@python.org</a>> wrote:<br>><br>> The teeobject has GC (hence the word 'and' in 'itertools.tee and its<br>> internal teedataobject' ;-) The problem with test_generators is that this
<br>> also leaks:<br>><br>> def leak():<br>> def gen():<br>> while True:<br>> yield g<br>> g = gen()<br>><br>> leak()<br><br>Please add a test for this to the leakers and remove the tee test in
<br>leakers if that no longer leaks.</blockquote><div><br>I added this test, but I didn't remove test_tee, because it still leaks. I also checked in a test of code that used to leak, but doesn't anymore. The cycle this nested generator creates, which is also involved in the test_tee leak, is not cleanable by the cycle-gc, and it looks like it hasn't been since the yield-expr/coroutine patch was included in the trunk. I believe the culprit to be this part of that patch:
<br></div><br></div>Index: Modules/gcmodule.c<br>===================================================================<br>--- Modules/gcmodule.c (revision 39238)<br>+++ Modules/gcmodule.c (revision 39239)<br>@@ -413,10 +413,8 @@
<br> assert(delstr != NULL);<br> return _PyInstance_Lookup(op, delstr) != NULL;<br> }<br>- else if (PyType_HasFeature(op->ob_type, Py_TPFLAGS_HEAPTYPE))<br>+ else<br> return op->ob_type->tp_del != NULL;
<br>- else<br>- return 0;<br> }<br><br> /* Move the objects in unreachable with __del__ methods into `finalizers`.<br><br>Now, reverting that part of the patch in revision 39239 triggers an assertion failure, but removing the assertion makes it work right; the above nested-generator cycle gets cleaned up again. Later in the trunk, the assertion was changed anyway, and it no longer fails if I just revert the gcmodule change. Of course, the reason the cycle is uncleanable is because generators grew a tp_del method, and that throws cycle-gc in a hissy fit; reverting the above patch just makes cycle-gc ignore the tp_del of heaptypes. I don't know enough about the cycle-gc to say whether that's good or bad, but not having generators be cleanable is not very good. For what it's worth, reverting the gcmodule patch doesn't seem to break any tests.
<br clear="all"><br>However, fixing _both_ those things (itertools.tee lack of GC, and GC's inability to clean generators) *still does not fix the 254 refleaks in test_generators*. When I backport the itertools.tee-GC changes to r39238 (one checkin before coroutines), test_generators doesn't leak, neither the r39238 version of test_generators, nor the trunk version. One revision later, r39239, either test_generators leaks 15 references. 'Fixing' gcmodule, which fixes the nested-generator leak, does not change the number of leaks. And, as you all may be aware of, in the trunk, test_generators leaks 254 references; this is also the case with the gcmodule fix. So there's more huntin' afoot.
<br><br>In the mean time, if people knowledgeable about the cycle-GC care to comment about the gcmodule change wrt. heaptypes, please do.<br><br>-- <br>Thomas Wouters <<a href="mailto:thomas@python.org">thomas@python.org
</a>><br><br>Hi! I'm a .signature virus! copy me into your .signature file to help me spread!