<br><div><span class="gmail_quote">On 4/15/06, <b class="gmail_sendername">Phillip J. Eby</b> <<a href="mailto:pje@telecommunity.com">pje@telecommunity.com</a>> wrote:</span><br><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
Interestingly, this code does *not* crash the interpreter on its own, only<br>when run as a doctest (with or without regrtest).</blockquote><div><br>Actually, it does, you just have to force GC to clean up stuff (rather than wait for the interpreter-exit to just toss the whole lot out without giving it a second glance.) You do that by making it unreachable and calling
gc.collect().<br><br>centurion:~/python/python/trunk > cat test_gencrash.py<br>import gc<br><br>class LazyList:<br> def __init__(self, g):<br> self.v = None<br> self.g = g<br><br> def __iter__(self):
<br> yield 1<br> if self.v is None:<br> self.v = self.g.next()<br> yield self.v<br><br>def loop():<br> for i in ll:<br> yield i<br><br>ll = LazyList(loop())<br>g=iter(ll)<br>g.next
()<br><br>g.next()<br><br>del g, ll<br>print gc.collect()<br><br>centurion:~/python/python/trunk > ./python test_gencrash.py<br>Segmentation fault (core dumped)<br><br><br></div><br><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
I still haven't figured<br>out what the actual problem is, but at least this cuts out all the merge()<br>and times() crud in the case this was derived from, reducing it to a pure<br>zen essence of non-meaning. :)</blockquote>
<div><br>I'm sure I know where the crash is coming from: one of the generator objects is being dealloced twice. The GC module tp_clears one of the frame objects, which deallocs a generator the frame referenced. That first dealloc calls _Py_ForgetReference and then calls gen_dealloc, which calls its tp_del (as it's a running generator, albeit a simple one), which raises an exception in the frame, which causes it to be cleaned up, which causes another generator object to be cleaned up and its tp_del method to be called, which - for some reason - tries to dealloc the first generator again. The second dealloc calls _Py_ForgetReference again, and _Py_ForgetReference doesn't like that (the ob_prev/ob_next debug ptrs are already wiped when it's called the second time, so it crashes.)
<br><br>The thing I don't understand is how the cleaning up of the second generator triggers a dealloc of the first generator. The refcount is 0 when the dealloc is called the first time (or _Py_ForgetReference, as well as a few other places, would have bombed out), and it is also 0 when it gets dealloced the second time. Since deallocation is triggered by DECREF'ing, that implies something is INCREF'ing the first generator somewhere -- but without having a valid reference, since the generator has been cleaned up by actually DECREF'ing the reference that a frame object held. Or, somehow, deallocation is triggered by something other than a DECREF. Hmm. No, the second dealloc is triggered by the Py_XDECREF in
ceval.c, when clearing the stack after an exception, which is the right thing to do. Time to set some breakpoints and step through the code, I guess.<br></div></div><br>(I first thought the problem was caused by gen_dealloc doing 'Py_DECREF(gen->gen_frame)' instead of 'frame = gen->gen_frame; gen->gen_frame = NULL; Py_DECREF(frame)', but that isn't the case. It should do it that way, I believe, but it's not the cause of this crash.)
<br><br>-- <br>Thomas Wouters <<a href="mailto:thomas@python.org">thomas@python.org</a>><br><br>Hi! I'm a .signature virus! copy me into your .signature file to help me spread!