[ python-Bugs-1470508 ] Error in PyGen_NeedsFinalizing()

SourceForge.net noreply at sourceforge.net
Fri Apr 14 21:31:55 CEST 2006

Bugs item #1470508, was opened at 2006-04-14 15:31
Message generated for change (Tracker Item Submitted) made by Item Submitter
You can respond by visiting: 

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Interpreter Core
Group: Python 2.5
Status: Open
Resolution: None
Priority: 7
Submitted By: Tim Peters (tim_one)
Assigned to: Phillip J. Eby (pje)
Summary: Error in PyGen_NeedsFinalizing()

Initial Comment:
There's an off-by-one error in PyGen_NeedsFinalizing():
 the loop starts by looking at
f->f_blockstack[f->f_iblock], but that's one beyond the
largest legitimate f_blockstack index.  As a result,
any generator with an active loop is very likely to
claim it needs finalizing (to the extent that it's
unlikely that reading up trash for b_type will match

The attached patch repairs that, but also tries to
remove some of the goofy .close() gimmicks added to old
tests in test_generators to worm around "leaks" caused
by that all generators said they needed finalizers in
2.5 at first.  (That's how I bumped into this:  I tried
removing the .close() gimmicks first, but -R:: runs
said they still leaked -- that started the trail
leading to PyGen_NeedsFinalizing()).

Alas, after applying the patch, test_generators craps
out in a debug build with a NULL-pointer dereference in
_Py_ForgetReference(), in the guts of the `if` test
deciding whether to produce an "UNREF invalid object"
error.  At the time, op_type is _PyGen_Type and
op->_ob_next = _ob_prev = NULL, probably meaning that
op was already passed to _Py_ForgetReference() previously.

We get into _Py_ForgetReference() via a chain starting
at the the decref in ceval.c's

	while (STACK_LEVEL() > b->b_level) {
		v = POP();

and we got into PyEval_EvalFrameEx via gen_send_ex via
gen_close via gen_dealloc via (eventually) a dict on
some instance going away ... and then I got lost. 
Looks like there are several more layers of generator
deallocs on the call stack too.

I'm out of time for looking at this now, so recording
it here.


You can respond by visiting: 

More information about the Python-bugs-list mailing list