[Python-Dev] Reference counting when entering and exiting scopes

[Matthew F. Barnes]

Someone on python-help suggested that I forward this question to
python-dev.

I've been studying Python's core compiler and bytecode interpreter as a
model for my own interpreted language, and I've come across what appears
to be a reference counting problem in the `symtable_exit_scope' function
in <Python/compile.c>.

At this point I assume that I'm just misunderstanding what's going on.  So
I was hoping to contact one of the core developers before I go filing what
could very well be a spurious bug report against Python's core.

Here's the function copied from CVS HEAD:

static int
symtable_exit_scope(struct symtable *st)
{
	int end;

	if (st->st_pass == 1)
		symtable_update_free_vars(st);
	Py_DECREF(st->st_cur);
	end = PyList_GET_SIZE(st->st_stack) - 1;
	st->st_cur = (PySymtableEntryObject *)PyList_GET_ITEM(st->st_stack,
							      end);
	if (PySequence_DelItem(st->st_stack, end) < 0)
		return -1;
	return 0;
}

My issue is with the use of PyList_GET_ITEM to fetch a new value for the
current scope.  As I understand it, PyList_GET_ITEM does not increment the
reference count for the returned value.  So in effect we're borrowing the
reference to the symtable entry object from the tail of the scope stack. 
But then we turn around and delete the object from the tail of the scope
stack, which DOES decrement the reference count.

So `symtable_exit_scope' has a net effect of decrementing the reference
count of the new current symtable entry object, when it seems to me like
it should stay the same.  Shouldn't the reference count be incremented
when we assign to "st->st_cur" (either explicitly or by fetching the
object using the PySequence API instead of PyList)?

Can someone explain the rationale here?

---------------------------------------------------------------------------

As an addendum to the previous question, further study of the
<Python/compile.c> code has made me believe that there's a reference
counting problem in the `symtable_enter_scope' function as well (pasted
below from CVS HEAD).  Namely, that `prev' should be Py_XDECREF'd at some
point in the function (at the end of the first IF block, perhaps?).

static void
symtable_enter_scope(struct symtable *st, char *name, int type,
		     int lineno)
{
	PySymtableEntryObject *prev = NULL;

	if (st->st_cur) {
		prev = st->st_cur;
		if (PyList_Append(st->st_stack, (PyObject *)st->st_cur) < 0) {
			st->st_errors++;
			return;
		}
	}
	st->st_cur = (PySymtableEntryObject *)
		PySymtableEntry_New(st, name, type, lineno);
	if (st->st_cur == NULL) {
		st->st_errors++;
		return;
	}
	if (strcmp(name, TOP) == 0)
		st->st_global = st->st_cur->ste_symbols;
	if (prev && st->st_pass == 1) {
		if (PyList_Append(prev->ste_children,
				  (PyObject *)st->st_cur) < 0)
			st->st_errors++;
	}
}

Thanks,
Matthew Barnes