Searching for the meaning of gcmodule.c assert:`gc->gc. gc_refs ==GC_REACHABLE'
tim.one at comcast.net
Fri Nov 14 04:26:32 CET 2003
> I am embedding python on Mac OSX (jaguar + panther) using
> boost::python. It works like a treat, but when using a python
> framework with asserts compiled in, I sometimes get an assert from
> gcmodule.c, line 215 when compiling a script.
What's on the C stack then (without a debugger, this is hopeless)?
> Modules/gcmodule.c:215: failed asserrtion `gc->gc.gc_refs ==
What's the value in gc->gc.gc_refs then?
> This is maybe the third script I compile, so the python environment
> should still be very clean.
> I tried reading the code in gcmodule.c but don't understand what the
> assert is warning me about or what is wrong here.
assert()s in Python's implementation aren't warnings, they're telling you
that something is so wrong that it should be impossible. It means something
else went deadly seriously wrong. But this is C code, and it can be
arbitrarily difficult to figure out exactly what went wrong. It may have
happened billions of cycles earlier. The asserts in gcmodule.c catch many C
coding mistakes throughout the interpreter and extension modules, but occur
long after the error was made -- gc only runs when there's a backlog of new
unexamined container objects waiting to analyze for cyclic trash. This
particular assert happens very shortly after gc begins, so it's not that gc
did something that confused itself: gc found something wildly wrong about
the input state of the objects it's *going* to analyze for dead cycles. All
evidence about *why* they went wrong is probably long gone.
> Since I'm embedding python I suspect that I'm doing something wrong
> with reference counting,
Not enough evidence here to say. I've never seen that particular assert
trigger before. The most likely way I can *imagine* it happening is a wild
store (C code writing into memory it has no business overwriting -- stuff
like an out-of-bounds indexed array store in C, or bad pointer arithmetic).
> but in a ref-counted world that would either lead to leaks or crashes.
Or it may not present any symptom. For example, if C code lets references
to Py_None leak (forgets to decref), there's no visible effect unless the
refcount on Py_None grows so large that it overflows the C int it's stored
in. OTOH, refcounts that are too small are the refcount equivalent of
dangling pointers, and can lead to arbitrary memory corruption (when the
refcount falls to 0, one part of the code hands the object memory back to
the allocator, which may hand it out to some other object; but, by
presumption, the refcount was too small, so there are pointers in the C code
that believe they still point to the old object, and use it as if that were
true); that may or may not lead to crashes or leaks; it *can* lead to any
sort of symptom, and usually in input- and timing- dependent ways.
> Can wrong ref-counting confuse the garbage collector so badly that it
As above, a dangling pointer can lead to anything whatsoever, and a
too-small refcount can lead to dangling pointers. That seems unlikely in
this case, but there's not enough info here to rule it out.
> Since I don't really need the garbage collector I've tried disabling
> it from python, and that fixes the problem. Only I'm worried that I've
> just managed to hide a bug, not remove it.
I'd purse that last thought: assert failures scream "bug". It may even be
in Python, although that's unlikely since we've seen just about every bug
Python's ever had during the long stretches of CVS development between
releases, and I've never seen that particular assert fail before.
> I would be infinitly thankful if anyone could point me in a general
> direction as to what the problem might be.
You can't out-think this one on "general principles", so the most helpful
thing I can tell you is to get into a debugger and start answering questions
about the state of the world when this happens (I asked two specific
questions at the start of this msg, to get you started; another obvious
question is what type of object update_refs() is looking at when this
happens; FROM_GC(gc) gives you a PyObject* p, and p->ob_type is then a
pointer to the type object).
More information about the Python-list