[Python-Dev] Segfault in python 2.5
tim.peters at gmail.com
Thu Oct 19 03:02:31 CEST 2006
>> I've been reading the bug report with interest, but unless I can
>> reproduce it it's mighty hard for me to debug, as I'm sure you know.
Note that I just attached a much simpler pure-Python script that fails
very quickly, on Windows, using a debug build. Read the new comment
to learn why both "Windows" and "debug build" are essential to it
failing reliably and quickly ;-)
>>> Unfortunately, I've been attempting for hours to reduce the problem to a
>>> completely self-contained script, but it is resisting my efforts
due to timing
Yes, but you did good! This is still just an educated guess on my
part, but my education here is hard to match ;-): this new business
of generators deciding to "clean up after themselves" if they're left
hanging appears to have made it possible for a generator to hold on to
a frame whose thread state has been free()'d, after the thread that
created the generator has gone away. Then when the generator gets
collected as trash, the new exception-based "clean up abandoned
generator" gimmick tries to access the generator's frame's thread
state, but that's just a raw C struct (not a Python object with
reachability-based lifetime), and the thread free()'d that struct when
the thread went away. The important timing-based vagary here is
whether dead-thread cleanup gets performed before the main thread
tries to clean up the trash generator.
> I've peered at the code, but my knowledge of the python core is
> superficial at best. The fact that it is occuring as a result of a
> long string of garbage collection/dealloc/etc. and involves threading
> lowers my confidence further. That said, I'm beginning to think that
> to reproduce this in a standalone script will require understanding
> the problem in greater depth regardless...
Or upgrade to Windows ;-)
>> Are you absolutely sure that the fault does not lie with any extension
>> modules you may be using? Memory scribbling bugs have been known to
>> cause arbitrarily confusing problems...
Unless I've changed the symptom, it's been reduced to minimal pure
Python. It does require a thread T, and creating a generator in T,
where the generator object's lifetime is controlled by the main
thread, and where T vanishes before the generator has exited of its
Offhand I don't know how to repair it. Thread states /aren't/ Python
objects, and there's no provision for a thread state to outlive the
thread it represents.
> I've had sufficient experience being arbitrarily confused to never be
> sure about such things, but I am quite confident. The script I posted
> in the bug report is all stock python save for the operation in <>'s.
> That operation is pickling and unpickling (using pickle, not cPickle)
> a somewhat complicated pure-python instance several times.
FYI, in my whittled script, your `getdocs()` became simply:
and it's called only once, via self.docIter.next(). In fact, the
"while True:" isn't needed there either (given that it's only resumed
More information about the Python-Dev