[capi-sig] Fix random crashes - embedding py/w PyObject pointers to app data.
rhamph at gmail.com
Wed Oct 22 08:52:18 CEST 2008
On Tue, Oct 21, 2008 at 10:54 PM, Campbell Barton <ideasman42 at gmail.com> wrote:
> On Wed, Oct 22, 2008 at 3:03 PM, Adam Olsen <rhamph at gmail.com> wrote:
>> On Tue, Oct 21, 2008 at 9:34 PM, Campbell Barton <ideasman42 at gmail.com> wrote:
>>> Hi, just spent a couple of hours looking into a bug that others might also have.
>>> If your app embeds python, and allows scripts to throw exceptions
>>> while the application keeps running you may also have this problem.
>>> sys.last_traceback will store the last exception along with the
>>> namespace dictionary of the script that just ran (with all its
>>> functions etc, variables etc).
>>> In our case this caused crashes because you could load in new data,
>>> making many of the PyObjects referenced from sys.last_traceback, point
>>> to invalid data. Once sys.last_traceback was set again it would free
>>> the existing data sometimes touching pointers with invalid addresses
>>> causing a segfault.
>>> Simple solution is to run this after PyErr_Print();
>>> PySys_SetObject( "last_traceback", Py_None);
>>> This can also free memory, if large datasets are references from the
>>> traceback. maybe some of your would fine this useful.
>> Do you mean you have a PyObject with a pointer to malloc'd memory, and
>> after the exception is handled (but not cleared) you free the malloc'd
>> memory, but leave the PyObject with a pointer to it? If so you should
>> be keeping track of any PyObjects and clearing them instead (or let
>> them take care of freeing the malloc'd memory).
> Yeah, this is the case and you're right, we should be aware of the
> PyObjects existence and invalidate them when the memory is freed, or
> let python do the freeing.
> But our application does not center around python, the PyAPI has its
> own directory and PyObjects are not supposed to be used outside that.
> Id be interested in knowing what other apps do - is this even a common problem?
Common in the sense that other apps probably do it right, yes. ;)
How are you calling into python? If you call a single python function
(which in turn calls many others), and this is what raises the
exception, this is the obvious point to trigger the clearing.
Adam Olsen, aka Rhamphoryncus
More information about the capi-sig