[C++-sig] Object destructor not always called in embedded interpreter

[Thomas Berg]

2008/10/13 Tanguy Fautré <tfautre at telenet.be>:
> Hi,
>
> I've been trying to sort this issue for some time now, but without any success. Long story short: some objects are never destroyed after executing a script using the embedded Python interpreter and Boost.Python.
>
> For example, when executing the following script in the embedded interpreter:
>
> def f() :
>        o2 = NamedObject('object2')
>
> o1 = NamedObject('object1')
> f()
>
>
> It will print:
>
> Creating NamedObject 'object1'
> Creating NamedObject 'object2'
> Destroying NamedObject 'object2'
>
>
> This shows that o1 is never destroyed. Note that if I call Py_Finalize() afterwards, o1 is correctly destroyed (although, from what I understand this function should not be called when using Boost.Python). This leads me to believe that something, somewhere, is holding a reference to that object.
>
> I've attached the sources of this example. You'll see that before I execute the python script, I create a new namespace object so that several scripts could be sequentially run without polluting each other's namespace (this seems to work). I would also expect that any python object defined in this namespace would be destroyed when the namespace object goes out of scoped in C++ (obviously this doesn't work, hence this email).
>
>
> Regards,
>
> Tanguy
>
>
> _______________________________________________
> C++-sig mailing list
> C++-sig at python.org
> http://mail.python.org/mailman/listinfo/c++-sig
>
>


Hi Tanguy,

I'm no python expert, but I think it works as follows:
When you import __main__, it is added to the internal list of modules
in python. It's __dict__ is therefore also referenced. So even though
you no longer reference it, the __dict__ will still be referenced by
the interpreter itself, and it's refcount doesn't reach 0 until you
finalize the interpreter...

So, what can you do? I also wanted to be able to clear the interpreter
and re-run scripts in clean namespaces, without re-initializing the
interpreter.

This can be achieved by calling clear() on the namespace (it's a dictionary).

However, that also clears the members "__builtins__", "__doc__" and
"__name__", which are needed (at least __builtins__) if you want to
execute code again. I solved this too, simply by re-initializing those
variables again after clearing the dictionary.

I also discovered that a similar initialization enables you to create
other modules than the "__main__" module (using PyImport_AddModule
instead of import) and execute scripts in those too, which can be
useful. I don't know whether this is the "correct" way of doing things
though...

See below for an example.

Regards,
Thomas

#include <boost/python.hpp>

using namespace boost::python;

Py_Initialize();
try {
    const char* code;
    object main_module = import("__main__");
    dict module_dict      = extract<dict>(main_module.attr("__dict__"));
    code = "a = 3";
    exec(code, module_dict, module_dict); // Should succeed
    code = "b = a";
    exec(code, module_dict, module_dict); // Should succeed

    module_dict.clear(); // deletes all variables
    module_dict["__builtins__"] = import("__builtin__");
    module_dict["__name__"]   = "__main__";
    module_dict["__doc__"]      = object();
    code = "c = b";
    exec(code, module_dict, module_dict); // Should fail, b not in namespace
} catch (error_already_set&) {
    //Should end up here because of the last exec, complaining about 'b'
    PyErr_Print();
}