Adam, you probably run into a similar problem that I ran into (not with boost::shared_ptr but with our custom implementation which strongly mimics boost::shared_ptr). I wanted to have a class in C++ that is extended in python, and it should store a weak_ptr to itself. Unfortunately I didn't solve this yet, I stored a shared_ptr which means that the object gets never deleted. I have to revisit this at a later point, but I'll try to explain you what's going on. You probably have a function like this void addCallback(share_ptr<> arg) { callbacks.push_back(weak_ptr<>(arg); } Or you immediately take the argument as a weak_ptr, that doesn't matter. The more important fact is, you created that object in some C++, you have that object in python and then you call addCallback from the python context. So, before the function exits your object `arg` exists at least in three places: 1) somewhere in C++ where it was created 2) in the python context 3) in the context of addCallback To understand what is going wrong, I need to explain that the shared pointer in context 1 and 3 are not the same! They share the same deallocator and the same object, but they are not the same shared pointer, i.e. their use count and weak_count are not the same. That is due to the way boost python handles step 2, that magic is in these three files (in boost/python): converter/shared_ptr_to_python.hpp converter/shared_ptr_from_python.hpp converter/shared_ptr_deleter.hpp In `shared_ptr_deleter.hpp` there is a certain magic implemented, that does the following: Assume you have class `Parent` and a class `Child` derived from it. Now you can do: - create an instance of Child C++ and bring it to python as shared_ptr<Child> - pass that instance to C++ (via shared_ptr<Child> or shared_ptr<Parent>) - get it later back from C++ but as a shared_ptr<Parent> - magic: you can treat that instance as a shared_ptr<Child> In C++ you would need to do a dynamic cast to get this functionallity, but because that object has been known to python to be an instance of Child, boost::python automatically makes it an instance of Child, nice right? Unfortunately your (and my) problem are a consequence of this. When you go from 2->3 boost::python prepares for doing its magic. It doesn't just return a copy of the shared_ptr from 1), it creates a new shared_ptr with a special Deallocator object. The use_count at that moment is 2: one for python 2) and one for addCallback() 3). When the function addCallback() finishes, the use_count=1 (from python) and weak_count=1 from 3). Once the python context ends then use_count=0 and weak_count=1, and I believe that is exactly what you observe. In this case as use_count drops to 0 the boost custom Deallocator gets called. This is usally not bad, as he just deregisters (decreases the use cound by 1) in the shared_ptr for context 1.Only if that use_count in context 1) would drop to 0 the object would get deleted (that's why you don't observe it to be deleted). The problem is now, that the two weak/shared ptrs (which still point to a healthy and alive object) are now disconnected. So when you try to turn the weak_ptr in context3 to a strong pointer you would get serious problems. I hope this is a correct explanation of the sitation. To solve this would need to revist the magic for shared_ptrs in boost::python. I plan to try and solve it some point later, but I am no regular developer for boost::python and I can not promise that I will succeed, nor when. -Holger On Tue, Mar 6, 2012 at 03:23, Adam Preble <adam.preble@gmail.com> wrote:

I am trying to break a smart pointer cycle in my application.  There is an interface that can be implemented in both c++ and Python, and it is used for messaging callbacks.  I can contain a list of these as shared pointers, but it only makes sense for cases where the owner of the container has exclusive control of the object in question.  These work fine in both c++ and Python.  I decided for the messaging callback list I would switch them to weak_ptr's.

I get into trouble if I register a Python implementation of the callback with ownership staying within Python.  When it comes time to communicate a message back, I see that the weak_ptr has:

use_count = 0 weak_count = 1

The pointer fails to cast.  Meanwhile, the object is looks very much alive in Python.  I have a destructor in the c++ definition interface for debugging and I see it never gets called.  Similarly, __del__ doesn't get called on the Python implementation, although I am lead to believe I can't trust it.  Perhaps most compelling is that I can continue to poke and prod at the object in the Python shell well after this failed weak_ptr cast.

I am wondering if there is anything I can do for the cast to succeed.  At this point all I can think of is making the owner of the container get a specific reference to the blasted thing, which I'd rather not have to do.

_______________________________________________ Cplusplus-sig mailing list Cplusplus-sig@python.org http://mail.python.org/mailman/listinfo/cplusplus-sig