[C++-sig] bad interactions between weak_ptrs and boost.python

Arthur Peters amp at singingwizard.org
Sat Feb 5 00:51:14 CET 2005

Hello all,
Thanks for boost.python in all its greatness.

I am having 2 problems caused by boost.python's way of handling
holders that are shared_ptrs. Sorry for how verbose this is. I could not
figure out how to make it more concise.

They are both related to the way boost.python converts a python object
to a shared_ptr. It creates a completely new temporary shared_ptr that
is managing a reference to the PyObject, instead of copying the holder
(which is a shared_ptr) so that the shared_ptr is managing the actual
C++ object.

The following walks through the problems as they are demonstrated by
the attached cpp file. If you look over the cpp file first it will be
easier to understand. In fact, it may be easier to read the cpp file
and not read this stuff at all.

The first problem is as follows:

>>> v = Derived()

Here a Derived object is created and put in a shared_ptr<Derived>.

>>> SavesWeakPtrDerived( v )

Here boost.python simply passes the holder as the argument to function
because they are both of type shared_ptr<Derived>. That means that the
weak_ptr<Base> that is created does not expire ...

>>> UsesSavedWeakPtr() # works

... and the function that uses it works.

>>> SavesWeakPtr( v )

Here, however, boost.python creates a temporary shared_ptr<Base> that
only manages a reference to the python object. That means that the
weak_ptr<Base> that is created will expire as soon as the function
exits and the temporary shared_ptr goes out of scope. Because of this

>>> UsesSavedWeakPtr() # fails even though v still exists. This chrashes the program

... UsesSavedWeakPtr now fails.

The second problem is basically caused by the first and looks like this:

>>> v = Derived()

Here a Derived object is create and put in a shared_ptr<Dervied>
holder. Since Base is dervied from enable_shared_from_this, the
shared_ptr constructor updates _internal_weak_this to be connected to

>>> v.MemberUsingSharedFromThis() # works

MemberUsingSharedFromThis works fine, since shared_from_this() will
return a shared_ptr connected to the holder of the python object
(which is a shared_ptr).

>>> SavesWeakPtr( v )

Now a shared_ptr<Base> needs to be created, so boost.python creates a
temporary shared_ptr as I described above. The problem is that again
the shared_ptr constructor updates _internal_weak_this to be connected
to it, but in this case the shared_ptr only lasts until the function
call finishes, causing _internal_weak_this to expire early.

>>> v.MemberUsingSharedFromThis() # fails with "RuntimeError: boost::bad_weak_ptr"

Now when we call MemberUsingSharedFromThis it throws because
_internal_weak_this has expired even though original shared_ptr in the
python object still exists.

The best way I can see to fix this is to make the from-python
conversion of objects with a shared_ptr as a holder go directly from
the holder (a shared_ptr) to the argument (a shared_ptr) instead of
converting the shared_ptr to a dumb pointer in between (as it does
now). This way the shared_ptr that appears in C++ land is a actually
associated (shares a reference count) with the holder.

In the case of classes derived from enable_shared_from_this, the
conversion to a dumb pointer isn't even a problem. All boost.python
would need to do would be to get the shared_ptr using
shared_from_this. But this is not a general solution.

If someone has a speific idea for a solution I would be willing to give
coding it a try. However, I don't know the boost.python code so that
might not work well.

Thanks very much for your time.

PS: Yes I did post about this a couple of months ago, but I did not get
back into working on it until now, so now there are complete examples
and such.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: weak_ptr_problem_example.cpp
Type: application/octet-stream
Size: 2237 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/cplusplus-sig/attachments/20050204/f64e9918/attachment.obj>

More information about the Cplusplus-sig mailing list