[C++-sig] call_method

[David Abrahams]

"Brett Calcott" <brett.calcott at paradise.net.nz> writes:

> From: "David Abrahams" <dave at boost-consulting.com>
> Sent: Monday, November 11, 2002 1:43 AM
>
>
>>
>> > Ok, I thought this looked weird too. But given I want:
>> >
>> > 1) An agent which I can instantiate from python but the execution of
>> > do_something happens in c++.
>> > 2) An agent which inherits all of the c++ agents properties, but can
>> > do_something in python.
>> >
>> > ...then the above code seemed the way to do it. You can stick some
>> > intermediary class in, but it is effectively empty. Hmm. I guess I
>> > am blindly following the examples.
>>
>> No, you are not. You should never expose py_agent explicitly.

Let me rephrase that, since I don't really know what you intend
py_agent to be. You should never explicitly expose a class that's
fulfilling the same role as BaseWrap does in
http://www.boost.org/libs/python/doc/tutorial/doc/class_virtual_functions.html.

> The problem is I wanted to expose agent and py_agent,

Why?

> both of which require using agent as the first template
> parameter. So this means I must use an intermediary class between
> the two - Right?

I don't have any idea what you're trying to accomplish, but no
addition of intermediary classes makes it OK to directly expose a
class fulfilling the same role as BaseWrap does in
http://www.boost.org/libs/python/doc/tutorial/doc/class_virtual_functions.html.

>> No, you should never expose a virtual funtion you intend to override
>> in Python that way. Take a look at how "default_f" is used in
>>
> http://www.boost.org/libs/python/doc/tutorial/doc/class_virtual_functions.ht
> ml.
>> If you made agent::do_something pure virtual, you wouldn't need to
>> expose it at all.
>
> Ok, I need to RTFM.

Always a good plan.

> <ponders>Hmmm. Why can't you do that? Mem fun pointers to virtual functions
> are ok. Aren't you storing them somewhere...</ponders>

Because when they're called from Python, they'll just end up calling
the override which tries to call them from Python again, and you'll
get an infinite recursion.

> I think I would blunder less if I actually knew how this stuff
> worked. I believe you are writing something about the internals - I
> look forward to reading it.

Don't hold your breath; that's not on the front burner at the moment.

> Ok, I have something that works. I don't think it resembles what you had in
> mind as I don't use extract<> or python::object. This is just a 'proof of
> concept' - I would like your feedback on what might be bad/wrong/silly about
> this.
>
> The outline is this:
> 1. I use the intrusive_ptr template in boost. This is currently
> undocumented, and I also needed to add a 'typedef T element_type' to the
> class to make it work with Boost.Python. Note that the pointee<> approach
> documented for the smart_ptr example doesn't work with msvc.

Of course not ;-)

> 2. I use a py_counted_base which is inherited by the class that is pointed
> too. I keep the PyObject *self in this class and Py_INCREF it if the use
> count of the intrusive_ptr goes above 1. (see the code)
>
> 3. I create a to_python converter that returns the PyObject *self out of the
> base class so we get object identity back in python.

Well, I think it's damned cool! I spent a bunch of time turning
approaches like this one over and over in my mind, but I couldn't see
how to make it work. At first I thought your design had to be broken,
but I failed to poke any holes in it. IIUC, the structure looks like
this:


      PyObject
    +-----------+
    | ob_refcnt |       py_agent
    +-----------+       +------------+
    | agent_ptr |------>| use_count_ |
    +-----------+       +------------+
        ^               |     pyobj_ |--+
        |               +------------+  |
        |               |     ...    |  |
        |               +------------+  |
        |                               |
        +-------------------------------+

The key is that as soon as the agent_ptr is copied, you bump
ob_refcnt. Now you know that the PyObject can't die, the only way
use_count_ can go to 1 is if all copies of the agent_ptr have been
destroyed. At that point you can decrement ob_refcnt and let Python's
regular reference management take over.

Your remarks here:

    // need to put them in the boost namespace so they get found
    // before the defaults
    namespace boost
    {
       inline void intrusive_ptr_add_ref(py_counted_base * p)
       {
          p->add_ref();
       }

       inline void intrusive_ptr_release(py_counted_base * p)
       {
          p->release();
       }
    } // end namespace boost

Are wrong, and putting these in namespace boost will not work on
conforming compilers. They need to go in the same namespace as
py_counted_base on most compilers. For compilers that don't do Koenig
lookup namespace boost will probably be OK.

I'd like to explore a few refinements with you. For example, it may be
possible to get real shared_ptr interoperability by deriving
py_counted_base from counted_base. Maybe we should ask Peter to weigh
in on this. Peter, Brett's code is attached to this page:
http://aspn.activestate.com/ASPN/Mail/Message/1430235

--
                       David Abrahams
   dave at boost-consulting.com * http://www.boost-consulting.com
Boost support, enhancements, training, and commercial distribution