[C++-sig] Marshalling Callbacks

William Trenker wtrenker at hotmail.com
Mon Dec 23 18:36:29 CET 2002

(Please excuse me if this was already posted.  I'm having trouble using 
gmane.org's fabulous service; their auto-authorization mechanism appears to 
be dropping my confirmation messages.  I'm working with them on that.)

On Sun, 22 Dec 2002 21:09:31 -0500
David Abrahams <dave at boost-consulting.com> wrote:

>"William Trenker" <wtrenker at hotmail.com> writes:
> > 3-rd party GUI libraries, like FLTK, handle external events (such as
> > mouse clicks) with a callback mechanism that is something like this
> > very sketchy pseudo-code;
> >
> > -------------------3rd Party----------------------
> > class Widget {     //low-level GUI object
> > 	void callback(void* func, void* data) {
>         void callback(void (*func)(void*), void* data)

Yes, I should have mentioned that my cooked-up example was a hybrid based on 
the FLTK API but watered down by other API's I have known.  FLTK's callback 
is, in fact, defined along the lines of your re-factoring.  Yet, there are 
many libraries out there that use void* all over the place, so I 

<snip, snip>

> > At (1) the applicaton registers the callback member function so that,
> > at (2), Widget::run knows what to call when the mouse is clicked.
>This is a somewhat regressive approach. More modern systems would
>use overridable virtual functions of a common base class that can
>package arbitrary data with the function, and avoids the type-safety
>problems associated with casting to/from void*:
>     struct callback
>     {
>         virtual void operator()() = 0;
>     };

Yes, again.  It sure is taking a long time for C++ to really take root in 
our profession.  (I never really started using C++ in it's own right until 
after I had worked for quite a while in Python and started thinking in 
classes.  As for templates?  I'm learning a lot in that department just by 
working with boost.)

>Or much better still, boost::function, which has many other

(Hmm, I'd better take some time and look at all those other boost libraries 
I've downloaded.)


> > The python equivalent might be:
> >
> > import ext
> > class A:
> >    def __init__(self):
> >       self.b = ext.Button()
> >    def onClick(self,appInfo):
> >       print "Mouse Clicked!"
> >    def execute(self,info):
> >       self.b.callback(self.onClick,info)
> > 	  self.b.run()
> > ---------------------------------------------------
>Most likely the Python equivalent would also use nullary function
>objects, which you could use lambda expressions user-defined function
>objects to bind data into... but anyway, I'll try to avoid redesigning
>your interface... ;-)

And again, yes.  I'm familiar with members like this:
class klass:
    def overloadme(self,*args):

And I'm familiar with lambda's, but not in this application.  Can you give 
me a quick example?

> > Where I'm stuck is what that marshalling mechanism looks like.  I'm
> > aware of call<> and call_method<>; I've tinkered with handle<>'s and
> > object's;  I've read the docs and studied the examples.
> >
> > Any advice or assistance is very greatly appreciated.
>Clearly you've got a problem here.  Everything in Python, including
>functions, are objects constructed at runtime.  A C++ function pointer
>refers to code which must be determined entirely at compile time.
>Q: What C++ function could you possibly write that would invoke
>different Python functions at runtime, unless you also have some
>additional data to indicate which Python function is to be invoked?
>A: None.
>So, there's no way to convert an arbitrary Python function argument
>(which could itself be a Python function) into a void (*)(void*) which
>captures the same meaning.

Agreed.  I just wanted to be sure I wasn't overlooking anything before 
rolling up my shirt sleeves.

>Fortunately, your callback interface has an additional void*
>argument. So one thing you could do is to store a pointer to the
>Python callable object in that void*.  Then you could write a thin
>wrapper as follows:
>     // helper function
>     void call_object(void* py_function)
>     {
>         PyObject* o = (PyObject*)py_function;
>         boost::python::call<void>(o);
>     }
>     // Wrap this one as a method of Widget; call it "callback":
>     void callback_wrapper(Widget& w, boost::python::object f)
>     {
>         w.callback(call_object, f.ptr());
>     }

Interesting, that's exactly what I did on my first BPL project (although it 
had only one callback to deal with).  Since any GUI, including FLTK, makes 
its living firing off callbacks maybe I should make a callback factory that 
uses one of the boost container libraries and boost::pool as a dynamic 
store.  Or I could use a BPL 'object' and employ embedded python for storage 
of these callback objects.  Does that make any sense?

>Notice that we now are back to having a nullary function interface
>(the Python function takes no arguments), because we have no
>additional arguments available to work with.
>We could do some things to get you back the ability to pass an
>additional argument if you really want to, basically by writing a C++
>binder, but even in doing so we'd run into your next big problem,
>which is an issue even if we don't write a binder: the FLTK interface
>doesn't give you any way to manage the lifetime of the callback data
>pointer.  What if all the references to the Python callback function
>disappear?  Well, then your data pointer will dangle.  This is why I
>say it's a regressive interface.
>You could cross your fingers and hope the function stays alive long
>enough, but if you're wrong you'll crash and that's not very Pythonic,
>is it?
>You could increment the reference count on f.ptr() so that it will
>leak.  Also not very Pythonic, but for most ordinary functions it
>won't matter because they're very stable.  The problem is that since
>it needs to carry its data, it's very unlikely to be an ordinary
>The problem with leaking is even worse with the "binder" solution
>which preserves your unary function interface, since a new binder gets
>created every time the callback() method is called, and they all have
>to be leaked for sure.

If I am understand correctly, your point is that when software components 
interact with each other's dynamically created objects those components 
should be handing out references to their objects using a mechanism that 
accounts for object life.

As for FLTK, like many others, there appears to be a simplifying assumption 
that if a clickable object is on the screen then the callback for that 
object must still be viable.  I guess the use of smart pointers only works 
if both ends of the pointer, so to speak, are smart-pointer aware.

>So it looks like you have to choose the least odious alternative of
>all of these.
>FWIW, this problem has basically nothing to do with Boost.Python; it's
>a problem of the FLTK interface.

And the FLTK interface is one of the better ones I've worked with.  I'm not 
complaining about BPL, or FLTK.  I'm just making sure I haven't overlooked 
something fundamental before I launch into the grunt work of hand-crafting 
wrappers for all of the many, member-laden classes that make up FLTK.

Thanks again, David, for steering me on the right BPL course.

Bill Trenker

The new MSN 8: smart spam protection and 3 months FREE*. 

More information about the Cplusplus-sig mailing list