[pypy-dev] cppyy and callbacks

[Alex Stewart]

Hi all,

So I've been working on trying to develop PyPy bindings for the Bullet
physics library (http://bulletphysics.org/) using cppyy.  So far, after a
bit of an initial learning curve, I was able to put together a
configuration that maps almost all of the standard Bullet classes and
functions, and worked surprisingly well "out of the box" (with no fixup
code or anything), and have even successfully ported a basic "hello world"
Bullet example from one of the tutorials to Python.  (I have to say, I'm
pretty impressed with how smoothly it all works!)

The one main issue I'm left with at this point, however, is that for any
sort of real application, Bullet makes substantial use of callbacks (both
in the form of some global C++ function pointers as well as some abstract
"callback" classes with virtual methods intended to be subclassed and
overridden).  My understanding is that cppyy does not currently support any
form of calling Python code from C++ (only the other way around), so this
presents a bit of a problem.

>From what I've read, there are currently efforts to switch cppyy to being
cling-based, and that this change might also allow some of this.  I was
wondering (a) how far off is this likely to be? (is it a "we'll have a beta
next week" sort of thing, or a "we haven't even started yet, and might
never get around to it" sort of thing?) and (b) will calling Python from
C++ automatically be an option as soon as the cling-based stuff is
available, or is that something that would be an additional step to add
(and thus take more time) after the basic cling-transition is done?

I've been futzing around with a workaround option in the meantime which I
think might work:

1. Wrap the C++ function pointers/virtual functions with stub code which
calls a C function pointer instead, passing C++ objects as "void *"
2. Write helper C++ functions which can accept "void *" and return a result
casted to the appropriate C++ type
3. Use cffi or ctypes to have the C function pointers call back into a
Python wrapper function, which then calls the helper conversion functions
via cppyy to convert the "void *"s back into the correct cppyy objects, and
then calls the actual Python callback function/method with those as
arguments.

Obviously, this is kinda clunky and involves a fair bit of overhead, which
I'd really like to avoid, so I'm also curious if anybody has any other
suggestions for better ways to do this sort of thing?

Any help would be much appreciated!

--Alex
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20140109/39913e11/attachment.html>