[C++-sig] pass function pointerto c extension function in python

[Nat Goodspeed]

________________________________________
From: c++-sig-bounces at python.org [mailto:c++-sig-bounces at python.org] On
Behalf Of Qinfeng(Javen) Shi 
Sent: Thursday, October 05, 2006 10:31 AM
To: c++-sig at python.org
Subject: Re: [C++-sig] (Stefan Seefeld)(Nat Goodspeed) pass function
pointerto Re: (Stefan Seefeld)(Nat Goodspeed) pass function pointerto c
extension function in python

[Nat] Stephan is trying to determine your functional requirements. I
blithely assumed some and tried to answer on that basis. Since you're
still asking, evidently we do not yet know what you need to be able to
do with this C++ function pointer in Python.

I assumed that you don't actually need to /call/ the function pointer
from Python: you only need Python code to /assign/ it, and the call will
happen later in C++ code. That's why I suggested that perhaps you could
represent the set of possible C++ target functions using symbolic values
in Python. You could populate a C++ map whose keys are those symbolic
values and whose values are the actual C++ function pointers. A Python
request to set the function pointer would pass a symbolic Python value.
The C++ function would look up that value and assign the corresponding
function pointer.

Stephan assumed that you might need to call the function in Python as
well as in C++. Therefore you'd use a Python callable, a first-class
Python object. Publish the whole set of possible target C++ functions
(add, minus, etc.) to Python. Don't even bother trying to represent a
C++ function pointer in Python. The Python call that sets the function
pointer will receive a boost::python::object, perhaps named something
like 'function'. Store it as a boost::python::object. When your C++ code
needs to call it, simply call function(args...).

There are several big advantages to Stephan's approach, among them:
- You can call the selected function from Python as well as from C++.
- You could later write a new Python function and select that rather
than any one of the original C++ target functions, thus extending the
functionality of process().

> 'process()' is implemented in c. It didn't lose much speed when I
> wrapped it into python. Speed is very important for it.

[Nat] I *think* you're saying here that you don't want process() to use
Python callables. You want it to directly call the C++ target function
via a C++ function pointer.

> The problem turns out to be:
> (1) I know how to wrap a function pointer op, but I don't know how
> to wrap it callable in python.

[Nat] But on the other hand, now you seem to be saying that you DO need
to be able to call the target function from Python as well as from C++.

> (2) And I know how to wrap a normal function callable in python,
> but I don't know how to wrap a function which has a function
> pointer argument. 

[Nat] Other Boosters will correct me if I'm wrong, but I do not believe
that it's possible to directly represent a C++ function pointer as a
Python data object. And I believe that's what would be necessary in
order for Boost.Python to be able to wrap a C++ function that accepts a
C++ function pointer as an argument. Therefore I do not believe it's
possible to directly wrap such a function.

I still think Stephan has the right idea. You should probably implement
it his way and do some performance testing. "Premature optimization is
the root of all evil..." Put differently, getting the algorithm right
has a much bigger performance impact than tweaking implementation
details.

But if making function calls indirectly through Python's calling
machinery turns out to slow down process() unacceptably, then here's
another thought.

- Keep the C++ function pointer and call it directly from within
process().
- Set it indirectly from Python as I described in my second paragraph
above.
- If you need to call it from Python, publish a new C++ function whose
only purpose is to call through the C++ function pointer.

In this alternative, Python never directly sees the C++ function
pointer.