embedding and extending issues (with Boost.Python)

Faheem Mitha faheem at email.unc.edu
Mon Jun 28 20:44:40 CEST 2004

Dear People,

I've been trying to make Python extension modules using Boost.Python,
and I've run into some issues.  I thought it might be useful to talk
about them and ask for suggestions/opinions. Perhaps there is
something obvious I am missing. Note that the issues are not
particularly Boost.Python specific. (I don't think so, anyway.)

I'm writing an extension module, and want to call a python function,
namely array from the numarray.strings module. This is part of the
Python Numarray library. However, this could be any Python
function. The general principle still applies. The question is how to
do this.

I have tried doing this with callbacks and it seems to work fine. The
trouble with callbacks is that it means I have to have a python
wrapper with an argument slot for every function I want to
import. This is clunky.

Suppose I have three functions, say f1, f2, f3.

Then I have f1 calling f2 calling f3. 

f1 -> f2 -> f3

Suppose f3 calls some python function (say `foo') from an external
module or something.

Then if I rewrite f3 using Boost.Python, I still need a wrapper, say
f1', to pass `foo' as an argument. Then I need

f1 -> f2 ->  f3' -> f3

I suppose I don't need `foo' passed down from f1 because if I import
foo into global namespace into the same module where I define f1, f2,
f3', then Python's scoping rules will ensure that if I use `foo' as an
argument to f3' then it will be picked up from the global namespace.

Furthermore, if I decide to rewrite f2 into Boost.Python, then I would
have to create another Python wrapper f2' for f2, and pass f3' as an
argument to f2' (ugh). So I'd get the following

f1 -> f2' -> f2 -> f3' -> f3 

I'd rather have 

f1 -> f2 -> f3

where f2 and f3 only exist as C++ functions. Much cleaner.

So I thought that perhaps I could imbed the interpreter inside (say)
f3, and persuade it to make `foo' available to me inside C++. I don't
know if this is even possible. I have seen no examples of this. In
fact, I have seen no examples of using the python interpeter inside
extension modules. In fact, I've been rather unsuccessful so far in
getting anything related to embedding to work.

Am I missing something? Thanks in advance for any
information/suggestions/opinions etc. Please CC me, I'm not subscribed
to this list. I do try to read it through a news gateway, but I can't
be sure to catch everything.


More information about the Python-list mailing list