[C++-sig] Interchanging SWIG & Boost bindings.

Jacek Generowicz jacek.generowicz at cern.ch
Fri Aug 22 04:20:59 CEST 2003


[I tried posting this earlier via gmane.org, but the latter seems to
send everything I submit to /dev/null, in recent times. Apologies if
you receive this more than once.]

I'm looking into the possibility of making bindings generated by Boost
and SWIG, to be interchangable.

Here is some code to illustrate what (I think) I mean.

Suppose I have some classes written in C++:

======= simple.cpp ==============================================<<<
class foo {
  int n;
public:
  foo(int N):n(N){}
  int square() const { return n*n; }
};

class bar {
  int n;
public:
  bar(int N):n(N){}
  int baz(const foo & f) const { return n + f.square(); }
};
>>>=================================================================

and I create an isomorphic set of classes in pure Python:

======= puresimple.py ===========================================<<<
class foo:

    def __init__ (self, n):
        self.n = n

    def square (self):
        return self.n * self.n


class bar:

    def __init__ (self, n):
        self.n = n

    def baz (self, f):
        return self.n + f.square()
>>>==================================================================

I create Python extension modules by Boostifying and SWIGifying
simple.cpp (calling them boostsimple and swigsimple, respectively),
and then run the following test program:

======== test.py =================================================<<<
# Implement a pair of simple test classes (foo and bar) both in C++
# and in Python. Wrap the C++ version using both Boost and SWIG.
import boostsimple as boost
import swigsimple as swig
import puresimple as pure

# Hold instances of different implementations in separate namespaces
class namespace:
    def __init__ (self, name, implementation, fooinit, barinit):
        self.name = name
        self.foo = implementation.foo(fooinit)
        self.bar = implementation.bar(barinit)

b = namespace("Boost", boost, 2, 3)
s = namespace("SWIG ", swig , 4, 5)
p = namespace("pure ", pure , 6, 7)

# The test consists of passing a foo object to the the baz method of a
# bar object. The test_pass function takes, as arguments, the
# namespaces (implementations) from which the foo and bar objects
# should be taken, and performs the test, reporting on its actions.
def test_pass(receiver, argument):
    try:
        print "Passing %s to %s:" % (argument.name, receiver.name),
        print receiver.bar.baz(argument.foo)
    except TypeError, datum:
        print  datum

# First, test the self-consistency of each implementation
test_pass(b, b) #  7 =  3 +  4
test_pass(s, s) # 21 =  5 + 16
test_pass(p, p) # 43 =  7 + 36 
print

# Now try "cross-calling": passing arguments created in one
# implementation to methods created by a different implementation

# Pure python as receiver
test_pass(p, s) # 23 =  7 + 16
test_pass(p, b) # 11 =  7 +  4
print

# Pure python as argument
test_pass(b, p) # 39 =  3 + 36
test_pass(s, p) # 40 =  4 + 36
print

# Cross-call between different extension implementations
test_pass(b, s) # 19 =  3 + 16
test_pass(s, b) #  9 =  5 +  4
>>>=================================================================

Clearly, the first block of tests (self-consistency) just works.

The second block (pure Python as receiver) also works, because the
Python implementation of the bar.baz method is happy to receive and
use _any_ object which responds to the "square()" message (signature
polymorphism).

The remaining blocks fail, however, because the Boost and SWIG
implementations of bar.baz only accept objects of type "C++ foo
wrapped by Boost" and "C++ foo wrapped by SWIG", respectively.

There are two different issues at play; two different levels at which
one can approach the problem.

The first is that the Boost and SWIG wrappings do not recognize that
they are actually wrapping the _same_ underlying C++ class. Would it
be possible to make a Boost-wrapped foo look sufficiently like a
SWIG-wrapped foo, in order for the SWIG-wrapped bar.baz to accept it
as an argument, and to know how to access the underlying C++ object
(and vice versa)? I'm thinking along the lines of some converter
utility which takes Boost/SWIG-wrapped objects and transforms them
into ones which look like SWIG/Boost-wrapped ones, _without_ need to
re-create the original bindings.

The second, more general, approach could be to try to loosen the type
checking in the Boost and SWIG wrapped versions, in order to make them
respect signature polymorphism.

The latter is of less direct interest to me, but I suspect that it
might provide a simpler and more general way of achieving the
former. In particular, if we (see below for a brief discussion of what
"we" means) can agree, a priori, on a recommended way of creating
Boost and SWIG bindings in a way that makes them respect signature
polymorphism, later interchangeability comes for free.

The broader context of my question is the following: It has been
agreed[+] that Python should be used as the scripting language, glue
language, and "softawre bus" of the physics applications used in the
LHC[*] computing project. The number of developers involved is huge;
numerous Python bindings already exist in both Boost and SWIG, as do
long-established (and often unshakeable :-) preferences for one or
other of the two packages. We expect (numerous developers) to create
many new bindings in the near future (using both Boost and SWIG), and
we expect the underlying implementations of these packages to overlap
in some (many) places.

The end-user Physicists are being encouraged to drive their
applications in Python, with the selling point that they will be able
to access, interactively, all the software that they need, via the
bindings which are being made, and make various disjoint packages talk
to eachother. What we are trying to anticipate (and avoid) is the
situation where an end-user is happily playing with his data in an
interactive session, suddenly to be confronted with the message:

   Type error: Expected _p_Histogram

when he is passing a histogram to a function which expects a
histogram. If at this point he asks the developers for advice and gets
told that "If you want to get histograms out of package FOO and use
them in package BAR, then you have to wait for the authors of FOO to
to make a second binding of FOO in SWIG ..." (by this time he has
fallen asleep and doesn't hear the rest of the explanation), then that
would be a BAD THING (TM). However, if we can offer the response "Just
convert your histogram with the function Boost_to_SWIG before you pass
it in" (or, better still, prevent the situation from arising in the
first place), then the incident does not throw a huge spanner in the
works.

So, my questions are (sorry it took so long to get to the point!), is
there any prior art in this area?

Do you foresee any show-stopping problems?

Do you see any obvious ways of doing it? (I haven't looked into the
grubby details myself in depth yet, but I didn't want to find myself
re-inventing the wheel, so I thought I'd ask a broad question early
on.)

(If you've got this far :-) Thanks for your time !


[+] To a first-order approximation :-)

[*] http://cern.ch/lhc




More information about the Cplusplus-sig mailing list