SWIG to the rescue (was Re: Modifying Python parms passed to C function?)

[Alex Martelli]

"Richard Harvey" <tririch at connect.net> wrote in message
news:EKvr5.3584$Jq2.8636 at dfw-read.news.verio.net...
> Ugh, I was afraid you were going to say that! The reason I'm wanting to do
> this is that I have a collection of functions that users can call through
a
> .DLL using C, or through Python using my extensions.  I would greatly
prefer
> if the interface and documentation would remain the same regardless of
what
> tool the caller is using. What bothers me most is that Python does allow
me
> to modify lists, tuples, or dictionaries using the SetItem functions, and
I

tuples are not mutable objects in Python, so I'd be surprised if any
attempts to mutate them should succeed (if it did, because Python's C
level did not make enough effort to stop a determined programmer from
thus misbehaving, the collateral damage would no doubt be much anyway).

Lists and dictionaries can be modified because they are mutable objects.
The modification does not change the object's _identity_: it changes its
_state_.  Tuples, string, and numbers are NOT mutable in Python: an
object of such a kind, with a given identity, has immutable state.

> can essentially "pass back" the modified value; why do the PyInt_,
PyFloat_,
> or PyStr_ classes not offer a "set" function from C?

To preserve the 'immutable' quality of such object, which is very
important.


> Well, I guess if that is the best Python offers I'll probably only allow a
> small subset of my API to be available through Python, because the work it
> would take to code and document 500-600 functions with specific Python
> versions seems too costly.

I think you would be well-advised to take a look at SWIG: it's a tool that
can take your C-oriented header files (maybe augmented with a reasonable
amount of SWIG-specific annotation) and generate the stubs needed for
Python interfacing (and other interfacing, too).  It's a useful tool
(like for any tool, it's important to resist the temptation to expect
a "silver bullet", for such expectations are generally disappointed
and may impede proper appreciation of tool's usefulness; still, it DOES
have the potential to help a lot with interfacing-to-Python tasks).

You can find out everything about SWIG starting from www.swig.org, but,
in particular, the handling of [in], [out], and [in,out] parameters is
supported through the "typemap" concept, which makes it sound hard...
but it really isn't.  So, here's a little example.

Suppose the C library we want to use from Python includes some
silly little function such as (in afunc.c):

int afunc(int byvalue, int* justin, int* inout, int* justout)
{
    *inout += byvalue+*justin;
    *justout = 100*(*inout);
    return 1000*(*inout);
}

To use SWIG, we prepare an interfacefile called ifunc.i, containing:

%module afunc
%include typemaps.i

extern int afunc(int, int* INPUT, int* BOTH, int* OUTPUT);


Now we run the command:

swig -python afunc.i

and swig generates afunc_wrap.c, which contains *all* we need to
wrap our afunc "library" into a Python-callable module.  It can
also generate documentation (in HTML, or Latex, or ASCII, etc),
with the help of some well-placed comments...


Now, we write a simple setup.py file, something like:

from distutils.core import setup, Extension

setup (name = "afunc",
       version = "1.0",
       maintainer = "Alex Martelli",
       maintainer_email = "aleaxit at yahoo.com",
       description = "A simple SWIG example",

       ext_modules = [
         Extension('afunc',
                   sources = ['afunc.c',
                         'afunc_wrap.c',
                         ],
            )
       ]
)

And we run
    python setup.py install

Voila -- afunc.pyd is built and placed in the Python
directory, ready for testing.  The test itself...:

D:\Python16\afunc>python
ActivePython 1.6, build 100 (ActiveState Tool Corp.)
based on Python 1.6b1 (#0, Aug 23 2000, 13:42:10) [MSC 32 bit (Intel)] on
win32
Copyright (c) Corporation for National Research Initiatives.
Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam.
>>> import afunc
>>> afunc.afunc(1,2)
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
TypeError: afunc requires exactly 3 arguments; 2 given
>>> afunc.afunc(1,2,3)
[6000, 6, 600]


We can see that the afunc wrapper was built to take exactly
3 arguments (the output-only one was omitted, as suggested)
and return a list of 3 values (the 'real' return value, first;
then, the resulting values of OUTPUT and BOTH arguments, in
order).

Note that we only work within reasonably high-level framework:
the afunc.i interface-description file (basically a C header
file, slightly augmented with a few SWIG-specific % directives
and the INPUT, BOTH, and OUTPUT 'keywords' recognized by
the typemaps.i that we %include'd); and the setup.py script
that Python's distutils require (it's a Python script so we
can do anything we need, but in practice we just call the
setup function with suitable parameters, "metadata" for the
extension we're building, including a list of sources to use).

The idea is not to have to even *look* at the generated C
wrapper source -- work, more productively, at a higher
abstraction level, by telling SWIG what it needs to know
about your code's semantics, that C itself doesn't express
clearly (which pointer-parameters are INPUT, OUTPUT, BOTH,
or, the SWIG default, "opaque" [not to be handled by script
code, but just obtained from and passed back to C functions,
much like, say, a FILE* parameter behaves in stdio]).

The required effort scales up quite reasonably when the
number of functions is in the hundreds, IMHO.  All in all,
it takes a reasonable amount of work, having developed a
large C library, to allow access to it from Python through
this avenue. The SWIG .i file would also help you offer
similar access to your library from many other languages:
Tcl, Python, Perl5, Java, and two popular dialects of Scheme
(Guile and MzScheme).

So, I do not think you need to "only allow a small subset of"
your "API to be available through Python" (or other scripting
languages).  Basically, the "documentation effort" needed is
essentially what you'll already have done for the C side... i.e.
to specify with pointers are INPUT, OUTPUT, BOTH, or opaque.
Which also basically coincides with the "coding" effort (in
the .i interfacefile you'll feed to SWIG).


Alex