SWIG and Callbacks

[Jim]

David Abrahams wrote:
> "Jim" <jbublitzNO at SPAMnwinternet.com> wrote in message

>>Template-based code does require special treatment with sip.
>>In most cases where I've wrapped template-based types or
>>classes, I manually write C++ code for conversion C++ <-> Py
>>(sip has some functions that make that easier) and implement
>>it as a sip 'mapped type', which lets me refer to the same
>>conversion code from any return value or arg.

> That's a given with Boost.Python; with v2 conversions are automatically
> registered in a global repository so they can be re-used automatically
> across extension modules.

For C++ code with a lot of templates Boost.Python has an
advantage, particularly if you're early on the learning curve.

>>I generally
>>assign a new name (eg QList<int> -> IntList) but wouldn't
>>have to (sip will parse QList<int> and look for a mapped
>>type definition corresponding).

>>sip won't handle all possible C++ syntax, and it sometimes
>>takes some creativity to shoehorn something in, but something
>>well over 80% of h file code translates directly (after
>>removing variable names), and only 1 or 2% really takes
>>major effort.

>     .../your/ h file code...

Sure - you can extend sip in almost any way you want also
(the bindings generated are C++ code), but it may be a
little trickier fitting things in the binding's framework.
It's also possible to embed custom Python or C++ code in
sip description files and I do that when appropriate (rarely).
It kind of goes against the design philosophy of sip IMHO,
so I try to avoid it personally.

>>The real problem wouldn't seem to be C++ types as such, but
>>finding a useful (to the Python programmer) representation
>>via PyObject, and still maintaining the range of abilities
>>built into the C++ code (esp things like subclassing).

> That's built-in to Boost.Python. V1 had to implement a complicated Python
> class look-alike itself; V2 uses Python's new-style classes so subclassing
> "just works". Making virtual functions overridable in Python requires a
> small amount of boilerplate on the part of the programmer. I often wish I
> had a good C++ parser front-end which I could use to automate that part.

New-style classes lock you into Python 2.2+ though - I've
got users still at 1.5.2 and place a lot of emphasis on long
term version support. sip does the virtual code boilerplate'
automatically, unless the method requires handwritten code
for some other reason (rare). On the stuff I do, I see a lot
more virtual code than templates, but that obviously varies
a lot.

>>>>The other features that are helpful on large projects are the
>>>>built-in support for versioning

>>>How so? Can you say more about this?

>>sip uses directives: %If (A - B) ... %End to wrap a block
>>of code for versioning over a range. It could also be
>>%If (A - ) (>= A) or %If ( - B) (< B). Acceptable values
>>for A and B are defined in a %Timeline declaration; actual
>>values for A and B are passed to sip via the command line,
>>usually  derived from a version #define in the C++ source
>>code via configure (at least on Linux). Evaluation is by
>>lexical ordering of the A and B values. A and B are
>>strings like 'KDE_3_0_0'.

>>Not very different from #ifdefs, but maybe a little
>>easier.

> OK. The C++ code you're wrapping still has to get by with #ifdef, I
> suppose...

Depends on the code I suppose. Qt and KDE just issue all
new files - no versioning in the C++ code.

>>>Do you mean docstring generation?
>>
>>No - similar, but sip provides %ExportDoc ... %End directives
>>inside which you could place just about anything you want I
>>suppose. For PyQt/PyKDE the directives wrap SGML marked-up text.
>>You can construct an input file which along with the proper sip
>>command line switch, will collect and concatenate all of the
>>doc blocks distributed over the description files. The
>>concatenated file can be processed (in the case of SGML) to
>>provide HTML docs, or whatever you require.

> Oh, there are plenty of automated tools for extracting doc from C++. Lots of
> people use Doxygen with Boost.Python. No need to invent a new way to do it!

It's basically no different than Python - just %ExportDoc instead
of """, and SGML and tools are hardly new. You could use doxygen
if you wanted to (sip supports comments - I'm guessing sip files
are sufficiently parseable for doxygen) and you don't have to
use %ExportDoc. You have to generate markup either way if you
want HTML or other formats.

>>My guess is that Boost does at least some of this a little more
>>naturally. I'm not sure the extent that Boost can modify things
>>like return values or args without resorting to C++

> There is no "resorting to C++" with Boost.Python -- Boost.Python *embraces*
> C++! Since you're coding the wrappers in C++ anyway, it's trivial to make a
> "thin wrapper function" which calls the function being wrapped if you want
> to change the interface somehow. Then you just wrap the thin wrapper.

I finally went back to your website - I took your 'like an
IDL' comments a little too literally. IMHO, the fact that
Boost.Python *is* C++ is a big advantage for people who
want/need that. Personally, I don't want to write C++,
which is how I got into this in the first place.

Thin wrappers are easily implementable with sip too. In
contrast to the very large packages like PyQt and PyKDE,
they tend to be very lightweight and quick to build. I
can see ways though that I might use Boost.Python to avoid
having to choose between writing my own thin wrapper and
having to bind an entire C++ fileset. OTOH, I've written
C++ wrappers where the only difference betwen the h file
and the sip file is the addition of a %HeaderCode block
for sip, and I could write a simple py script to do that.
Doesn't happen often, but normally isn't a lot more work
than that.

>>Yes, but I don't write it - I just edit it mostly to correct
>>mistakes my description file generator creates and to handwrite
>>code where the generator has flagged that. Crude but effective (TM)
>>Even without automation, the process is easiest if you start by
>>saving the h file with a .sip extension and then (mostly) delete
>>the things that sip doesn't want/need, so it's still mostly
>>editing (sometimes A LOT of editing).

> That seems to be the key difference. It's not clear wether it's really an
> advantage, but I suppose that time will tell.

I doubt that it's ever clearly an advantage. It works very
well for PyQt and PyKDE because of the C++ style those projects
use generally - very consistent, relatively small amount of
template usage, lots of enums and of course sip has features to
support Qt. sip has been used outside of the Qt/KDE area, but
perhaps not on anything available publicly yet.

There's certainly plenty of opportunities for something like
"benchmarketing". Either one of us could contrive examples to
make one or the other look clearly better, but like most things
it's probably more rational to look at what the user actually
needs to accomplish. I've avoided actual code comparisons,
because the simple cases tend to be one-sided one way or the
other.

>>I should probably go back and review Boost. I had thought that
>>with Boost it was possible to wrap as much or as little of a
>>C++ library as you want to.

> That's true.

Big selling point IMHO.

>>I had the impression SWIG was supposed to do most of that in a single 
 >>step. Just wondering if I could implement something like that with 
Boost,
 >>or need to code a line manually for each method/class/whatever.

> If you have something which can mostly parse your headers, you can generate
> much of the wrapping code automatically.

Yes - it looks like that should be possible. Virtual code
might require a little more work in generating Boost.Python
code automatically, but the tradeoff is probably less things
flagged as requiring handwritten code.

> Yep. BTW, Boost is much more than just my Python binding library, so please,
> for the sake of the other authors: "Boost.Python" when discussing my lib.

Noted - I've tried to do better in this post.

> Not sure that you'll find much to steal; they're so different in approach...

I can be very creative when stealing other people's code :)

Actually "under the hood" sip and Boost.Python have similarities.


Jim