[Python-Dev] SWIG (was Re: Ctypes and the stdlib)

[Eli Bendersky]

<snip>

> I've sometimes thought it might be interesting to create a Swig replacement
> purely in Python.  When I work on the PLY project, this is often what I
> think about.   In that project, I've actually built a number of the parsing
> tools that would be useful in creating such a thing.   The only catch is
> that when I start thinking along these lines, I usually reach a point where
> I say "nah, I'll just write the whole application in Python."
>
> Anyways, this is probably way more than anyone wants to know about Swig.
> Getting back to the original topic of using it to make standard library
> modules, I just don't know.   I think you probably could have some success
> with an automatic code generator of some kind.  I'm just not sure it should
> take the Swig approach of parsing C++ headers.  I think you could do better.
>
>
Dave,

Having written a full C99 parser (http://code.google.com/p/pycparser/) based
on your (excellent) PLY library, my impression is that the problem is with
the problem, not with the solution. Strange sentence, I know :-) What I mean
is that parsing C++ (even its headers) is inherently hard, which is why the
solutions tend to grow so complex. Even with the modest C99, clean and
simple solutions based on theoretical approaches (like PLY with its
generated LALR parsers) tend to run into walls [*]. C++ is an order of
magnitude harder.

If I went to implement something like SWIG today, I would almost surely base
my implementation on Clang (http://clang.llvm.org/). They have a full C++
parser (carefully hand-crafted, quite admirably keeping a relatively
comprehensible code-base for such a task) used in a real compiler front-end,
and a flexible library structure aimed at creating tools. There are also
Python bindings that would allow to do most of the interesting
Python-interface-specific work in Python - parse the C++ headers using
Clang's existing parser into ASTs - then generate ctypes / extensions from
that, *in Python*.

The community is also gladly accepting contributions. I've had some fixes
committed for the Python bindings and the C interfaces that tie them to
Clang, and got the impression from Clang's core devs that further
contributions will be most welcome. So whatever is missing from the Python
bindings can be easily added.

Eli

[*]
http://eli.thegreenplace.net/2011/05/02/the-context-sensitivity-of-c%E2%80%99s-grammar-revisited/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110829/45c4dbc5/attachment.html>