[pypy-dev] Python FFI

[Stefan Behnel]

Maciej Fijalkowski, 15.05.2012 22:01:
> On Tue, May 15, 2012 at 9:22 PM, <exarkun at twistedmatrix.com> wrote:
>> I don't think it's true that using strings instead of types (or other rich
>> objects) simplifies anything.  Quite the opposite, it takes all of the
>> complexity which must exist and throws a huge wall up to prevent anyone
>> from understanding it without undertaking a huge amount of extra work.
>>
>> C is C, whether you hide the representation of its structure in a string
>> or whether you expose that representation as objects with useful, easy to
>> use APIs.
> 
> The point is that using strings you can use (a subset of) C syntax. That
> means that you need to learn only one (and limitations) and not two ways to
> express stuff. I guess.

I'm with Jean-Paul here. Moving code into strings just breaks editors,
specifically formatting and syntax highlighting, and requires special
support in IDEs in order to look up the declarations. It also blurs the
line between output, documentation and pure code internals and makes code
generally harder to analyse statically (not only for compilation).

I don't think there's anything wrong with requiring declarations for
external C code to live in external modules. It may seem like overhead for
very small C APIs, but in most cases, even the initially small ones will
grow large enough over time to require a suitable code modularisation on
user side. Even just a struct declaration (including documentation) tends
to take so much space that it's best kept out of the main source code.

So why not go all the way to move declarations into importable modules all
by themselves and do the auto-wrapping (or at least the necessary
preprocessing) in an import hook? That allows these files to use any kind
of special syntax, including plain C and C++ (with which the Cython
declaration syntax is mostly identical but easier to read and parse, BTW,
so nothing new to learn here). It also makes them available to code outside
of a single module so that they can be reused and even shipped as packages
in their own right through PyPI. Some people have already started doing
that with Cython declarations, e.g.

https://github.com/twiecki/CythonGSL

I think there's value in that.

I'm not saying that the syntax necessarily has to be exactly that of
Cython. We may eventually settle on a common declaration syntax that
includes special macro declarations and maybe even additional ABI
information. However, I think that Cython's declaration syntax has at least
largely proven its real world applicability, so something entirely new
would have to stand up against that. Also note that most major IDEs already
support this syntax, specifically PyCharm, PyDev and WingIDE.

Stefan