[Cython] Gsoc project

[Stefan Behnel]

Dag Sverre Seljebotn, 29.03.2012 05:28:
> On 03/28/2012 07:58 PM, Philip Herron wrote:
>> I am implemented a very crude and simplistic and very badly programmed
>> version of a pxd generator i think i understand what were after now
>> but i would appreciate if you look over what i did to make sure i have
>> grasped the basic idea for now:
>>
>> So if i have:
>>
>> #include "test.h"
>>
>> int add (int x, int y)
>> {
>>    return x + y;
>> }
>>
>> #ifndef __TEST_H__
>> #define __TEST_H__
>>
>> extern int add (int, int);
>>
>> struct foobar {
>>    int a;
>>    unsigned char * b;
>> };
>>
>> #endif //__TEST_H_
>>
>> We run gcc -fplugin=./python.so -fplugin-arg-python-script=walk.py test.c
>>
>> And i output out.pxd:
>>
>> cdef extern from "test.h":
>>     int add (int, int)
>>
>>     ctypedef struct foobar:
>>         int a
>>         unsigned char * b
> 
> Nice.

Yep.


> But please use 4 spaces (see PEP 8) :-)

Yep.


> More ideas for project proposal:
> 
> Another slight complication is that you should ideally turn
> 
> #define FOO 3
> #define BAR 4
> 
> into
> 
> cdef extern from "foo.h":
>     enum:
>         FOO
>         BAR
> 
> so you need to hook in before the preprocessor and after the preprocessor
> and dig out different stuff.
> 
> Then what happens if you have
> 
> #ifdef FOO
> #define BAR 3
> #else
> #define BAR 4
> #endif
> 
> ?? I'm not saying it is hard, but perhaps no longer completely trivial :-)

These things bring us close to what SWIG wants to achieve, though. I think
we should find a point where we should stop and leave it to users writing
their own supplemental .pxd file.

When generating the initial manual .pxd file, we could still move all stuff
in there that we think users should take care of. It's easy enough to
delete or comment out declarations once they're there - much easier than
wading through a header file trying to figure out what might work and what
might not.


> And like Robert hinted at, supporting all the aspects of C++ might take a
> little more work, there's just so much different syntax in C++, and there's
> several C++ features Cython just does not support and must be either
> ignored or hacked around (e.g., "const").

It would still be good to have them sort-of supported in the generator,
even if there's no declaration code coming out of it. In the specific case
of "const", we still want to support it in Cython at some point, and I'm
sure there are other things of that kind.


> Supporting stuff like
> 
> #define MACRO(x) ((x)->field*2)
> 
> probably belongs in the category of "must be done manually".

I think that's a good case for generating "something" into the editable
file (maybe even commented out) and letting the user make sense of it. We
could have standard comments for different constructs that we write into
the user file, e.g.

    ## please add types for macro MACRO, defined as
    ## MACRO(x) ((x)->field*2)
    # cdef ? MACRO(? x)

Probably also with some help text at the top of the file that shows how to
define different signatures for the same function declaration (using
explicit cnames) plus a link to the relevant docs.


>> it could be interesting porting this to other front-ends of gcc like
>> gccgo.
> 
> Does gccgo use the C ABI so that Cython could call it? If so, go for it!

Absolutely. We already have fwrap for Fortran, but if there's a way to use
GCC in order to generate glue code (i.e. not just .pxd files but also
header files and some adaptation code) that talks to even more languages,
that would be more than you could ever dream of as result of a GSoC. As
long as a language supports the C-ABI, it may even be easier to do than for
C/C++, because most languages simply don't have preprocessor macros and
similarly hard to analyse features. Even if it's just a "you have to write
your functions in this specific way in language X and only then the tool
can adapt them for you", that would still be extremely useful.

It certainly sounds like Philip has the right background for something like
that.

Stefan