[Cython] [cython-users] Confused by user's guide

[Robert Bradshaw]

On Mon, Aug 12, 2013 at 9:58 PM, Stefan Behnel <stefan_ml at behnel.de> wrote:
> Robert Bradshaw, 13.08.2013 06:13:
>> def do_math2(top_temp, bottom_temp):
>>     cdef int top = top_temp
>>     cdef int bottom = bottom_temp
>>     return do_math_c(top, bottom)
>>
>> cdef do_math_c(int top, int bottom):
>>     return top / bottom
>>
>> which doesn't really matter to the user except that there may be some
>> cases (in the future) where we can extract and call do_math_c
>> directly.
>>
>> Of course, this is exactly what cpdef is, which has got me thinking
>> whether we need a separate keyword for this, or if we could simply
>> make all def functions cpdef automatically. One place that's different
>> is for exported cdef classes where c(p)def methods must be declared in
>> the .pxd file, but we could perhaps handle this case by allowing them
>> to be declared as plain cdef (or "def") methods there. Does anyone see
>> any drawbacks to this? (I suppose there's a little overhead in the
>> check-if-overridden dispatch.)
>
> We could detect which def functions/methods are actually being called
> inside of the module, and then only drop those calls into C. We already do
> this for final classes.

Yep. We could dynamically expose/detect this across modules as well
(including callables from projects like numba once we define a common
ABI).

> I think the main problem is the way cpdef is currently implemented. It
> declares a cdef function with a def wrapper. If it declared a def function
> instead, it would be easier to do the switch later in the pipeline for
> normal def functions. But as you noted, if we just unified that into always
> declaring the two entry points up-front, for both def and cpdef functions,
> preferably using the existing signature override mechanism, then we'd have
> the means to switch at any point.

Yes, I'm imagining we'd unify this by always creating def functions
and which optionally C entry point as well. Maybe cpdef would become a
synonym for def and the language could be thus simplified. (It is
somewhat annoying that "cdef" and "def" are parsed so differently.) We
would need to augment def to allow a return type as well as the gil
and except clauses (so it's more like getting rid of def and renaming
cpdef to def).

> Trying to do this across modules is less straight forward, as it would
> override existing user declarations in .pxd files (if they exist). It could
> be done if we automatically generate a .pxd from a .pyx file, though.

Ideally, you could declare everything as cdef in your pxd file, and
make a choice as to whether to use cdef or def in your pyx file. Those
def functions not declared in the pxd file just wouldn't get a slot in
the vtab. This may not be possible (without additional overhead) due
to the need to support overloading in Python space whenever a
superclass creates a def wrapper, so perhaps we would allow cdef and
def members in the class declaration.

When pxd files are generated from pyx, or one does cimports from pyx
directly, this becomes even less of an issue.

- Robert