[Cython] non-virtual methods

[Vitja Makarov]

2011/8/30 Robert Bradshaw <robertwb at math.washington.edu>:
> On Sun, Aug 28, 2011 at 4:00 AM, Vitja Makarov <vitja.makarov at gmail.com> wrote:
>> 2011/8/27 Vitja Makarov <vitja.makarov at gmail.com>:
>>> 2011/8/27 Vitja Makarov <vitja.makarov at gmail.com>:
>>>> 2011/8/26 Vitja Makarov <vitja.makarov at gmail.com>:
>>>>> 2011/8/25 Vitja Makarov <vitja.makarov at gmail.com>:
>>>>>> 2011/8/25 Vitja Makarov <vitja.makarov at gmail.com>:
>>>>>>> 2011/8/25 Stefan Behnel <stefan_ml at behnel.de>:
>>>>>>>> Vitja Makarov, 25.08.2011 20:32:
>>>>>>>>>
>>>>>>>>> 2011/8/25 Stefan Behnel<stefan_ml at behnel.de>:
>>>>>>>>>>
>>>>>>>>>> Vitja Makarov, 25.08.2011 18:11:
>>>>>>>>>>>
>>>>>>>>>>> 2011/8/24 Stefan Behnel:
>>>>>>>>>>>>
>>>>>>>>>>>> Vitja Makarov, 24.08.2011 21:17:
>>>>>>>>>>>>>
>>>>>>>>>>>>> I tried final classes:
>>>>>>>>>>>>> 2. In this example foo call is done through virtual table
>>>>>>>>>>>>>
>>>>>>>>>>>>> cimport cython
>>>>>>>>>>>>>
>>>>>>>>>>>>> @cython.final
>>>>>>>>>>>>> cdef class Foo:
>>>>>>>>>>>>>     cdef foo(self):
>>>>>>>>>>>>>         print 'haha'
>>>>>>>>>>>>>
>>>>>>>>>>>>> def test():
>>>>>>>>>>>>>     cdef Foo a = Foo()
>>>>>>>>>>>>>     a.foo()
>>>>>>>>>>>>>
>>>>>>>>>>>>>   __pyx_t_1 = ((struct __pyx_vtabstruct_3yyy_Foo
>>>>>>>>>>>>> *)__pyx_v_a->__pyx_vtab)->foo(__pyx_v_a); if (unlikely(!__pyx_t_1))
>>>>>>>>>>>>> {__pyx_filename = __pyx_f[0]; __pyx_lineno = 12; __pyx_clineno =
>>>>>>>>>>>>> __LINE__; goto __pyx_L1_error;}
>>>>>>>>>>>>
>>>>>>>>>>>> Right, this is not implemented yet. Feel free to do so. Also see
>>>>>>>>>>>>
>>>>>>>>>>>> http://trac.cython.org/cython_trac/ticket/474
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>> 3. I can't use final decorator for methods (error reported)
>>>>>>>>>>>>
>>>>>>>>>>>> http://trac.cython.org/cython_trac/ticket/586
>>>>>>>>>>>
>>>>>>>>>>> What is the prefered syntax keyword inline or final decorator?
>>>>>>>>>>
>>>>>>>>>> "final" is the right option here.
>>>>>>>>>>
>>>>>>>>>> They are orthogonal concepts. Only because you declare a method "final"
>>>>>>>>>> does
>>>>>>>>>> not mean you want to inline it, and just because you declare it "inline"
>>>>>>>>>> does not (necessarily) mean that you cannot override it. Admittedly, the
>>>>>>>>>> semantics of an overridable inline method may turn out to be somewhat
>>>>>>>>>> obscure and error prone, so I think it's a good idea to let "inline"
>>>>>>>>>> imply
>>>>>>>>>> "final". But not the other way round.
>>>>>>>>>
>>>>>>>>> But both inline and final methods should bypass vtab, right?
>>>>>>>>
>>>>>>>> Yes. But in the "final" case, it's always clear which method implementation
>>>>>>>> to use - it's not overridable, so there is only one choice. In the "inline"
>>>>>>>> case, it could still be overridable and we may have a subtype of the
>>>>>>>> declared type in our hands at runtime, thus choosing the wrong method at
>>>>>>>> compile time. That's why only the "final" case is safe.
>>>>>>>>
>>>>>>>> Note that I'm only talking about the semantics of the qualifier themselves
>>>>>>>> here. If we allow "inline" methods, I think we should force them to be
>>>>>>>> "final" as well. But that's a practical choice, not a semantic implication.
>>>>>>>>
>>>>>>>>
>>>>>>>>> Also I'm not sure about C inline qualifier here.
>>>>>>>>
>>>>>>>> That's what "inline" requests.
>>>>>>>>
>>>>>>>>
>>>>>>>>> I see three options:
>>>>>>>>>
>>>>>>>>>  - non-virtual: bypass vtab
>>>>>>>>>  - final: non-virtual, non-overridable
>>>>>>>>
>>>>>>>> How would you want to bypass the vtable in the "non-virtual" case if the
>>>>>>>> method is overridable?
>>>>>>>>
>>>>>>>>
>>>>>>>>>  - inline: non-virtual, C inline qualifier is used
>>>>>>>>
>>>>>>>> Correct.
>>>>>>>>
>>>>>>>
>>>>>>> Ok.
>>>>>>>
>>>>>>> I think it's better to implement final method then user could choose
>>>>>>> to use inline qualifier or not.
>>>>>>>
>>>>>>
>>>>>> I tried it here:
>>>>>> https://github.com/vitek/cython/commit/ddf80a80dc75aced2cd92dc32afa77a7bcf2de02
>>>>>>
>>>>>> There is one problem: vtab bypassing should be enabled if final method
>>>>>> is defined in the same module.
>>>>>> I don't know how to check that final method comes from cimport (it's
>>>>>> okay with pxd, the problem is pyx)
>>>>>>
>>>>>
>>>>>
>>>>> https://github.com/vitek/cython/commit/6e91fc257a683018ba6be340d384f9a7c34ef425
>>>>>
>>>>> Here is update version. I've add tree asserts and final method's prototypes.
>>>>>
>>>>> --
>>>>> vitja.
>>>>>
>>>>
>>>> I've created ticket for compiler crash when cython.final is used
>>>> inside pxd file:
>>>>
>>>> http://trac.cython.org/cython_trac/ticket/722
>>>>
>>>> Also I've updated final methods test case (added tree path assertions)
>>>> https://github.com/vitek/cython/commit/92edb09419c9b77a792f7c43e6ddd760b00c4e74
>>>>
>>>>
>>>> About declaration origin detection may be it's a good idea to have a
>>>> flag at scope level something like is_pxd_scope or
>>>> is_declaration_scope?
>>>>
>>>
>>> It seems to me that I found a way to fix pxd/pyx cimport problem. I've
>>> created pull request:
>>>
>>> https://github.com/cython/cython/pull/59
>>>
>>
>> I've add support for inline methods, now you can declare inline method
>> in pxd file:
>>
>> cdef class Foo:
>>   cdef inline foo(self):
>>       return 1
>
> Cool.
>
> I suppose for cross-module calls, non-inline methods vtables are best
> one can do short of actually linking the modules together.

Sure. This is how it works now. But noitice that final isn't supported
now at pxd scope. So it might not work.


https://github.com/cython/cython/pull/59/files#L5R1888

For subclassing I check that both 3parent and children are in the same scope.
When pxd is cimported vtable bypassing should be disabled in case it's
cimported from anpther module.
Not sure how to implement this. I think we should first fix final at pxd.

> One
> question about your code--I'm not seeing how you're disallowing
> overriding final cpdef methods from Python. (Should we even allow
> final cpdef methods on non-final classes?)
>

I think it's hard and tricky to disallow cpdef overriding but I think
it's better to have cpdef final methods.
And I wouldn't argue if you say that final cpdef methods shouldn't be
allowed at all.

-- 
vitja.