[Cython] PEP 3135 -- New Super

[Robert Bradshaw]

On Thu, Jul 7, 2011 at 2:13 PM, Vitja Makarov <vitja.makarov at gmail.com> wrote:
>
>
> 2011/7/8 mark florisson <markflorisson88 at gmail.com>
>>
>> On 7 July 2011 22:39, Vitja Makarov <vitja.makarov at gmail.com> wrote:
>> >
>> >
>> > 2011/7/8 mark florisson <markflorisson88 at gmail.com>
>> >>
>> >> On 7 July 2011 22:15, Vitja Makarov <vitja.makarov at gmail.com> wrote:
>> >> >
>> >> >
>> >> > 2011/7/7 mark florisson <markflorisson88 at gmail.com>
>> >> >>
>> >> >> On 6 July 2011 10:01, Vitja Makarov <vitja.makarov at gmail.com> wrote:
>> >> >> > 2011/7/6 Stefan Behnel <stefan_ml at behnel.de>:
>> >> >> >> Vitja Makarov, 06.07.2011 09:05:
>> >> >> >>>
>> >> >> >>> 2011/7/6 Stefan Behnel<stefan_ml at behnel.de>:
>> >> >> >>>>
>> >> >> >>>> Stefan Behnel, 05.07.2011 10:04:
>> >> >> >>>>>
>> >> >> >>>>> Vitja Makarov, 05.07.2011 09:17:
>> >> >> >>>>>>
>> >> >> >>>>>> 2011/7/5 Stefan Behnel:
>> >> >> >>>>>>>
>> >> >> >>>>>>> Vitja Makarov, 05.07.2011 08:21:
>> >> >> >>>>>>>>
>> >> >> >>>>>>>> I was thinking about implementing new super() with no
>> >> >> >>>>>>>> arguments.
>> >> >> >>>>>>>
>> >> >> >>>>>>> http://trac.cython.org/cython_trac/ticket/696
>> >> >> >>>>>>>
>> >> >> >>>>>>>> The problem is where to store __class__, I see two options
>> >> >> >>>>>>>> here:
>> >> >> >>>>>>>>
>> >> >> >>>>>>>> 1. Add func_class member to CyFunction, this way __class__
>> >> >> >>>>>>>> will
>> >> >> >>>>>>>> be
>> >> >> >>>>>>>> private and not visible for inner functions:
>> >> >> >>>>>>>> 2. Put it into closure
>> >> >> >>>>>>>
>> >> >> >>>>>>> The second option has the advantage of requiring the field
>> >> >> >>>>>>> only
>> >> >> >>>>>>> when
>> >> >> >>>>>>> super()
>> >> >> >>>>>>> is used, whereas the first impacts all functions.
>> >> >> >>>>>>>
>> >> >> >>>>>>> I would expect that programs commonly have a lot more
>> >> >> >>>>>>> functions
>> >> >> >>>>>>> than
>> >> >> >>>>>>> specifically methods that use a no-argument call to super(),
>> >> >> >>>>>>> so
>> >> >> >>>>>>> this
>> >> >> >>>>>>> may
>> >> >> >>>>>>> make a difference.
>> >> >> >>>>>>>
>> >> >> >>>>>>
>> >> >> >>>>>> So, now classes are created the following way:
>> >> >> >>>>>>
>> >> >> >>>>>> class_dict = {}
>> >> >> >>>>>> class_dict.foo = foo_func
>> >> >> >>>>>> class = CreateClass(class_dict)
>> >> >> >>>>>>
>> >> >> >>>>>> So after class is created I should check its dict for
>> >> >> >>>>>> CyFunction
>> >> >> >>>>>> members (maybe only ones that actually require __class__)
>> >> >> >>>>>> and set __class__:
>> >> >> >>>>>>
>> >> >> >>>>>> for value in class.__dict__.itervalues():
>> >> >> >>>>>> if isinstance(value, CyFunction) and value.func_class is
>> >> >> >>>>>> WantClass:
>> >> >> >>>>>> value.func_class = class
>> >> >> >>>>>>
>> >> >> >>>>>> Btw, first way requires cyfunction signature change, it would
>> >> >> >>>>>> accept
>> >> >> >>>>>> cyfunction object as first argument.
>> >> >> >>>>>
>> >> >> >>>>> We currently pass the binding (i.e. owning) object, right?
>> >> >> >>>>
>> >> >> >>>> So, how would this work for methods? We need to pass the 'self'
>> >> >> >>>> object
>> >> >> >>>> there, which the CyFunction doesn't know. If anything, it only
>> >> >> >>>> knows
>> >> >> >>>> the
>> >> >> >>>> class it was defined in, which doesn't help here.
>> >> >> >>>
>> >> >> >>> From PEP: "super() is equivalent to:
>> >> >> >>> super(__class__,<firstarg>)"
>> >> >> >>
>> >> >> >> I wasn't speaking of super(). What I meant, was: how do we pass
>> >> >> >> 'self'
>> >> >> >> when
>> >> >> >> we pass the CyFunction object as the first argument?
>> >> >> >>
>> >> >> >
>> >> >> >
>> >> >> > Oh, ok. Now we pass closure or nothing in self. So method's self
>> >> >> > is
>> >> >> > passed via tuple.
>> >> >> > Instancemethod do this for us. Now CyFucntion uses
>> >> >> > PyCFunction_Call
>> >> >> > we
>> >> >> > can override this and change signature of cyfunction to:
>> >> >> >
>> >> >> > PyObject func(CyFunction *func, PyObject *self, PyObject *args,
>> >> >> > PyObject *kwargs);
>> >> >> >
>> >> >> > This way we should implement new instancemethod type.
>> >> >>
>> >> >> Would it be easier to make scope objects attributes of functions?
>> >> >> Then
>> >> >> you could still utilize PyCFunction_Call without needing to check
>> >> >> the
>> >> >> argument flags and such right?
>> >> >>
>> >> >
>> >> > Sure, scope object is already functions attribute now it's stored
>> >> > inside
>> >> > self.
>> >> > But, instancemethods desc_get builds new args tuple with self as
>> >> > first
>> >> > element.
>> >> > We can implement cython version of instance method and change method
>> >> > signature a little bit.
>> >>
>> >> Right, and you pass it in as the first argument to the C function.
>> >> However, if you want to change the signature, you have to override
>> >> PyCFunction_Call, which, if I'm not mistaken, means you will have to
>> >> interpret the flags from the PyMethodDef and call the C function in
>> >> the right way (e.g. with or without the args tuple and kwargs) and
>> >> check if it's being called correctly. If you leave it as an attribute
>> >> of the function, you can pass in the function and access the scope
>> >> object from the function object in the closure C function. So I think
>> >> changing the signature might be a bit harder?
>> >>
>> >
>> > Yes, but how to handle instantmethods self arg?
>> > We can't store it inside function object as it is different for each
>> > instance.
>>
>> In descr_get you create and return a new CyFunction with a __self__
>> set (note, not m_self) and in __call__ you put __self__ in the args
>> tuple and the function as m_self, and then call the CyFunction using
>> PyCFunction_Call. But copying and modifying PyCFunction_Call works
>> just as well I suppose :)
>
> Yes, that would work) But I think CyFunction is too heavy for methods.
> I've just realized that descr_get is called each time you access method,
> each time you call it and so on.

True, but this is also the case for CPython methods and method
descriptors. There's also an inefficiency in calling bound methods in
that a new tuple needs to be created with the bound parameter as the
first component. Perhaps we could do away with that as well with
specialized unpacking code. It would simultaneously be a step forward
and backwards in terms of compatibility, but we could do away with
ever generating PyCFunctions, methods, and method descriptors. Perhaps
a CEP is in order to nail down the behavior of the CyFunction object.

- Robert