[Python-Dev] [PEP 576/580] Comparing PEP 576 and 580

[INADA Naoki]

First of all, I'm sorry to I forgot change my mail title.

(I though about reserving one more slot for Cython for
further Cython-to-Cython call optimization, but I rejected
my idea because I'm not sure it really help Cython.)


On Mon, Jul 30, 2018 at 11:55 PM Jeroen Demeyer <J.Demeyer at ugent.be> wrote:
>
> On 2018-07-30 15:35, INADA Naoki wrote:
> > As repeatedly said, PEP 580 is very complicated protocol
> > when just implementing callable object.
>
> Can you be more concrete what you find complicated? Maybe I can improve
> the PEP to explain it more. Also, I'm open to suggestions to make it
> less complicated.

When thinking from extension writer's point of view, almost all of PEP 580 is
complicated comparing PEP 576.  Remember they don't need custom method/function
type.  So PEP 576/580 are needed only when implementing callable object,
like itemgetter or lru_cache in stdlib.

* We continue to use PyMethodDef and METH_* when writing
  tp_methods.  They should learn PyCCallDef and CCALL_* flags in addition
  to PyMethodDef and METH_*.

* In PEP 576, just put function pointer to type slot.  On the other hand, when
  implementing callable object with PEP 580, (1) Put PyCCallDef somewhere,
  (2) Put CCallRoot in instance, (3) put offset of (2) to tp_ccall.

* Difference between cc_parent and cc_self are unclear too.


I think PEP 580 is understandable only for people who tried to implement
method objects.  It's complete rewrite of PyCFunction and method_descriptor.
But extension author can write extension without knowing implementation of
them.


>
> > It is optimized for implementing custom method object, although
> > almost only Cython want the custom method type.
>
> For the record, Numba also seems interested in the PEP:
> https://groups.google.com/a/continuum.io/d/msg/numba-users/2G6k2R92MIM/P-cFKW7xAgAJ
>

OK, Numba developer interested in:

* Supporting FASTCALL for Dispatcher type:  PEP 576 is more simple
  for it as I described above.

* Direct C function calling (skip PyObject calling abstraction).
  While it's not part of PEP 580, it's strong motivation for PEP 580.

I want to see PoC of direct C calling.
And I think PoC can be implemented without waiting PEP 580.

* Cython can have specialization for CyFunction, like it have for CFunction.
  (Note that Cython doesn't utilize LOAD_METHOD / CALL_METHOD for
  CFunction too.  So lacking support for LOAD_METHOD / CALL_METHOD
  is not a big problem for now.)
* Cython can implement own C signature and embed it in CyFunction.

After that, we (including Numba, Cython, and PyPy developers) can discuss
how portable C signature can be embedded in PyCCallDef.


> > I'm not sure adding such complicated protocol almost only for Cython.
> > If CyFunction can be implemented behind PEP 576, it may be better.
>
> I recall my post
> https://mail.python.org/pipermail/python-dev/2018-July/154238.html
> explaining the main difference between PEP 576 and PEP 580.

I wrote my mail after reading the mail, of course.
But it was unclear without reading PEP and implementation carefully.
For example, "hook which part" seems meta-discussion to me before
reading your implementation.
I think only way to understand PEP 580 is reading implementation
and imagine how Cython and Numba use it.

>
> I would like to stress that PEP 580 was designed for maximum
> performance, both today and for future extensions (such as calling with
> native C types).
>

I don't know what the word *stress* mean here.  (Sorry, I'm not good at English
enough for such hard discussion).
But I want to see PoC of real benefit of PEP 580, as I said above.


> > * PEP 576 and 580 are not strictly mutually exclusive; PEP 576 may be
> >    accepted in addition to PEP 580
>
> I don't think that this is a good idea: you will mostly end up with the
> disadvantages of both approaches.
>

Hm, My point was providing easy and simple way to support FASTCALL
in callable object like functools.partial or functools.lru_cache.
But it should be discussed after PEP 580.

--
INADA Naoki  <songofacandy at gmail.com>