Can we make METH_FASTCALL public, from Python 3.7? (ref: PEP 579
Hi, All. First of all, thank you Jeroen for writing nice PEPs. When I read PEP 579, I think "6. METH_FASTCALL is private and undocumented" should be solved first. I don't have any idea about changing METH_FASTCALL more. If Victor and Serhiy think so, and PyPy maintainers like it too, I want to make it public as soon as possible. _PyObject_FastCall* APIs are private in Python 3.7. But METH_FASTCALL is not completely private (start without underscore, but not documented) Can we call it as public, stable by adding document, if Ned allows? It's used widely in Python internals already. I suppose that making it public doesn't make Python 3.7 unstable much. If we can't at Python 3.7, I think we should do it at 3.8. Regards, -- INADA Naoki <songofacandy@gmail.com>
Hi, I chose to make it private because I wasn't sure about the API. I was right: Serhiy removed keyword arguments from METH_FASTCALL, you now have to use METH_FASTCALL | METH_KEYWORDS to also pass keyword arguments ;-) I don't recall if this change was done in 3.7 or also in 3.6. FASTCALL has been introduced in 3.6 if I recall correctly. I didn't write much documentation about FASTCALL, only 2 articles. The most interesting one: https://vstinner.github.io/fastcall-microbenchmarks.html METH_FASTCALL is already used by Cython, so I'm not sure that it's fully private :-)
_PyObject_FastCall* APIs are private in Python 3.7.
I'm not sure that it's worth it to make these functions public, they are already used internally, when using PyObject_CallFunction() for example. And it may be painful for PyPy (and others) to have to explain all these new functions.
If we can't at Python 3.7, I think we should do it at 3.8.
What's the rationale to make it public in 3.7? Can't it wait for 3.8? The new PEPs target 3.8 anyway, no? IMHO it's too late for 3.7. Victor 2018-06-20 17:42 GMT+02:00 INADA Naoki <songofacandy@gmail.com>:
Hi, All.
First of all, thank you Jeroen for writing nice PEPs.
When I read PEP 579, I think "6. METH_FASTCALL is private and undocumented" should be solved first.
I don't have any idea about changing METH_FASTCALL more. If Victor and Serhiy think so, and PyPy maintainers like it too, I want to make it public as soon as possible.
_PyObject_FastCall* APIs are private in Python 3.7. But METH_FASTCALL is not completely private (start without underscore, but not documented) Can we call it as public, stable by adding document, if Ned allows?
It's used widely in Python internals already. I suppose that making it public doesn't make Python 3.7 unstable much.
If we can't at Python 3.7, I think we should do it at 3.8.
Regards, -- INADA Naoki <songofacandy@gmail.com>
On Wed, 20 Jun 2018 18:09:00 +0200 Victor Stinner <vstinner@redhat.com> wrote:
If we can't at Python 3.7, I think we should do it at 3.8.
What's the rationale to make it public in 3.7? Can't it wait for 3.8? The new PEPs target 3.8 anyway, no?
IMHO it's too late for 3.7.
Agreed with Victor. Also Jeroen's work might lead us to change the protocol for better flexibility or performance. Let's not make it a public API too early. Regards Antoine.
2018年6月21日(木) 1:17 Antoine Pitrou <solipsis@pitrou.net>:
On Wed, 20 Jun 2018 18:09:00 +0200 Victor Stinner <vstinner@redhat.com> wrote:
If we can't at Python 3.7, I think we should do it at 3.8.
What's the rationale to make it public in 3.7? Can't it wait for 3.8? The new PEPs target 3.8 anyway, no?
IMHO it's too late for 3.7.
Agreed with Victor. Also Jeroen's work might lead us to change the protocol for better flexibility or performance.
Unless libraries are written with METH_FASTCALL (or using Cython), tp_ccall can't have any gain for 3rd party functions written in C. In other words, if many libraries start supporting FASTCALL, tp_ccall will have more gain at the time when Python 3.8 is released. Let's not make it a
public API too early.
Ok. Even though it's private at 3.7, extension authors can start using it at their risk if we decide METH_FASTCALL is public in 3.8 without any change from 3.7.
On Wed, 20 Jun 2018 at 09:49 INADA Naoki <songofacandy@gmail.com> wrote:
2018年6月21日(木) 1:17 Antoine Pitrou <solipsis@pitrou.net>:
On Wed, 20 Jun 2018 18:09:00 +0200 Victor Stinner <vstinner@redhat.com> wrote:
If we can't at Python 3.7, I think we should do it at 3.8.
What's the rationale to make it public in 3.7? Can't it wait for 3.8? The new PEPs target 3.8 anyway, no?
IMHO it's too late for 3.7.
Agreed with Victor. Also Jeroen's work might lead us to change the protocol for better flexibility or performance.
Unless libraries are written with METH_FASTCALL (or using Cython), tp_ccall can't have any gain for 3rd party functions written in C.
In other words, if many libraries start supporting FASTCALL, tp_ccall will have more gain at the time when Python 3.8 is released.
Let's not make it a
public API too early.
Ok.
Even though it's private at 3.7, extension authors can start using it at their risk if we decide METH_FASTCALL is public in 3.8 without any change from 3.7.
People can still wait for 3.8. Waiting 1.5 years for a feature is nothing when the software you're talking about is already 28 years. :) It's simply not worth the risk. Or you can push for Lukasz's idea of doing annual releases and speed it up a little. ;)
Even though it's private at 3.7, extension authors can start using it at their risk if we decide METH_FASTCALL is public in 3.8 without any change from 3.7.
People can still wait for 3.8. Waiting 1.5 years for a feature is nothing when the software you're talking about is already 28 years. :) It's simply not worth the risk.
Of course. My idea is providing information to "early adaptors" who writes C extension manually. PEP 580 is trying to expand METH_FASTCALL to custom function types in 3rd party library written in tools like Cython. But METH_FASTCALL cannot be used widely even for normal function types in 3rd party library yet. Without publicating METH_FASTCALL, PEP 580 is useful only for libraries using private APIs. That's unhealthy. So I think we should discuss about METH_FASTCALL publication before evaluating PEP 580. That's my main point, and "from 3.7" part is just a bonus, sorry. -- INADA Naoki <songofacandy@gmail.com>
20.06.18 18:42, INADA Naoki пише:
First of all, thank you Jeroen for writing nice PEPs.
When I read PEP 579, I think "6. METH_FASTCALL is private and undocumented" should be solved first.
I don't have any idea about changing METH_FASTCALL more. If Victor and Serhiy think so, and PyPy maintainers like it too, I want to make it public as soon as possible.
I don't have objections against making the METH_FASTCALL method calling convention public. But only for positional-only parameters, the protocol for keyword parameters is more complex and still can be changed. We should to provide also APIs for calling functions using this protocol (_PyObject_FastCall) and for parsing arguments (_PyArg_ParseStack). We may want to bikeshed names and the order of arguments for them.
It's used widely in Python internals already. I suppose that making it public doesn't make Python 3.7 unstable much.
If we can't at Python 3.7, I think we should do it at 3.8.
It is too late for 3.7 in any case.
2018年6月21日(木) 1:59 Serhiy Storchaka <storchaka@gmail.com>:
20.06.18 18:42, INADA Naoki пише:
First of all, thank you Jeroen for writing nice PEPs.
When I read PEP 579, I think "6. METH_FASTCALL is private and undocumented" should be solved first.
I don't have any idea about changing METH_FASTCALL more. If Victor and Serhiy think so, and PyPy maintainers like it too, I want to make it public as soon as possible.
I don't have objections against making the METH_FASTCALL method calling convention public. But only for positional-only parameters, the protocol for keyword parameters is more complex and still can be changed.
We should to provide also APIs for calling functions using this protocol (_PyObject_FastCall) and for parsing arguments (_PyArg_ParseStack). We may want to bikeshed names and the order of arguments for them.
Calling API can be determined later. Even without the API, methods can be called faster from Python core. But for parsing API, you're right. It should be public with METH_FASTCALL. Only positional arguments can be received without it.
Serhiy Storchaka schrieb am 20.06.2018 um 18:56:
20.06.18 18:42, INADA Naoki пише:
I don't have any idea about changing METH_FASTCALL more. If Victor and Serhiy think so, and PyPy maintainers like it too, I want to make it public as soon as possible.
I don't have objections against making the METH_FASTCALL method calling convention public. But only for positional-only parameters, the protocol for keyword parameters is more complex and still can be changed.
That's also the level that Cython currently uses/supports, exactly because keyword arguments are a) quite a bit more complex, b) a lot less often used and c) pretty much never used in performance critical code. Cython also currently limits the usage to Py3.6+, although I'm considering to generally enable it for everything since Py2.6 as soon as Cython starts using the calling convention for its own functions, just in case it ends up calling itself without prior notice. :) Stefan
participants (6)
-
Antoine Pitrou -
Brett Cannon -
INADA Naoki -
Serhiy Storchaka -
Stefan Behnel -
Victor Stinner