[issue28805] Add documentation for METH_FASTCALL
New submission from Stefan Krah: It looks like METH_FASTCALL gives nice speedups (#28754). Is this an internal interface or can it be documented like METH_KEYWORDS etc.? ---------- assignee: docs@python components: Documentation messages: 281766 nosy: docs@python, haypo, serhiy.storchaka, skrah priority: normal severity: normal stage: needs patch status: open title: Add documentation for METH_FASTCALL type: enhancement versions: Python 3.6 _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue28805> _______________________________________
Serhiy Storchaka added the comment: I suggest to keep it internal. It's meaning may be changed in future. I have some ideas about making it more convenient and faster (but I'm not sure that they would work). ---------- _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue28805> _______________________________________
Stefan Krah added the comment: There are a couple of problems with using Argument Clinic for third party projects. First, it makes no stability promises: "Currently Argument Clinic is considered internal-only for CPython. ..." Then, for large projects that already use some generated code (like NumPy), IMO the result would be unmaintainable. So it would be nice to have a public stable API -- of course there is no hurry if Serhiy has plans to improve it. ---------- _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue28805> _______________________________________
Serhiy Storchaka added the comment:
Ah? Can you please elaborate your secret plan? :-)
I already said about this. Move the part of parsing keyword argument outside of the function. The caller should unpack keyword arguments and pass just a raw array of PyObject*. Missed arguments are set to NULL. This could make _PyArg_ParseStack() simpler and might even allow to inline arguments parsing code. ---------- _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue28805> _______________________________________
Barry A. Warsaw <barry@python.org> added the comment: This and the _PyObject_FastCall* APIs really do need to be documented, especially if they're going to be used in the interpreter itself. The leading underscore signifies that it's not part of the public API (well, METH_FASTCALL doesn't have a leading underscore). When these are not documented, then they aren't discoverable, so there are cases where it may be useful but will be overlooked. It also means that folks will be inclined to cargo cult the few existing uses without perhaps understanding the full semantics. Can we please document these for Python 3.7 at least? ---------- nosy: +barry versions: +Python 3.7 _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue28805> _______________________________________
Change by Barry A. Warsaw <barry@python.org>: ---------- title: Add documentation for METH_FASTCALL -> Add documentation for METH_FASTCALL and _PyObject_FastCall*() _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue28805> _______________________________________
STINNER Victor <victor.stinner@gmail.com> added the comment:
Can we please document these for Python 3.7 at least?
I chose to not document FASTCALL on purpose in Python 3.6, I wanted to keep everything private, until we get enough feedback and stabilize the API. Recently, the FASTCALL API changed: METH_FASTCALL doesn't support keywords anymore. You have to pass METH_FASTCALL|METH_KEYWORDS. It seems like the API became stable. I don't know if we should make the API public or not. PyPy would complain as usual that we give them more work to do :-D ---------- _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue28805> _______________________________________
STINNER Victor <victor.stinner@gmail.com> added the comment: I suggest to document the following 4 functions/macros: PyAPI_FUNC(PyObject *) _PyObject_FastCallDict( PyObject *callable, PyObject **args, Py_ssize_t nargs, PyObject *kwargs); PyAPI_FUNC(PyObject *) _PyObject_FastCallKeywords( PyObject *callable, PyObject **args, Py_ssize_t nargs, PyObject *kwnames); #define _PyObject_FastCall(func, args, nargs) \ _PyObject_FastCallDict((func), (args), (nargs), NULL) #define _PyObject_CallNoArg(func) \ _PyObject_FastCallDict((func), NULL, 0, NULL) And the METH_FASTCALL and METH_FASTCALL|METH_KEYWORDS calling convention. ---------- _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue28805> _______________________________________
STINNER Victor <victor.stinner@gmail.com> added the comment:
It would also be nice if Cython could use it automatically.
Cython is using FASTCALL since Python 3.6. ---------- _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue28805> _______________________________________
Jeroen Demeyer <J.Demeyer@UGent.be> added the comment: Victor, of the four functions you mentioned: - _PyObject_FastCallDict: documented by PEP 590 - _PyObject_FastCallKeywords: renamed _PyObject_Vectorcall and documented by PEP 590 - _PyObject_FastCall: not sure if it's useful enough (just use _PyObject_Vectorcall with kwnames=NULL) - _PyObject_CallNoArg: see #37194 So that leaves documenting METH_FASTCALL. ---------- nosy: +jdemeyer _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue28805> _______________________________________
Change by Jeroen Demeyer <J.Demeyer@UGent.be>: ---------- keywords: +patch pull_requests: +13938 stage: needs patch -> patch review pull_request: https://github.com/python/cpython/pull/14079 _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue28805> _______________________________________
Inada Naoki <songofacandy@gmail.com> added the comment: New changeset 5600b5e1b24a3491e83f1b3038a7ea047a34c0bf by Inada Naoki (Jeroen Demeyer) in branch 'master': bpo-28805: document METH_FASTCALL (GH-14079) https://github.com/python/cpython/commit/5600b5e1b24a3491e83f1b3038a7ea047a3... ---------- nosy: +inada.naoki _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue28805> _______________________________________
Change by miss-islington <mariatta.wijaya+miss-islington@gmail.com>: ---------- pull_requests: +13978 pull_request: https://github.com/python/cpython/pull/14136 _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue28805> _______________________________________
Change by miss-islington <mariatta.wijaya+miss-islington@gmail.com>: ---------- pull_requests: +13979 pull_request: https://github.com/python/cpython/pull/14137 _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue28805> _______________________________________
miss-islington <mariatta.wijaya+miss-islington@gmail.com> added the comment: New changeset b101fa7783615051a89500e488708b955eac94c5 by Miss Islington (bot) in branch '3.7': bpo-28805: document METH_FASTCALL (GH-14079) https://github.com/python/cpython/commit/b101fa7783615051a89500e488708b955ea... ---------- nosy: +miss-islington _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue28805> _______________________________________
miss-islington <mariatta.wijaya+miss-islington@gmail.com> added the comment: New changeset e784f9f1c3fdd2102aae3fc0fe226408ff3a6029 by Miss Islington (bot) in branch '3.8': bpo-28805: document METH_FASTCALL (GH-14079) https://github.com/python/cpython/commit/e784f9f1c3fdd2102aae3fc0fe226408ff3... ---------- _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue28805> _______________________________________
Batuhan <batuhanosmantaskaya@gmail.com> added the comment: PR 14079 has been merged, is there anything left as unresolved about this issue? ---------- nosy: +BTaskaya _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue28805> _______________________________________
Serhiy Storchaka added the comment: I suggest to keep it internal. It's meaning may be changed in future. I have some ideas about making it more convenient and faster (but I'm not sure that they would work). ---------- _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue28805> _______________________________________
Stefan Krah added the comment: There are a couple of problems with using Argument Clinic for third party projects. First, it makes no stability promises: "Currently Argument Clinic is considered internal-only for CPython. ..." Then, for large projects that already use some generated code (like NumPy), IMO the result would be unmaintainable. So it would be nice to have a public stable API -- of course there is no hurry if Serhiy has plans to improve it. ---------- _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue28805> _______________________________________
Serhiy Storchaka added the comment:
Ah? Can you please elaborate your secret plan? :-)
I already said about this. Move the part of parsing keyword argument outside of the function. The caller should unpack keyword arguments and pass just a raw array of PyObject*. Missed arguments are set to NULL. This could make _PyArg_ParseStack() simpler and might even allow to inline arguments parsing code. ---------- _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue28805> _______________________________________
Barry A. Warsaw <barry@python.org> added the comment: This and the _PyObject_FastCall* APIs really do need to be documented, especially if they're going to be used in the interpreter itself. The leading underscore signifies that it's not part of the public API (well, METH_FASTCALL doesn't have a leading underscore). When these are not documented, then they aren't discoverable, so there are cases where it may be useful but will be overlooked. It also means that folks will be inclined to cargo cult the few existing uses without perhaps understanding the full semantics. Can we please document these for Python 3.7 at least? ---------- nosy: +barry versions: +Python 3.7 _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue28805> _______________________________________
Change by Barry A. Warsaw <barry@python.org>: ---------- title: Add documentation for METH_FASTCALL -> Add documentation for METH_FASTCALL and _PyObject_FastCall*() _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue28805> _______________________________________
STINNER Victor <victor.stinner@gmail.com> added the comment:
Can we please document these for Python 3.7 at least?
I chose to not document FASTCALL on purpose in Python 3.6, I wanted to keep everything private, until we get enough feedback and stabilize the API. Recently, the FASTCALL API changed: METH_FASTCALL doesn't support keywords anymore. You have to pass METH_FASTCALL|METH_KEYWORDS. It seems like the API became stable. I don't know if we should make the API public or not. PyPy would complain as usual that we give them more work to do :-D ---------- _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue28805> _______________________________________
STINNER Victor <victor.stinner@gmail.com> added the comment: I suggest to document the following 4 functions/macros: PyAPI_FUNC(PyObject *) _PyObject_FastCallDict( PyObject *callable, PyObject **args, Py_ssize_t nargs, PyObject *kwargs); PyAPI_FUNC(PyObject *) _PyObject_FastCallKeywords( PyObject *callable, PyObject **args, Py_ssize_t nargs, PyObject *kwnames); #define _PyObject_FastCall(func, args, nargs) \ _PyObject_FastCallDict((func), (args), (nargs), NULL) #define _PyObject_CallNoArg(func) \ _PyObject_FastCallDict((func), NULL, 0, NULL) And the METH_FASTCALL and METH_FASTCALL|METH_KEYWORDS calling convention. ---------- _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue28805> _______________________________________
STINNER Victor <victor.stinner@gmail.com> added the comment:
It would also be nice if Cython could use it automatically.
Cython is using FASTCALL since Python 3.6. ---------- _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue28805> _______________________________________
Jeroen Demeyer <J.Demeyer@UGent.be> added the comment: Victor, of the four functions you mentioned: - _PyObject_FastCallDict: documented by PEP 590 - _PyObject_FastCallKeywords: renamed _PyObject_Vectorcall and documented by PEP 590 - _PyObject_FastCall: not sure if it's useful enough (just use _PyObject_Vectorcall with kwnames=NULL) - _PyObject_CallNoArg: see #37194 So that leaves documenting METH_FASTCALL. ---------- nosy: +jdemeyer _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue28805> _______________________________________
Change by Jeroen Demeyer <J.Demeyer@UGent.be>: ---------- keywords: +patch pull_requests: +13938 stage: needs patch -> patch review pull_request: https://github.com/python/cpython/pull/14079 _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue28805> _______________________________________
Inada Naoki <songofacandy@gmail.com> added the comment: New changeset 5600b5e1b24a3491e83f1b3038a7ea047a34c0bf by Inada Naoki (Jeroen Demeyer) in branch 'master': bpo-28805: document METH_FASTCALL (GH-14079) https://github.com/python/cpython/commit/5600b5e1b24a3491e83f1b3038a7ea047a3... ---------- nosy: +inada.naoki _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue28805> _______________________________________
Change by miss-islington <mariatta.wijaya+miss-islington@gmail.com>: ---------- pull_requests: +13978 pull_request: https://github.com/python/cpython/pull/14136 _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue28805> _______________________________________
Change by miss-islington <mariatta.wijaya+miss-islington@gmail.com>: ---------- pull_requests: +13979 pull_request: https://github.com/python/cpython/pull/14137 _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue28805> _______________________________________
miss-islington <mariatta.wijaya+miss-islington@gmail.com> added the comment: New changeset b101fa7783615051a89500e488708b955eac94c5 by Miss Islington (bot) in branch '3.7': bpo-28805: document METH_FASTCALL (GH-14079) https://github.com/python/cpython/commit/b101fa7783615051a89500e488708b955ea... ---------- nosy: +miss-islington _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue28805> _______________________________________
miss-islington <mariatta.wijaya+miss-islington@gmail.com> added the comment: New changeset e784f9f1c3fdd2102aae3fc0fe226408ff3a6029 by Miss Islington (bot) in branch '3.8': bpo-28805: document METH_FASTCALL (GH-14079) https://github.com/python/cpython/commit/e784f9f1c3fdd2102aae3fc0fe226408ff3... ---------- _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue28805> _______________________________________
Batuhan <batuhanosmantaskaya@gmail.com> added the comment: PR 14079 has been merged, is there anything left as unresolved about this issue? ---------- nosy: +BTaskaya _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue28805> _______________________________________
participants (8)
-
Barry A. Warsaw -
Batuhan
-
Inada Naoki
-
Jeroen Demeyer
-
miss-islington
-
Serhiy Storchaka
-
Stefan Krah
-
STINNER Victor