Re: [Python-Dev] Let's change to C API!
My first impression is that making things faster and hiding implementation details in the ABI are contrary goals. I agree with hiding implementation details in the API but not in the ABI.
For example, you mention that you want to make Py_INCREF() a function call instead of a macro. But since Py_INCREF is very common, I would guess that this would make performance worse (not by much maybe but surely measurable).
Jeroen.
2018-07-29 23:41 GMT+02:00 Jeroen Demeyer J.Demeyer@ugent.be:
For example, you mention that you want to make Py_INCREF() a function call instead of a macro. But since Py_INCREF is very common, I would guess that this would make performance worse (not by much maybe but surely measurable).
For the very specific case of Py_INCREF(), yes, I agree that performance is an issue. But I don't see how I would hide completely the PyObject structure without converting Py_INCREF() macro with a function call. (I have reasons to want to hide everything, explained in the project.)
The open question is if the cost of using function calls for Py_INCREF/DECREF versus the benefit of having the ability to modify deeply CPython internals.
I'm not sure that it's worth to bet at this point, it's too early, and we can decide that later. Moreover, it's also possible to keep Py_INCREF() as a macro in the "backward compatible" mode, but require a function call in the mode which hides all implementation details (the one where you can experiment deep CPython internals changes).
Victor
On Mon, Jul 30, 2018 at 10:46 AM, Victor Stinner vstinner@redhat.com wrote:
2018-07-29 23:41 GMT+02:00 Jeroen Demeyer J.Demeyer@ugent.be:
For example, you mention that you want to make Py_INCREF() a function call instead of a macro. But since Py_INCREF is very common, I would guess that this would make performance worse (not by much maybe but surely measurable).
For the very specific case of Py_INCREF(), yes, I agree that performance is an issue. But I don't see how I would hide completely the PyObject structure without converting Py_INCREF() macro with a function call. (I have reasons to want to hide everything, explained in the project.)
The open question is if the cost of using function calls for Py_INCREF/DECREF versus the benefit of having the ability to modify deeply CPython internals.
I'm not sure that it's worth to bet at this point, it's too early, and we can decide that later. Moreover, it's also possible to keep Py_INCREF() as a macro in the "backward compatible" mode, but require a function call in the mode which hides all implementation details (the one where you can experiment deep CPython internals changes).
If the macro and function are absolutely 100% compatible, it would be possible to set compilation to use the function by default, and have a directive that switches to using the macro. It'd improve performance at the price of locking you to the exact CPython build. So within CPython itself, there'd be no performance cost (ergo if you mess with the internals, you have to recompile all of CPython), most extension libraries would pay a small (probably immeasurable) price for compatibility, and a small handful could opt to improve performance at the price of breaking if anything changes.
ChrisA
The API leaking all implementation details will remain available as an opt-in option for Cython, cffi and debug tools. But this API will only be usable on the "slow" Python runtime, the one which keeps maximum backward compatibility. To get new optimizations, you have to use Py_INCREF() and avoid accessing C strucuture fields, which may or may not need to modify your code.
Hum, please, join the capi-sig mailing list, since I already explained that in my long reply to Stefan on capi-sig ;-)
Victor
Le lundi 30 juillet 2018, Chris Angelico rosuav@gmail.com a écrit :
On Mon, Jul 30, 2018 at 10:46 AM, Victor Stinner vstinner@redhat.com
wrote:
2018-07-29 23:41 GMT+02:00 Jeroen Demeyer J.Demeyer@ugent.be:
For example, you mention that you want to make Py_INCREF() a function
call
instead of a macro. But since Py_INCREF is very common, I would guess
that
this would make performance worse (not by much maybe but surely
measurable).
For the very specific case of Py_INCREF(), yes, I agree that performance is an issue. But I don't see how I would hide completely the PyObject structure without converting Py_INCREF() macro with a function call. (I have reasons to want to hide everything, explained in the project.)
The open question is if the cost of using function calls for Py_INCREF/DECREF versus the benefit of having the ability to modify deeply CPython internals.
I'm not sure that it's worth to bet at this point, it's too early, and we can decide that later. Moreover, it's also possible to keep Py_INCREF() as a macro in the "backward compatible" mode, but require a function call in the mode which hides all implementation details (the one where you can experiment deep CPython internals changes).
If the macro and function are absolutely 100% compatible, it would be possible to set compilation to use the function by default, and have a directive that switches to using the macro. It'd improve performance at the price of locking you to the exact CPython build. So within CPython itself, there'd be no performance cost (ergo if you mess with the internals, you have to recompile all of CPython), most extension libraries would pay a small (probably immeasurable) price for compatibility, and a small handful could opt to improve performance at the price of breaking if anything changes.
ChrisA _______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe:
https://mail.python.org/mailman/options/python-dev/vstinner%40redhat.com
On 30 Jul 2018, at 10:20, Victor Stinner vstinner@redhat.com wrote:
The API leaking all implementation details will remain available as an opt-in option for Cython, cffi and debug tools. But this API will only be usable on the "slow" Python runtime, the one which keeps maximum backward compatibility. To get new optimizations, you have to use Py_INCREF() and avoid accessing C strucuture fields, which may or may not need to modify your code.
Hum, please, join the capi-sig mailing list, since I already explained that in my long reply to Stefan on capi-sig ;-)
Interesting. I didn’t know that list exists. I’ll respond to your message on that list.
Ronald
participants (4)
-
Chris Angelico -
Jeroen Demeyer -
Ronald Oussoren -
Victor Stinner