Last call for comments on PEP 573 (Module State Access from C Extension Methods)

Hi all, I think that PEP 573 is ready to be accepted, to greatly improve the state of extension modules in CPython 3.9. https://www.python.org/dev/peps/pep-0573/ It has come a long way since the original proposal and went through several iterations and discussions by various interested people, effectively reducing its scope quite a bit. So this is the last call for comments on the latest version of the PEP, before I will pronounce on it. Please keep the discussion in this thread. Stefan

I have a few minor copy-editing comments, but I'll submit those as a PR to the PEPs repo (it's nothing substantial, just a few wording clarifications, and making sure the list of added methods is complete). Thanks to you and everyone else for the work on this!

On Wed, 4 Dec 2019 at 09:44, Nick Coghlan <ncoghlan@gmail.com> wrote:
I have a few minor copy-editing comments, but I'll submit those as a PR to the PEPs repo (it's nothing substantial, just a few wording clarifications, and making sure the list of added methods is complete).
Belatedly working on those copy-editing updates, I noticed one semantic change I'd like to make to the PEP. At the moment PyType_GetModule() and PyType_GetModuleState() are specified as raising SystemError if they are passed: * a non-type object * a non-heap type * a heap type without ht_module set Raising TypeError would be more appropriate than raising SystemError, since this is really a standard case of "you passed in an object of a type the API wasn't expecting". A PR with this change (and the previously mentioned copy-editing updates) is up at https://github.com/python/peps/pull/1264 Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia

On 2019-11-25 13:15, Stefan Behnel wrote:
I have received substantial private feedback, and I'll need to make another update to: ## Better separate C-API additions: - PyType_FromModuleAndSpec - PyType_GetModule - PyType_DefiningTypeFromSlotFunc - PyType_GetModuleState - METH_METHOD flag - PyCMethod (function signature) from CPython implementation details: - ht_module (member of the undocumented PyHeapTypeObject) - PyCMethodObject (extends the undocumented PyCFunctionObject; used internally to hold information to pass to PyCMethod) - PyCFunction_GET_CLASS (helper for the internal class) - defining_class clinic converter (See https://mail.python.org/archives/list/capi-sig@python.org/message/B2VDVLABM4... for a proposal to formalize this distinction as "rings", and for some reasons why it is good.) ## Clarify that ht_module is not inherited ## Specify what "defining class" means more rigorously

On 2020-01-03 14:36, Petr Viktorin wrote:
The update is here: https://github.com/python/peps/pull/1275 It also includes a more drastic change: it removes the MRO walker from the proposal. Reflecting on the feedback, it became clear to me that a MRO walker, as it was described, won't give correct results in all cases: specifically, is a slot is overridden by setting a special method from Python code, the walker won't be able to find module. Think something like: c_add = Class.__add__ # where nb_add uses the MRO walker Class.__add__ = lambda *args: "evil" c_add(Class(), 0) # Exception: Defining type has not been found. This can be solved, but it will need a different design and more discussion. I'd rather defer it to the future. Meanwhile, extension authors can use their own MRO walker if they're OK with some surprising edge cases.

Hi Petr! Petr Viktorin schrieb am 14.01.20 um 14:37:
I read the last update. I can't say I'm happy about the removal since I was seeing the MRO walker function as a way to hide internals so that extension authors can start using it and CPython can adapt the internals later. But I do see that there are issues with it, and I accept your choice to keep the PEP even more minimal than it already was. Are there any more points to discuss? If not, I would soon like to accept the PEP, so that we can focus more on the implementation and usage. Stefan

I have a few minor copy-editing comments, but I'll submit those as a PR to the PEPs repo (it's nothing substantial, just a few wording clarifications, and making sure the list of added methods is complete). Thanks to you and everyone else for the work on this!

On Wed, 4 Dec 2019 at 09:44, Nick Coghlan <ncoghlan@gmail.com> wrote:
I have a few minor copy-editing comments, but I'll submit those as a PR to the PEPs repo (it's nothing substantial, just a few wording clarifications, and making sure the list of added methods is complete).
Belatedly working on those copy-editing updates, I noticed one semantic change I'd like to make to the PEP. At the moment PyType_GetModule() and PyType_GetModuleState() are specified as raising SystemError if they are passed: * a non-type object * a non-heap type * a heap type without ht_module set Raising TypeError would be more appropriate than raising SystemError, since this is really a standard case of "you passed in an object of a type the API wasn't expecting". A PR with this change (and the previously mentioned copy-editing updates) is up at https://github.com/python/peps/pull/1264 Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia

On 2019-11-25 13:15, Stefan Behnel wrote:
I have received substantial private feedback, and I'll need to make another update to: ## Better separate C-API additions: - PyType_FromModuleAndSpec - PyType_GetModule - PyType_DefiningTypeFromSlotFunc - PyType_GetModuleState - METH_METHOD flag - PyCMethod (function signature) from CPython implementation details: - ht_module (member of the undocumented PyHeapTypeObject) - PyCMethodObject (extends the undocumented PyCFunctionObject; used internally to hold information to pass to PyCMethod) - PyCFunction_GET_CLASS (helper for the internal class) - defining_class clinic converter (See https://mail.python.org/archives/list/capi-sig@python.org/message/B2VDVLABM4... for a proposal to formalize this distinction as "rings", and for some reasons why it is good.) ## Clarify that ht_module is not inherited ## Specify what "defining class" means more rigorously

On 2020-01-03 14:36, Petr Viktorin wrote:
The update is here: https://github.com/python/peps/pull/1275 It also includes a more drastic change: it removes the MRO walker from the proposal. Reflecting on the feedback, it became clear to me that a MRO walker, as it was described, won't give correct results in all cases: specifically, is a slot is overridden by setting a special method from Python code, the walker won't be able to find module. Think something like: c_add = Class.__add__ # where nb_add uses the MRO walker Class.__add__ = lambda *args: "evil" c_add(Class(), 0) # Exception: Defining type has not been found. This can be solved, but it will need a different design and more discussion. I'd rather defer it to the future. Meanwhile, extension authors can use their own MRO walker if they're OK with some surprising edge cases.

Hi Petr! Petr Viktorin schrieb am 14.01.20 um 14:37:
I read the last update. I can't say I'm happy about the removal since I was seeing the MRO walker function as a way to hide internals so that extension authors can start using it and CPython can adapt the internals later. But I do see that there are issues with it, and I accept your choice to keep the PEP even more minimal than it already was. Are there any more points to discuss? If not, I would soon like to accept the PEP, so that we can focus more on the implementation and usage. Stefan
participants (3)
-
Nick Coghlan
-
Petr Viktorin
-
Stefan Behnel