On 04/15/2016 10:59 AM, Nikita Nemkin wrote:
Thanks for your input. I now see how things evolved to the present state.
in the context of PEP 451, my proposal would have been to move all default module creation tasks to ModuleType.tp_new (taking an optional spec parameter), making separate create and exec unnecessary. Too late, I guess.
Once the question is narrowed down to "How can an extension module fully support subinterpreters and multiple Py_Initialize/Finalize cycles without incurring PEP 3121's performance overhead?" then the short answer becomes "We don't know, but ideas for that are certainly welcome, either here or over on import-sig".
I mentioned the way to avoid state access overhead in my first post. It's independent of module loading mechanism:
1) define a new "calling convention" flag like METH_GLOBALS. 2) store module ref in PyCFunctionObject.m_module (currently it stores only the module name)
Wouldn't that break backwards compatibility, though?
3) pass module ref as an extra arg to methods with METH_GLOBALS flag. 4) PyModule_State, reimplemented as a macro, would amount to one indirection from the passed parameter.
I suspect that most C ABIs allow to pass the extra arg unconditionally, (this is certainly the case for x86 and x64 on Windows and Linux). Meaning that METH_GLOBALS won't increase the actual number of possible dispatch targets in PyCFunction_Call and won't impact Python-to-C call performance at all.
My planned approach is a bit more flexible: - Add a reference to the module (ht_module) to heap types - Create a calling convention METH_METHOD, where methods are passed the class that defines the method (which might PyTYPE(self) or a superclass of it) This way methods can get both module state and the class they are defined on, and the replacement for PyModule_State is two indirections. Still, both approaches won't work with slot methods (e.g. nb_add), where there's no space in the API to add an extra argument. Nick proposed a solution in import-sig [0], which is workable but not elegant. But, I think: - METH_METHOD would be useful even if it doesn't solve the problem with slot methods. - A good solution to the slot methods problem is unlikely to render METH_METHOD obsolete. so perhaps solving the 90% case first would be OK. [0] https://mail.python.org/pipermail/import-sig/2015-July/001035.html