On 24 August 2013 23:19, Stefan Behnel
Nick Coghlan, 24.08.2013 13:36:
On 24 August 2013 15:51, Nick Coghlan wrote:
My current plan is to create an experimental prototype of this approach this weekend. That will include stdlib test cases, so it will also show how it looks from the extension developer's point of view.
I prototyped as much as I could without PEP 451's ModuleSpec support here:
https://bitbucket.org/ncoghlan/cpython_sandbox/commits/branch/new_extension_...
Cool. I'll take a look.
The new _PyImport_CreateAndExecExtensionModule function does the heavy lifting: https://bitbucket.org/ncoghlan/cpython_sandbox/src/081f8f7e3ee27dc309463b48e... One key point to note is that it *doesn't* call _PyImport_FixupExtensionObject, which is the API that handles all the PEP 3121 per-module state stuff. Instead, the idea will be for modules that don't need additional C level state to just implement PyImportExec_NAME, while those that *do* need C level state implement PyImportCreate_NAME and return a custom object (which may or may not be a module subtype). Such modules can still support reloading (e.g. to pick up reloaded or removed module dependencies) by providing PyImportExec_NAME as well. (in a PEP 451 world, this would likely be split up as two separate functions, one for create, one for exec)
On systems that use dynload_shlib (at least Linux & the BSDs), this branch allows extension modules to be imported if they provide a PyImportExec_NAME hook. The new hook is preferred to the existing PyInit_NAME hook, so extension modules using the stable ABI can provide both and degrade to the legacy initialisation API on older versions of Python.
Hmm, right, good call. Since both init schemes have to be part of the stable ABI, we can's rely on people compiling out one or the other. So using the old one as a fallback should work. However, only actual usage in code will tell us how it feels on user side. Supporting both in the same binary will most likely complicate things quite a bit.
It shouldn't be too bad - the PyInit_NAME fallback would just need to do the equivalent of calling PyImportCreate_NAME (or PyModule_Create if not using a custom object), call PyImportExec_NAME on it, and then return the result. Modules that genuinely *needed* the new behaviour wouldn't be able to provide a sensible fallback, and would thus be limited to Python 3.4+
The PyImportExec hook is called with a pre-created module object that the hook is then expected to populate. To aid in this task, I added two new APIs:
PyModule_SetDocString PyModule_AddFunctions
These cover setting the docstring and adding module level functions, tasks that are handled through the PyModule_Create API when using the PyInit_NAME style hook.
What are those needed for? If you subtype the module type, or provide an arbitrary extension type as implementation, you'd get these for free, wouldn't you? It's in no way different from setting up an extension type.
The idea is to let people use an import system provided module object if they don't define a custom PyImportCreate_NAME hook. Setting the docstring and adding module level functions were the two things that PyModule_Create previously handled neatly through the Py_ModuleDef struct. The two new API functions just break out those subsets as separate operations to call on the import system provided module.
The _testimportexec.c module
Where can I find that module?
Oops, forgot to add it to the repo. Uploaded now: https://bitbucket.org/ncoghlan/cpython_sandbox/src/081f8f7e3ee27dc309463b48e... Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia