[Import-SIG] PEP 489: Multi-phase extension module initialization; version 5
Petr Viktorin
encukou at gmail.com
Wed May 20 10:41:30 CEST 2015
On 05/20/2015 02:33 AM, Eric Snow wrote:
> On Tue, May 19, 2015 at 8:40 AM, Petr Viktorin <pviktori at redhat.com> wrote:
>> Here is an overview of how the modified importers will operate.
>> Details such as logging or handling of errors and invalid states
>> are left out, and C code is presented with a concise Python-like syntax.
>>
>> The framework that calls the importers is explained in PEP 451
>> [#pep-0451-loading]_.
>
> I know. I wrote that PEP. :)
>
>>
>> importlib/_bootstrap.py:
>>
>> class BuiltinImporter:
>> def create_module(self, spec):
>> module = _imp.create_builtin(spec)
>>
>> def exec_module(self, module):
>> _imp.exec_dynamic(module)
>>
>> def load_module(self, name):
>> # use a backwards compatibility shim
>> _load_module_shim(self, name)
>
> Won't frozen modules be likewise affected?
No, frozen modules are Python source, just not loaded from a file.
[...]
>> Python/import.c (the _imp module):
>>
>> def create_dynamic(spec):
>> name = spec.name
>> path = spec.origin
>>
>> # Find an already loaded module that used single-phase init.
>> # For multi-phase initialization, mod is NULL, so a new module
>> # is always created.
>> mod = _PyImport_FindExtensionObject(name, name)
>> if mod:
>> return mod
>>
>> return _PyImport_LoadDynamicModuleWithSpec(spec)
>>
>> def exec_dynamic(module):
>> def = PyModule_GetDef(module)
>
> This is the point where custom module types get ignored, right?
Yes. The actual code has a check for non-modules, to skip exec_dynamic
rather than have PyModule_GetDef raise. I'll add this to the overview to
make things clearer.
>> state = PyModule_GetState(module)
>> if state is NULL:
>> PyModule_ExecDef(module, def)
>
> Ah, it is idempotent.
Yes, this is the part that disables reload().
[...]
> It may also be worth outlining how PyModuleDef_Init will work.
That's hard to do in Python syntax, since most of what it does is ensure
the def is a valid PyObject. I'll explain it in a different section.
It's a very small, idempotent function:
PyObject*
PyModuleDef_Init(struct PyModuleDef* def)
{
if (def->m_base.m_index == 0) {
max_module_number++;
Py_REFCNT(def) = 1;
Py_TYPE(def) = &PyModuleDef_Type;
def->m_base.m_index = max_module_number;
}
return (PyObject*)def;
}
The code is lifted straight from PyModule_Create2.
The m_index is bookkeeping for for PyState_FindModule, so it's unused
for modules with multi-phase init, but I didn't want to break the
invariant that it's set up together with Py_TYPE.
--
Petr Viktorin
More information about the Import-SIG
mailing list