[Import-SIG] PEP 489: Redesigning extension module loading

[Petr Viktorin]

On 04/02/2015 12:17 PM, Nick Coghlan wrote:
> On 31 March 2015 at 22:49, Petr Viktorin <encukou at gmail.com> wrote:
>> On 03/30/2015 03:21 PM, Nick Coghlan wrote:
>> Also, some modules wrap a library that has global state (haven't checked
>> stdlib, but curses, readline, locale are candidates).
>> It doesn't make sense to allow loading more instances of such modules.
>> Perhaps there should be a flag to distinguish them? Or just let them use
>> PyState_AddModule/PyState_FindModule to prevent re-import?
>>
>> The need for flags would be a good argument to after all have a ModuleExport
>> structure wrapping around slots. Such a structure could also share
>> PyModuleDef_Base, making it usable with PyState_AddModule/PyState_FindModule
>> (there'd be new functions, but the machinery/data structure could be
>> reused). So I'm starting to be more inclined to do this again:
>>
>> typedef struct PyModule_Export {
>>    PyModuleDef_Base m_base;
>>    const char* m_doc;
>>    int m_flags;
>>    PyModule_Slot *m_slots; /* terminated by slot==0. */
>> } PyModule_Export;
>
> The flag variable on types has proven useful many times, so that does
> sound potentially valuable here. For the specific case you're talking
> about, we could have a "PyModule_EXPORT_SINGLETON" flag that caused
> the import machinery to automatically call
> PyState_AddModuleFromExport() (or whatever the new function was
> called)

It's needed on a lower level (_PyImport_FixupExtensionObject), but 
calling PyState_AddModuleFromExport as well sounds good.



There's another possibility I'm considering.
PyModuleDef has a "m_reload" member, which is currently unused and must 
be set to NULL. Could we repurpose that to hold the slots?
I realize renaming a member of a publicly available structure, and 
changing it from function pointer to data pointer, isn't a trivial 
change. But it shouldn;t break ABI. Maybe it can be made an union, for 
the rename case?
Doing that would mean we wouldn't need additional 
PyState_AddModule/PyModule_GetDef equivalents, modules could avoid a 
md_slots member, and things like module_dealloc would only have one 
place to look for their hooks.
In this scenario, flags could be implemented as a data-less slots, 
{Py_mod_flag_singleton, NULL}, or in a flag slot, {Py_mod_flags, &flags}.

What do you think?