[Python-Dev] Make extension module initialisation more like Python module initialisation

[Nick Coghlan]

On 6 August 2013 15:02, Stefan Behnel <stefan_ml at behnel.de> wrote:
> Alternatives I see:
>
> 1) Expose a struct that points to the extension module's PyModuleDef struct
> and the init function and expose that struct instead.
>
> 2) Expose both the PyModuleDef and the init function as public symbols.
>
> 3) Provide a public C function as entry point that returns both a
> PyModuleDef pointer and a module init function pointer.
>
> 4) Change the m_init function pointer in PyModuleDef_base from func(void)
> to func(PyObject*) iff the PyModuleDef struct is exposed as a public symbol.
>
> 5) Duplicate PyModuleDef and adapt the new one as in 4).
>
> Alternatives 1) and 2) only differ marginally by the number of public
> symbols being exposed. 3) has the advantage of supporting more advanced
> setups, e.g. heap allocation for the PyModuleDef struct. 4) is a hack and
> has the disadvantage that the signature of the module init function cannot
> be stored across reinitialisations (PyModuleDef has no "flags" or "state"
> field to remember it). 5) would fix that, i.e. we could add a proper
> pointer to the new module init function as well as a flags field for future
> extensions. A similar effect could be achieved by carefully designing the
> struct in 1).
>
> I think 1-3 are all reasonable ways to do this, although I don't think 3)
> will be necessary. 5) would be a clean fix, but has the disadvantage of
> duplicating an entire struct just to change one field in it.
>
> I'm currently leaning towards 1), with a struct that points to PyModuleDef,
> module init function and a flags field for future extensions. I understand
> that this would need to become part of the stable ABI, so explicit
> extensibility is important to keep up backwards compatibility.
>
> Opinions?

I believe a better option would be to migrate module creation over to
a dynamic PyModule_Slot and PyModule_Spec approach in the stable ABI,
similar to the one that was defined for types in PEP 384.

A related topic is that over on import-sig, we're currently tinkering
with the idea of changing the way *Python* module imports happen to
include a separate "ImportSpec" object (exact name TBC). The spec
would contain preliminary info on all of the things that the import
system can figure out *without* actually importing the module. That
list includes all the special attributes that are currently set on
modules:

    __loader__
    __name__
    __package__
    __path__
    __file__
    __cached__

(Note that the attributes on the spec *may not* be the same as those
in the module's own namespace - for example, __name__ and
__spec__.name would differ in a module executed with -m, and __path__
and __spec__.path would end up differing in packages that directly
manipulated their __path__ attribute during __init__ execution)

The intent is to clean up some of the ad hoc hackery that was needed
to make PEP 420 work, and reduce the amount of duplicated
functionality needed in loader implementations.

If you wanted to reboot this thread on import-sig, that would probably
be a good thing :)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia