[Import-SIG] On singleton modules, heap types, and subinterpreters

Nick Coghlan ncoghlan at gmail.com
Mon Jul 27 16:07:30 CEST 2015

On 27 July 2015 at 16:44, Stefan Behnel <stefan_ml at behnel.de> wrote:
> Petr Viktorin schrieb am 26.07.2015 um 12:39:
>> This is a follow-up to PEP 489 and discussions regarding per-module
>> data PyState_FindModule.
>> It turned out to be quite the rabbit hole. Apologies for the long
>> mail, I hope it ends up sufficiently clear.
>> Using single-phase initialization (the pre-PEP 489 solution),
>> extension modules are effectively singletons – there's up to one
>> instance of a particular module in any given subinterpreter. Cython
>> modules only allow one instance *per process*.
>> Using the new multiple-phase init, one can create several modules from
>> one PyModuleDef – either (again) one per subinterpreter, or for
>> testing purposes. As per the goal of PEP 489, this brings extension
>> modules closer to how Python modules behave.
>> The problem is that classes defined in a module don't have a reference
>> to the module object.
>> [lots of of other tricky details stripped]
> Sorry for cutting it short here, but isn't this a hint that linking the new
> initialisation to subinterpreter support might not be a good idea? I mean,
> there are a couple of advantages of the new initialisation scheme, e.g. for
> relative imports etc., which are completely unrelated to subinterpreters.
> And yet, the PEP suggests that supporting the new module setup scheme
> should indicate that subinterpreters are supported as well. Given how
> complex the support seems to be for any non-trivial module, linking the two
> use cases might end up preventing us from getting any benefit at all out of
> this for quite some time.

It shouldn't be too complicated when there aren't any custom slot
implementations involved - regular methods would just get a slightly
different signature where the defining class is also passed in.

However, I still suspect you're right that we'll end up wanting to
offer "singleton mode" to allow folks to do a two step upgrade of
extension modules, first to multi-phase initialisation, and then to
supporting subinterpreters and reloading.


Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

More information about the Import-SIG mailing list