[Import-SIG] On singleton modules, heap types, and subinterpreters

Petr Viktorin encukou at gmail.com
Thu Jul 30 10:01:53 CEST 2015


On Wed, Jul 29, 2015 at 11:00 PM, Eric Snow <ericsnowcurrently at gmail.com> wrote:
> On Wed, Jul 29, 2015 at 2:01 PM, Petr Viktorin <encukou at gmail.com> wrote:
>> On Wed, Jul 29, 2015 at 8:57 PM, Eric Snow <ericsnowcurrently at gmail.com> wrote:
>>> The slot methods would have to do `type(self).<"global">`, which is
>>> what any other method would do.  So in the relevant _csv.reader
>>> methods we would use the equivalent "type(self).Error" where needed.
>>
>> We need the class that defines the method. type(self) might return a
>> subclass of that. So we either need to walk the MRO until the defining
>> class is found, or use Nick's mechanism and record the defining class
>> for each relevant special method.
>
> If you explicitly bind the module-scoped object to the class that
> needs it, then the methods of that class can access it.  There's no
> need for anything more complicated (even for type slot methods).  In
> Python it would look like this:
>
>     class Error(Exception):
>         ...
>
>     class Spam:
>         Error = Error
>         def fail(self):
>             # Look up Error on Spam instead of from globals().
>             raise type(self).Error()
>
> We could even get more generic about it:
>
>     class Spam:
>         __globals__ = globals()
>         def fail(self):
>             raise self.__globals__["Error"]()
>
> Obviously it's not that simple if we are trying to provide an implicit
> Python-style scoping lookup for C extension functions/methods...

The problem is here:

base.py:
    class Error(Exception):
        "Base error"
    class Spam:
        __globals__ = globals()
        def fail(self):
            raise self.__globals__["Error"]()

other.py:
    import .base
    Error = "different error"
    class Eggs(base.Spam):
        __globals__ = globals()
    Eggs().fail()

Some kind of namespacing is needed here – fail() needs to know that it
was defined in Spam, not Eggs.
This information is not passed to special methods when they're called,
and their signatures can't be extended.

Also, module state lookup needs to be fast – MRO walking or dict
lookup would probably be too slow.

>>> However, it sounds like you're suggesting that PyState_FindModule
>>> should be fixed, replaced, or supplemented, assuming I've understood
>>> correctly that it's part of the problem here.  I'd say it depends on
>>> the actual impact of the lack of implicit Python-level scoping/lookup
>>> in C extension functions/methods.
>>
>> PyState_FindModule bypasses the need for passing a module reference
>> around: you give it a PyModuleDef (static data from which a module is
>> constructed), and from that it looks up the corresponding module in
>> current subinterpreter state.
>> This  assumes the same set of modules is loaded in every
>> subinterpreter. (Or at least that the required module is loaded in the
>> current subinterpreter.)
>> If we want to support subinterpreters safely, PyState_FindModule must go.
>
> Hence we *are* looking for any alternative lookup mechanism
> (effectively an equivalent to Python's scoping lookup).  So my
> question is, is it worth it relative to C extension functions/methods?
>
> -eric


More information about the Import-SIG mailing list