On Wed, Apr 29, 2020 at 1:52 AM Paul Moore <p.f.moore@gmail.com> wrote:
One thing that isn't at all clear to me here is that when you say "Subinterpreters run all Python code", do you *just* mean the core language? Or the core language plus all builtins? Or the core language, builtins and the standard library? Because I think that the vast majority of users would expect a core/stdlib function like subinterpreters to support the full core+stdlib language.
Agreed.
So my question would be, do all of the stdlib C extension modules support subinterpreters[1]? If they don't, then I think it's very reasonable to expect that to be fixed, in the spirit of "eating our own dogfood" - if we aren't willing or able to make the stdlib support subinterpreters, it's not exactly reasonable or fair to expect 3rd party extensions to do so.
That is definitely the right question. :) Honestly I had not thought of it that way (nor checked of course). While many stdlib modules have been updated to use heap types (see PEP 384) and support PEP 489 (Multi-phase Extension Module Initialization), there are still a few stragglers. Furthermore, I expect that there are few modules that would give us trouble (maybe ssl, cdecimal). It's all about global state that gets shared inadvertently between subinterpreters. Probably the best way to find out is to run the entire test suite in a subinterpreter. I'll do that as soon as I can.
If, on the other hand, the stdlib *is* supported, then I think that "all of Python and the stdlib, plus all 3rd party pure Python packages" is a significant base of functionality, and an entirely reasonable starting point for the feature.
Yep, that's what I meant. I just need to identify modules where we need fixes. Thanks for bringing this up!
It certainly still excludes big parts of the Python ecosystem (notably scientific / data science users) but that seems fine to me - big extension users like those can be expected to have additional limitations. It's not really that different from the situation around C extension support in PyPy.
Agreed. -eric