On 25 May 2017 at 13:30, Guido van Rossum <guido@python.org> wrote:
> Hm... Curiously, I've heard a few people at PyCon mention they thought
> subinterpreters were broken and not useful (and they share the GIL anyways)
> and should be taken out.

Taking them out entirely would break mod_wsgi (and hence the use of
Apache httpd as a Python application server), so I hope we don't
consider going down that path :)

As far as the GIL goes, Eric has a few ideas around potentially
getting to a tiered locking approach, where the GIL becomes a
Read/Write lock shared across the interpreters, and there are separate
subinterpreter locks to guard actual code execution. That becomes a
lot more feasible in a subinterpreter model, since the eval loop and
various other structures are already separate - the tiered locking
would mainly need to account for management of "object ownership" that
prevented multiple interpreters from accessing the same object at the
same time.

However, I do think subinterpreters can be accurately characterised as
fragile, especially in the presence of extension modules. I also think
a large part of that fragility can be laid at the feet of them
currently being incredibly difficult to test - while _testembed
includes a rudimentary check [1] to make sure the subinterpreter
machinery itself basically works, it doesn't do anything in the way of
checking that the rest of the standard library actually does the right
thing when run in a subinterpreter.

So I'm +1 for the idea of exposing a low-level CPython-specific
_interpreters API in order to start building out a proper test suite
for the capability, and to let folks interested in working with them
do so without having to write a custom embedding application ala
mod_wsgi.

However, I think it's still far too soon to be talking about defining
a public supported API for them - while their use in mod_wsgi gives us
assurance that they do mostly work in CPython, other implementations
don't necessarily have anything comparable (even as a private
implementation detail), and the kinds of software that folks run
directly under mod_wsgi isn't necessarily reflective of the full
extent of variation in the kinds of code that Python developers write
in general.