It is not clear to me (as an end-user of Python) what is the value of exposing subinterpreters at the Python level except for multi core concurrency, but that requires one GIL per subinterpreter.

Paul Moore mentions “alternative code structuring” as a benefit, but I don’t know what that means in concrete terms.

If Paul or anyone else would make a case for the subinterpreters sharing a single GIL, that would be great.

I understand many PEPs have to do with reducing technical debt in CPython internals, which is great. But PEP 554 is about exposing a feature to Python end users, so without use cases for them I believe this PEP should be put in the back burner until such cases are articulated more clearly for the wider Python community.

Thanks,

Luciano

On Sat, Apr 18, 2020 at 07:09 Paul Moore <p.f.moore@gmail.com> wrote:
On Sat, 18 Apr 2020 at 00:03, Eric Snow <ericsnowcurrently@gmail.com> wrote:
>
> On Fri, Apr 17, 2020 at 2:59 PM Nathaniel Smith <njs@pobox.com> wrote:

> > I know you want folks to consider PEP 554 on its own merits, ignoring
> > the GIL-splitting work, but let's be realistic: purely as a
> > concurrency framework, there's at least a dozen more
> > mature/featureful/compelling options in the stdlib and on PyPI, and as
> > an isolation mechanism, subinterpreters have been around for >20 years
> > and in that time they've found 3 users and no previous champions.
> > Obviously the GIL stuff is the only reason PEP 554 might be worth
> > accepting.
>
> Saying it's "obviously" the "only" reason is a bit much. :)  PEP 554
> exposes existing functionality that hasn't been all that popular
> (until recently for some reason <wink>) mostly because it is old, was
> never publicized (until recently), and involved using the C-API.  As
> soon as folks learn about it they want it, for various reasons
> including (relative) isolation and reduced resource usage in
> large-scale deployment scenarios.  It becomes even more attractive if
> you say subinterpreters allow you to work around the GIL in a single
> process, but that isn't the only reason.

As some context in contrast to Nathaniel's view, I remember back
around 15 years ago, when I first discovered the subinterpreter
support in the C API, I was tremendously interested in it, and very
frustrated that it wasn't exposed in Python. (To be fair, I was still
using Perl at the time, and it reminded me of Perl's implementation of
fork on Windows using subinterpreters, so maybe I was primed to see
use cases for it). I never did enough extension coding to use it in C,
but I remained interested in it for some time.

This PEP is very exciting for me, precisely because it gives me a
chance to explore subinterpreters from Python, which I've been
interested in for all those years. I've not really thought of it in
terms of concurrency, so the GIL isn't key to me, but more as an
alternative code structuring option. I don't know if I'll use it much
in production code, but being able to experiment and assess the
options is something I'm looking forward to. So I for one am grateful
to you for doing this work, and I'm in favour of it being in 3.9, just
so I can get my hands on it at last.

I do think that making it initially provisional may be a sensible
option, as it gives us the option to refine the Python API while
people are still learning how to use the new model.

Paul
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/LIGC4BUYDUU7JIH7MCJLNAEQYKFW2LHX/
Code of Conduct: http://python.org/psf/codeofconduct/
--
Luciano Ramalho
|  Author of Fluent Python (O'Reilly, 2015)
|     http://shop.oreilly.com/product/0636920032519.do
|  Technical Principal at ThoughtWorks
|  Twitter: @ramalhoorg