It is not clear to me (as an end-user of Python) what is the value of exposing subinterpreters at the Python level except for multi core concurrency, but that requires one GIL per subinterpreter. Paul Moore mentions “alternative code structuring” as a benefit, but I don’t know what that means in concrete terms. If Paul or anyone else would make a case for the subinterpreters sharing a single GIL, that would be great. I understand many PEPs have to do with reducing technical debt in CPython internals, which is great. But PEP 554 is about exposing a feature to Python end users, so without use cases for them I believe this PEP should be put in the back burner until such cases are articulated more clearly for the wider Python community. Thanks, Luciano On Sat, Apr 18, 2020 at 07:09 Paul Moore <p.f.moore@gmail.com> wrote:
On Sat, 18 Apr 2020 at 00:03, Eric Snow <ericsnowcurrently@gmail.com> wrote:
On Fri, Apr 17, 2020 at 2:59 PM Nathaniel Smith <njs@pobox.com> wrote:
I know you want folks to consider PEP 554 on its own merits, ignoring the GIL-splitting work, but let's be realistic: purely as a concurrency framework, there's at least a dozen more mature/featureful/compelling options in the stdlib and on PyPI, and as an isolation mechanism, subinterpreters have been around for >20 years and in that time they've found 3 users and no previous champions. Obviously the GIL stuff is the only reason PEP 554 might be worth accepting.
Saying it's "obviously" the "only" reason is a bit much. :) PEP 554 exposes existing functionality that hasn't been all that popular (until recently for some reason <wink>) mostly because it is old, was never publicized (until recently), and involved using the C-API. As soon as folks learn about it they want it, for various reasons including (relative) isolation and reduced resource usage in large-scale deployment scenarios. It becomes even more attractive if you say subinterpreters allow you to work around the GIL in a single process, but that isn't the only reason.
As some context in contrast to Nathaniel's view, I remember back around 15 years ago, when I first discovered the subinterpreter support in the C API, I was tremendously interested in it, and very frustrated that it wasn't exposed in Python. (To be fair, I was still using Perl at the time, and it reminded me of Perl's implementation of fork on Windows using subinterpreters, so maybe I was primed to see use cases for it). I never did enough extension coding to use it in C, but I remained interested in it for some time.
This PEP is very exciting for me, precisely because it gives me a chance to explore subinterpreters from Python, which I've been interested in for all those years. I've not really thought of it in terms of concurrency, so the GIL isn't key to me, but more as an alternative code structuring option. I don't know if I'll use it much in production code, but being able to experiment and assess the options is something I'm looking forward to. So I for one am grateful to you for doing this work, and I'm in favour of it being in 3.9, just so I can get my hands on it at last.
I do think that making it initially provisional may be a sensible option, as it gives us the option to refine the Python API while people are still learning how to use the new model.
Paul _______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/LIGC4BUY... Code of Conduct: http://python.org/psf/codeofconduct/
-- Luciano Ramalho | Author of Fluent Python (O'Reilly, 2015) | http://shop.oreilly.com/product/0636920032519.do | Technical Principal at ThoughtWorks | Twitter: @ramalhoorg