On Sat, 18 Apr 2020 at 00:03, Eric Snow <ericsnowcurrently@gmail.com> wrote:
On Fri, Apr 17, 2020 at 2:59 PM Nathaniel Smith <njs@pobox.com> wrote:
I know you want folks to consider PEP 554 on its own merits, ignoring the GIL-splitting work, but let's be realistic: purely as a concurrency framework, there's at least a dozen more mature/featureful/compelling options in the stdlib and on PyPI, and as an isolation mechanism, subinterpreters have been around for >20 years and in that time they've found 3 users and no previous champions. Obviously the GIL stuff is the only reason PEP 554 might be worth accepting.
Saying it's "obviously" the "only" reason is a bit much. :) PEP 554 exposes existing functionality that hasn't been all that popular (until recently for some reason <wink>) mostly because it is old, was never publicized (until recently), and involved using the C-API. As soon as folks learn about it they want it, for various reasons including (relative) isolation and reduced resource usage in large-scale deployment scenarios. It becomes even more attractive if you say subinterpreters allow you to work around the GIL in a single process, but that isn't the only reason.
As some context in contrast to Nathaniel's view, I remember back around 15 years ago, when I first discovered the subinterpreter support in the C API, I was tremendously interested in it, and very frustrated that it wasn't exposed in Python. (To be fair, I was still using Perl at the time, and it reminded me of Perl's implementation of fork on Windows using subinterpreters, so maybe I was primed to see use cases for it). I never did enough extension coding to use it in C, but I remained interested in it for some time. This PEP is very exciting for me, precisely because it gives me a chance to explore subinterpreters from Python, which I've been interested in for all those years. I've not really thought of it in terms of concurrency, so the GIL isn't key to me, but more as an alternative code structuring option. I don't know if I'll use it much in production code, but being able to experiment and assess the options is something I'm looking forward to. So I for one am grateful to you for doing this work, and I'm in favour of it being in 3.9, just so I can get my hands on it at last. I do think that making it initially provisional may be a sensible option, as it gives us the option to refine the Python API while people are still learning how to use the new model. Paul