It was nice to speak with many of you at the online language summit this week. It was the next best thing to seeing you all in person! :)
With the 3.9 feature freeze coming up I'm considering options for PEP
I'm looking for feedback on a nagging worry I have with merging PEP 554 into 3.9 (if it is accepted) without having the per-interpreter GIL. See below. I'll update the PEP later with relevant info from this discussion.
Many folks have conflated PEP 554 with having a per-interpreter GIL. In fact, I was careful to avoid any mention of parallelism or the GIL in the PEP. Nonetheless some are expecting that when PEP 554 lands we will reach multi-core nirvana.
While PEP 554 might be accepted and the implementation ready in time for 3.9, the separate effort toward a per-interpreter GIL is unlikely to be sufficiently done in time. That will likely happen in the next couple months (for 3.10).
So...would it be sufficiently problematic for users if we land PEP 554 in 3.9 without per-interpreter GIL?
Here are the options as I see them (if the PEP is accepted in time for 3.9):
Like I said, PEP 554 purposefully does not talk about parallelism or the GIL. Instead it focuses strictly on the following (with an emphasis on minimalism):
Regarding that first one, the value is in exposing the existing functionality to a broader group of people. keep in mind that subinterpreters currently have various other limitations (aside from sharing the GIL):
Currently, there is a chicken-and-egg problem. It is unlikely that there will be sufficient motivation to address those limitations until the community starts really using subinterpreters. Hence PEP 554.
At this point I think we should go with option #1 (or possibly #2). IMHO, we would be fine merging PEP 554 without per-interpreter GIL. Here's why:
My only concern is that folks try them out in 3.9, get frustrated by the limitations, and then there is a mental barrier to trying them again in the future when the limitations have been reduced or eliminated. However, I don't think that will be a problem for people that would benefit from serious use of subinterpreters.
Furthermore, if we're clear that "for now" subinterpreters still share the GIL in 3.9 then I'm not so worried about folks getting confused. In fact, as long as we're clear about all the current limitations then there isn't much downside to option #1.
Marking the module as "provisional" may help communicate that there are limitations that impact use.