Hi all, It was nice to speak with many of you at the online language summit this week. It was the next best thing to seeing you all in person! :) With the 3.9 feature freeze coming up I'm considering options for PEP 554. I'm hopeful to have a pronouncement from Antoine in the near future. If that comes in time for 3.9, we will have the implementation ready to go. It is already nearly complete and the remaining work should be done in the next week or so. I'm looking for feedback on a nagging worry I have with merging PEP 554 into 3.9 (if it is accepted) without having the per-interpreter GIL. See below. I'll update the PEP later with relevant info from this discussion. wishing-you-were-all-here-ly yours, -eric +++++++++++++++++++++++++++++++++++++++++++++++++ (++++++++++ Dilemma ============ Many folks have conflated PEP 554 with having a per-interpreter GIL. In fact, I was careful to avoid any mention of parallelism or the GIL in the PEP. Nonetheless some are expecting that when PEP 554 lands we will reach multi-core nirvana. While PEP 554 might be accepted and the implementation ready in time for 3.9, the separate effort toward a per-interpreter GIL is unlikely to be sufficiently done in time. That will likely happen in the next couple months (for 3.10). So...would it be sufficiently problematic for users if we land PEP 554 in 3.9 without per-interpreter GIL? Options ============ Here are the options as I see them (if the PEP is accepted in time for 3.9): 1. merge PEP 554 into 3.9 even if per-interpreter GIL doesn't get into 3.9 (they get parallelism for free in 3.10) 2. like 1, but mark the module as provisional until per-interpreter GIL lands 3. do not merge PEP 554 until per-interpreter GIL is merged 4. like 3, but publish a 3.9-only module to PyPI in the meantime Context ============ Like I said, PEP 554 purposefully does not talk about parallelism or the GIL. Instead it focuses strictly on the following (with an emphasis on minimalism): * exposing the existing C-API * supporting a new concurrency model (CSP) Regarding that first one, the value is in exposing the existing functionality to a broader group of people. keep in mind that subinterpreters currently have various other limitations (aside from sharing the GIL): * nothing about them has been optimized (e.g. interpreter startup, data sharing, data passing) * extension modules with process-global state may break * some bugs still remain in the corners Currently, there is a chicken-and-egg problem. It is unlikely that there will be sufficient motivation to address those limitations until the community starts really using subinterpreters. Hence PEP 554. My Position ============ At this point I think we should go with option #1 (or possibly #2). IMHO, we would be fine merging PEP 554 without per-interpreter GIL. Here's why: * the two objectives of the PEP have value for the community (as explained in the PEP) * the sooner we can get the functionality of subinterpreters out to the broader world, the better * I don't think the lack of parallelism in 3.9 will trip anyone up My only concern is that folks try them out in 3.9, get frustrated by the limitations, and then there is a mental barrier to trying them again in the future when the limitations have been reduced or eliminated. However, I don't think that will be a problem for people that would benefit from serious use of subinterpreters. Furthermore, if we're clear that "for now" subinterpreters still share the GIL in 3.9 then I'm not so worried about folks getting confused. In fact, as long as we're clear about all the current limitations then there isn't much downside to option #1. Marking the module as "provisional" may help communicate that there are limitations that impact use.