On 5/25/2020 8:29 AM, redradist@gmail.com wrote:
Edwin Zimmerman wrote:
Edwin Zimmerman wrote: On 5/25/2020 5:56 AM, redradist@gmail.com wrote: Edwin Zimmerman wrote: Sub-interpreters are a work in progress. The API is not anywhere near being finalized. True parallel execution will occur only if the GIL is changed to a per interpreter lock, and that requires some rather large changes to Python that haven't happened yet. Why ? True parallel execution is possible with sub-interpreters !! Sub-interpretter should be run in separate thread not controlled by GIL and sub-interpretter will notify main interpreter with atomic variable that it is finished (like worker) !! We just need to provide two APIs: 1) Synchronized: run, run_string, that will wait until thread notify with setting atomic variable in true that it finished 2) Asynchronized (based on async): run_async, run_string_async, event_loop will wait on atomic variable reading it periodically All sub-interpreters and all threads are subject to the GIL, that is why it is called the GLOBAL Interpreter Lock. There is no such thing as a "separate thread not controlled by GIL." This is not a problem with the sub-interpreter API. It is a design decision embedded very deeply through the entire CPython code. It is not trivial to change this. However, there is work being done on this. Search the python-dev mailing
On Monday, May 25, 2020 redradist@gmail.com [mailto:redradist@gmail.com] wrote list for the "PoC: Subinterpreters 4x faster than sequential execution or threads on CPU-bound workaround" thread if you want to see more. --Edwin Maybe I did not know something ... but the reason that sub-interpreter was added, because of separate GIL in each sub-interpreter ? No, there isn't a separate GIL for each sub-interpreter. That would be a great feature in my mind, but it isn't there.
--Edwin
Am I right ? We should not bind all interpreters with main interpreter ... only with atomic or condition variable ... otherwise sub-interpreters is not needed at all !!