On Mon, Apr 20, 2020 at 4:26 PM Edwin Zimmerman <edwin@211mainstreet.net> wrote:
On 4/20/2020 6:30 PM, Nathaniel Smith wrote:
We already have robust support for threads for low-isolation and subprocesses for high-isolation. Can you name some use cases where neither of these are appropriate and you instead want an in-between isolation – like subprocesses, but more fragile and with odd edge cases where state leaks between them? I don't know if this has been mentioned before or not, but I'll bring it up now: massively concurrent networking code on Windows. Socket connections could be passed off from the main interpreter to sub-interpreters for concurrent processing that simply isn't possible with the global GIL (provided the GIL actually becomes per-interpreter). On *nix you can fork, this would give CPython on Windows similar capabilities.
Both Windows and Unix have APIs for passing sockets between related or unrelated processes -- no fork needed. On Windows, it's exposed as the socket.share method: https://docs.python.org/3/library/socket.html#socket.socket.share The APIs for managing and communicating between processes are definitely not the most obvious or simplest to use, but they're very mature and powerful, and it's a lot easier to wrap them up in a high-level API than it is to effectively reimplement process separation from scratch inside CPython. -n -- Nathaniel J. Smith -- https://vorpus.org