Le mar. 21 avr. 2020 à 00:50, Nathaniel Smith <njs@pobox.com> a écrit :
Why do you believe that subinterpreters will have reduced resource usage? I assume you're comparing them to subprocesses here. Subinterpreters are "shared-nothing"; all code, data, etc. has to be duplicated, except for static C code ... which is exactly the same as how subprocesses work. So I don't see any theoretical reason why they should have reduced resource usage.
As you, I would like to see benchmarks of subinterpreters running in parallel before saying that "yeah, that approach works to speed up this specific use case". But I don't think that it should hold the PEP 554. Eric is transparent about his intent and limitations of the current implementation. There is no silver bullet for parallelism and concurrency. For example, multiprocessing uses threads in addition to processes, and asyncio also uses threads internally to execute blocking code like DNS resolution. IMHO it's worth it to explore the subinterpreters approach. I don't think that it's going to be a major maintenance burden: the code is already written and tests. The PEP is only about adding a public Python API.
tl;dr: accepting PEP 554 is effectively a C API break, and will force many thousands of people worldwide to spend many hours wrangling with subinterpreter support.
I fail to follow your logic. When the asyncio PEP was approved, I don't recall that suddenly the whole Python community started to rewrite all projects to use coroutines everywhere. I tried hard to replace eventlet with asyncio in OpenStack and I failed because such migration was a very large project with dubious benefits (people impacted by eventlet issues were the minority). When asyncio landed in Python 3.4, a few people started to experiment it. Some had a bad experience. Some others were excited and put a few applications in production. Even today, asyncio didn't replace threads, multiprocessing, concurrent.futures, etc. There are even competitor projects like Twisted, trio and curio! (Also eventlet and gevent based on greenlet which is a different approach). I only started to see very recently project like httpx which supports both blocking and asynchronous API. I see a slow adoption of asyncio because asyncio solves very specific use cases. And that's fine! I don't expect that everyone will suddenly spend months of work to rewrite their C code and Python code to be more efficient or fix issues with subinterpreters, until a critical mass of users proved that subinterpreters are amazing and way more efficient! Victor -- Night gathers, and now my watch begins. It shall not end until my death.