On Sat, Oct 6, 2012 at 3:24 PM, Antoine Pitrou <solipsis@pitrou.net> wrote:
On Sat, 6 Oct 2012 15:00:54 -0700 Guido van Rossum <guido@python.org> wrote:
(2) We're at a fork in the road here. On the one hand, we could choose to deeply integrate greenlets/gevents into the standard library. (It's not monkey-patching if it's integrated, after all. :-) I'm not sure how this would work for other implementations than CPython, or even how to address CPython on non-x86 architectures. But users seem to like the programming model: write synchronous code, get async operation for free. It's easy to write protocol parsers that way. On the other hand, we could reject this approach: the integration would never be completely smooth, there's the issue of other implementations and architectures, it probably would never work smoothly even for CPython/x86 when 3rd party extension modules are involved. Callback-based APIs don't have these downsides, but they are harder to program; however we can make programming them easier by using yield-based coroutines. Even Twisted offers those (inline callbacks).
greenlets/gevents only get you half the advantages of single-threaded "async" programming: they get you scalability in the face of a high number of concurrent connections, but they don't get you the robustness of cooperative multithreading (because it's not obvious when reading the code where the possible thread-switching points are).
I used to think that too, long ago, until I discovered that as you add abstraction layers, cooperative multithreading is untenable -- sooner or later you will lose track of where the threads are switched.
(I don't actually understand the attraction of gevent, except for extreme situations; threads should be cheap on a decent OS)
I think it's the observation that the number of sockets you can realistically have open in a single process or machine is always 1-2 orders of maginuted larger than the number of threads you can have -- and this makes sense since the total amount of memory (kernel and user) to represent a socket is just much smaller than needed for a thread. Just check the configuration limits of your typical Linux kernel if you don't believe me. :-) -- --Guido van Rossum (python.org/~guido)