Hi Guido and folks, On 07.10.12 17:04, Guido van Rossum wrote:
On Sat, 6 Oct 2012 17:23:48 -0700 Guido van Rossum <guido@python.org> wrote:
On Sat, Oct 6, 2012 at 3:24 PM, Antoine Pitrou <solipsis@pitrou.net> wrote:
greenlets/gevents only get you half the advantages of single-threaded "async" programming: they get you scalability in the face of a high number of concurrent connections, but they don't get you the robustness of cooperative multithreading (because it's not obvious when reading the code where the possible thread-switching points are). I used to think that too, long ago, until I discovered that as you add abstraction layers, cooperative multithreading is untenable -- sooner or later you will lose track of where the threads are switched. Even with an explicit notation like "yield" / "yield from"? If you strictly adhere to using those you should be safe (though distinguishing between the two may prove challenging) -- but in
On Sun, Oct 7, 2012 at 3:09 AM, Antoine Pitrou <solipsis@pitrou.net> wrote: practice it's hard to get everyone and every API to use this style. So you'll have some blocking API calls hidden deep inside what looks like a perfectly innocent call to some helper function.
IIUC in Go this is solved by mixing threads and lighter-weight constructs (say, greenlets) -- if a greenlet gets blocked for I/O, the rest of the system continues to make progress by spawning another thread.
My own experience with NDB is that it's just too hard to make everyone use the async APIs all the time -- so I gave up and made async APIs an optional feature, offering a blocking and an async version of every API. I didn't start out that way, but once I started writing documentation aimed at unsophisticated users, I realized that it was just too much of an uphill battle to bother.
So I think it's better to accept this and deal with it, possibly adding locking primitives into the mix that work well with the rest of the framework. Building a lock out of a tasklet-based (i.e. non-threading) Future class is easy enough.
I'm digging in, a bit late. Still trying to read the myriad of messages. For now just a word: Guido: How much I would love to use your time machine and invite you to discuss Pythons future in 1998. Then we would have tossed greenlet/stackless and all that crap. Entering a different context could have been folded deeply into Python, by making it able to pickle program state in certain positions. Just dreaming out loud :-) It is great that this discussion is taking place, and I'll try to help. cheers - Chris -- Christian Tismer :^) <mailto:tismer@stackless.com> Software Consulting : Have a break! Take a ride on Python's Karl-Liebknecht-Str. 121 : *Starship* http://starship.python.net/ 14482 Potsdam : PGP key -> http://pgp.uni-mainz.de phone +49 173 24 18 776 fax +49 (30) 700143-0023 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/