[Python-ideas] asyncore: included batteries don't fit

[Christian Tismer]

Hi Guido and folks,

On 07.10.12 17:04, Guido van Rossum wrote:
> On Sun, Oct 7, 2012 at 3:09 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
>> On Sat, 6 Oct 2012 17:23:48 -0700
>> Guido van Rossum <guido at python.org> wrote:
>>> On Sat, Oct 6, 2012 at 3:24 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
>>>> greenlets/gevents only get you half the advantages of single-threaded
>>>> "async" programming: they get you scalability in the face of a high
>>>> number of concurrent connections, but they don't get you the robustness
>>>> of cooperative multithreading (because it's not obvious when reading
>>>> the code where the possible thread-switching points are).
>>> I used to think that too, long ago, until I discovered that as you add
>>> abstraction layers, cooperative multithreading is untenable -- sooner
>>> or later you will lose track of where the threads are switched.
>> Even with an explicit notation like "yield" / "yield from"?
> If you strictly adhere to using those you should be safe (though
> distinguishing between the two may prove challenging) -- but in
> practice it's hard to get everyone and every API to use this style. So
> you'll have some blocking API calls hidden deep inside what looks like
> a perfectly innocent call to some helper function.
>
> IIUC in Go this is solved by mixing threads and lighter-weight
> constructs (say, greenlets) -- if a greenlet gets blocked for I/O, the
> rest of the system continues to make progress by spawning another
> thread.
>
> My own experience with NDB is that it's just too hard to make everyone
> use the async APIs all the time -- so I gave up and made async APIs an
> optional feature, offering a blocking and an async version of every
> API. I didn't start out that way, but once I started writing
> documentation aimed at unsophisticated users, I realized that it was
> just too much of an uphill battle to bother.
>
> So I think it's better to accept this and deal with it, possibly
> adding locking primitives into the mix that work well with the rest of
> the framework. Building a lock out of a tasklet-based (i.e.
> non-threading) Future class is easy enough.

I'm digging in, a bit late.
Still trying to read the myriad of messages.

For now just a word:
Guido: How much I would love to use your time machine and invite
you to discuss Pythons future in 1998.

Then we would have tossed greenlet/stackless and all that crap.
Entering a different context could have been folded deeply into Python,
by making it able to pickle program state in certain positions.

Just dreaming out loud :-)
It is great that this discussion is taking place, and I'll try to help.

cheers - Chris

-- 
Christian Tismer             :^)   <mailto:tismer at stackless.com>
Software Consulting          :     Have a break! Take a ride on Python's
Karl-Liebknecht-Str. 121     :    *Starship* http://starship.python.net/
14482 Potsdam                :     PGP key -> http://pgp.uni-mainz.de
phone +49 173 24 18 776  fax +49 (30) 700143-0023
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
       whom do you want to sponsor today?   http://www.stackless.com/