[Web-SIG] PEP 444 / WSGI 2 Async

[Guido van Rossum]

On Thu, Jan 6, 2011 at 8:49 PM, P.J. Eby <pje at telecommunity.com> wrote:
> At 05:47 PM 1/6/2011 -0800, Alice Bevan­McGregor wrote:
>>
>> Tossing the idea around all day long will then, of course, be happening
>> regardless.  Unfortunately for that particular discussion, PEP 3148 /
>> Futures seems to have won out in the broader scope.
>
> Do any established async frameworks or server (e.g. Twisted, Eventlet,
> Gevent,  Tornado, etc.) make use of futures?

PEP 3148 Futures are meant for a rather different purpose than those
async frameworks. Those frameworks all are trying to minimize the
number of threads using some kind of callback-based non-blocking I/O
system. PEP 3148 OTOH doesn't care about that -- it uses threads or
processes proudly. This is useful for a different type of application,
where there are fewer, larger tasks, and the overhead of threads
doesn't matter.

The Monocle framework, which builds on top of Tornado or Twisted, uses
something not entirely unlike Futures, though they call it Callback.

I don't think the acceptance of PEP 3148 should be taken as forcing
the direction that async frameworks should take.

>>  Having a ratified and incorporated language PEP (core in 3.2 w/
>> compatibility package for 2.5 or 2.6+ support) reduces the scope of async
>> discussion down to: "how do we integrate futures into WSGI 2" instead of
>> "how do we define an async API at all".
>
> It would be helpful if you addressed the issue of scope, i.e., what features
> are you proposing to offer to the application developer.
>
> While the idea of using futures presents some intriguing possibilities, it
> seems to me at first glance that all it will do is move the point where the
> work gets done.  That is, instead of simply running the app in a worker, the
> app will be farming out work to futures.  But if this is so, then why
> doesn't the server just farm the apps themselves out to workers?
>
> I guess what I'm saying is, I haven't heard use cases for this from the
> application developer POV -- why should an app developer care about having
> their app run asynchronously?
>
> So far, I believe you're the second major proponent (i.e. ones with concrete
> proposals and/or implementations to discuss) of an async protocol...  and
> what you have in common with the other proponent is that you happen to have
> written an async server that would benefit from having apps operating
> asynchronously.  ;-)
>
> I find it hard to imagine an app developer wanting to do something
> asynchronously for which they would not want to use one of the big-dog
> asynchronous frameworks.  (Especially if their app involves database access,
> or other communications protocols.)
>
> This doesn't mean I think having a futures API is a bad thing, but ISTM that
> a futures extension to WSGI 1 could be defined right now using an x-wsgi-org
> extension in that case...  and you could then find out how many people are
> actually interested in using it.
>
> Mainly, though, what I see is people using the futures thing to shuffle off
> compute-intensive tasks...  but if they do that, then they're basically
> trying to make the server's life easier...  but under the existing spec, any
> truly async server implementing WSGI is going to run the *app* in a "future"
> of some sort already...
>
> Which means that the net result is that putting in async is like saying to
> the app developer: "hey, you know this thing that you just could do in WSGI
> 1 and the server would take care of it for you?  Well, now you can manage
> that complexity by yourself!  Isn't that wonderful?"   ;-)
>
> I could be wrong of course, but I'd like to see what concrete use cases
> people have for async.  We dropped the first discussion of async six years
> ago because someone (I think it might've been James) pointed out that, well,
> it isn't actually that useful.  And every subsequent call for use cases
> since has been answered with, "well, the use case is that you want it to be
> async."
>
> Only, that's a *server* developer's use case, not an app developer's use
> case...  and only for a minority of server developers, at that.
>
> _______________________________________________
> Web-SIG mailing list
> Web-SIG at python.org
> Web SIG: http://www.python.org/sigs/web-sig
> Unsubscribe:
> http://mail.python.org/mailman/options/web-sig/guido%40python.org
>



-- 
--Guido van Rossum (python.org/~guido)