On Sun, Oct 14, 2012 at 12:27 AM, Daniel McDougall
<daniel.mcdougall@liftoffsoftware.com> wrote:
(This is a response to GVR's Google+ post asking for ideas; I
apologize in advance if I come off as an ignorant programming newbie)
-- snip snip snip --
import async # The API of the future ;)
async.async_call(retrieve_log_playback, settings, tws,
mechanism=multiprocessing)
# tws == instance of tornado.web.WebSocketHandler that holds the open connection
Is this a CPU-bound problem?
My opinion is that the goal of any async module that winds up in
Python's standard library should be simplicity and portability. In
terms of features, here's my 'async wishlist':
* I should not have to worry about what is and isn't pickleable when I
decide that a task should be performed asynchronously.
Certainly. My above question is important, because this should only matter for IPC.
* I should be able to choose the type of event loop/async mechanism
that is appropriate for the task: For CPU-bound tasks I'll probably
want to use multiprocessing. For IO-bound tasks I might want to use
threading. For a multitude of tasks that "just need to be async" (by
nature) I'll want to use an event loop.
Ehhh, maybe. This sounds like it confounds the tools for different use cases. You can quite easily have threads and processes on top of an event loop; that works out particularly nicely for processes because you still have to talk to your processes.
Examples:
twisted.internet.reactor.spawnProcess (local processes)
twisted.internet.threads.deferToThread (local threads)
ampoule (remote processes)
It's quite easy to do blocking IO in a thread with deferToThread; in fact, that's how twisted's adbapi, an async wrapper to dbapi, works.
* Any async module should support 'basics' like calling functions at
an interval and calling functions after a timeout occurs (with the
ability to cancel).
* Asynchronous tasks should be able to access the same namespace as
everything else. Maybe wishful thinking.
With twisted, this is already the case; general caveats for shared mutable state across threads of course still apply. Fortunately in most Twisted apps, that's a tiny fraction of the total code, and they tend to be fractions that are well-isolated or at least easily isolatable.
* It should support publish/subscribe-style events (i.e. an event
dispatcher). For example, the ability to watch a file descriptor or
socket for changes in state and call a function when that happens.
Preferably with the flexibility to define custom events (i.e don't
have it tied to kqueue/epoll-specific events).
Like connectionMade, connectionLost, dataReceived etc?
Thanks for your consideration; and thanks for the awesome language.
--
Dan McDougall - Chief Executive Officer and Developer
Liftoff Software ✈ Your flight to the cloud is now boarding.
904-446-8323
_______________________________________________
Python-ideas mailing list
Python-ideas@python.org
http://mail.python.org/mailman/listinfo/python-ideas