On Sun, Oct 14, 2012 at 5:32 AM, Laurens Van Houtven _@lvh.cc wrote:
import async # The API of the future ;) async.async_call(retrieve_log_playback, settings, tws, mechanism=multiprocessing) # tws == instance of tornado.web.WebSocketHandler that holds the open connection
Is this a CPU-bound problem?
It depends on the host. On embedded platforms (e.g. the BeagleBone) it is more IO-bound than CPU bound (fast CPU but slow disk and slow memory). On regular x86 systems it is mostly CPU-bound.
- I should be able to choose the type of event loop/async mechanism
that is appropriate for the task: For CPU-bound tasks I'll probably want to use multiprocessing. For IO-bound tasks I might want to use threading. For a multitude of tasks that "just need to be async" (by nature) I'll want to use an event loop.
Ehhh, maybe. This sounds like it confounds the tools for different use cases. You can quite easily have threads and processes on top of an event loop; that works out particularly nicely for processes because you still have to talk to your processes.
Examples:
twisted.internet.reactor.spawnProcess (local processes) twisted.internet.threads.deferToThread (local threads) ampoule (remote processes)
It's quite easy to do blocking IO in a thread with deferToThread; in fact, that's how twisted's adbapi, an async wrapper to dbapi, works.
As I understand it, twisted.internet.reactor.spawnProcess is all about spawning subprocesses akin to subprocess.Popen(). Also, it requires writing a sophisticated ProcessProtocol. It seems to be completely unrelated and wickedly complicated. The complete opposite of what I would consider ideal for an asynchronous library since it is anything but simple.
I mean, I could write a separate program to generate HTML playback files from logs, spawn a subprocess in an asynchronous fashion, then watch it for completion but I could do that with termio.Multiplex (see: https://github.com/liftoff/GateOne/blob/master/gateone/termio.py) .
- Any async module should support 'basics' like calling functions at
an interval and calling functions after a timeout occurs (with the ability to cancel).
- Asynchronous tasks should be able to access the same namespace as
everything else. Maybe wishful thinking.
With twisted, this is already the case; general caveats for shared mutable state across threads of course still apply. Fortunately in most Twisted apps, that's a tiny fraction of the total code, and they tend to be fractions that are well-isolated or at least easily isolatable.
- It should support publish/subscribe-style events (i.e. an event
dispatcher). For example, the ability to watch a file descriptor or socket for changes in state and call a function when that happens. Preferably with the flexibility to define custom events (i.e don't have it tied to kqueue/epoll-specific events).
Like connectionMade, connectionLost, dataReceived etc?
Thanks for your consideration; and thanks for the awesome language.
-- Dan McDougall - Chief Executive Officer and Developer Liftoff Software ✈ Your flight to the cloud is now boarding. 904-446-8323 _______________________________________________ Python-ideas mailing list Python-ideas@python.org http://mail.python.org/mailman/listinfo/python-ideas
-- cheers lvh