[Python-ideas] async: switch_to_loop and loop_affinity

[Jonathan Slenders]

Do you know about run_in_executor?

http://docs.python.org/3.4/library/asyncio-eventloop.html#asyncio.BaseEventLoop.run_in_executor


2014-02-12 10:24 GMT+01:00 David Foster <davidfstr at gmail.com>:

> There has been much talk of the use of async libraries (such as Tulip)
> within the general programming community to avoid the "christmas tree"
> pattern of code indentation in code that uses a lot of callbacks. For
> example, consider the following callback-based code:
>
> def duplicate_file_async(filepath, done):
>     def exists_async(does_exist):
>         if not does_exist:
>             done(False)
>             return
>         def entire_file_read(contents):
>             if not contents:
>                 done(False)
>                 return
>             def file_written(success):
>                 done(success)
>             write_entire_file_async(filepath + ' copy', contents,
> file_written)
>         read_entire_file_async(filepath, entire_file_read)
>     exists_async(filepath)
>
> This code be rewritten in Tulip as:
>
> @async.coroutine
> def duplicate_file_async(filepath):
>     if not yield from exists_async(filepath):
>         return False
>     contents = yield from read_entire_file_async(filepath)
>     if not contents:
>         return False
>     return yield from write_entire_file_async(filepath + ' copy',
> contents)
>
> Much more clear, no?
>
> Tulip, however, does not appear to address a slightly different problem I
> run into where I need different parts of the same conceptual function to
> run on specific *different* threads. Consider the following code which
> needs different parts executed on a UI thread, database thread, and
> background thread:
>
> def tree_node_clicked(tree_node):
>     print('Clicked: ' + repr(tree_node))
>     def db_task():
>         db_node = fetch_node_from_db(tree_node)
>
>         def process_node(db_node):
>             tree_node.set_contents(db_node.contents)
>             # (done)
>
>         if db_node is not None:
>             def bg_task():
>                 db_node = fetch_node_from_internet(tree_node.url)
>                 def db_task():
>                     insert_into_db(db_node)
>                     ui_call_soon(process_node, db_node)
>                 db_call_soon(db_task)
>             bg_call_soon(bg_task)
>         else:
>             ui_call_soon(process_node, db_node)
>     db_call_soon(db_task)
>
> Imagine if you could write this function like the following:
>
> ui_loop = ...
> db_loop = ...
> bg_loop = ...
>
> @async.coroutine
> def tree_node_clicked(tree_node):
>     print('Clicked: ' + repr(tree_node))
>
>     yield from switch_to_loop(db_loop)
>     db_node = fetch_node_from_db(tree_node)
>     if db_node is not None:
>         yield from switch_to_loop(bg_loop)
>         db_node = fetch_node_from_internet(tree_node.url)
>
>         yield from switch_to_loop(db_loop)
>         insert_into_db(db_node)
>
>     yield from switch_to_loop(ui_loop)
>     tree_node.set_contents(db_node.contents)
>
> Or even better: If you created a decorators like @loop_affinity(*_loop)
> that automatically called switch_to_loop(...) if the current event loop
> wasn't correct, you could even write the following even-more simplified
> version:
>
> @async.coroutine
> def tree_node_clicked(tree_node):
>     print('Clicked: ' + repr(tree_node))
>     db_node = yield from fetch_node_from_db(tree_node)
>     if db_node is not None:
>         db_node = yield from fetch_node_from_internet(tree_node.url)
>         yield from insert_into_db(db_node)
>     yield from tree_node.set_contents(db_node.contents)
>
> @async.loop_affinity(db_loop)
> def fetch_node_from_db(...): ...
>
> @async.loop_affinity(bg_loop)
> def fetch_node_from_internet(...): ...
>
> @async.loop_affinity(db_loop)
> def insert_into_db(...): ...
>
> @async.loop_affinity(ui_loop)
> def set_contents(...): ...
>
> Wouldn't that be just grand? I have prototyped switch_to_loop(...)
> successfully in Tulip, albeit with a race condition I was unable to isolate.
>
> How about some equivalent to switch_to_loop(...) and loop_affinity(...) in
> a future version of Tulip?
>
> --
> David Foster
> http://dafoster.net/
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140212/445a70b9/attachment-0001.html>