Question about asyncio and blocking operations
Rustom Mody
rustompmody at gmail.com
Tue Jan 26 01:01:27 EST 2016
On Monday, January 25, 2016 at 9:16:13 PM UTC+5:30, Ian wrote:
> On Mon, Jan 25, 2016 at 8:32 AM, Ian Kelly wrote:
> >
> > On Jan 25, 2016 2:04 AM, "Frank Millman" wrote:
> >>
> >> "Ian Kelly" wrote in message
> >>>
> >>> This seems to be a common misapprehension about asyncio programming.
> >>> While coroutines are the focus of the library, they're based on
> >>> futures, and so by working at a slightly lower level you can also
> >>> handle them as such. So while this would be the typical way to use
> >>> run_in_executor:
> >>>
> >>> async def my_coroutine(stuff):
> >>> value = await get_event_loop().run_in_executor(None,
> >>> blocking_function, stuff)
> >>> result = await do_something_else_with(value)
> >>> return result
> >>>
> >>> This is also a perfectly valid way to use it:
> >>>
> >>> def normal_function(stuff):
> >>> loop = get_event_loop()
> >>> coro = loop.run_in_executor(None, blocking_function, stuff)
> >>> task = loop.create_task(coro)
> >>> task.add_done_callback(do_something_else)
> >>> return task
> >>
> >>
> >> I am struggling to get my head around this.
> >>
> >> 1. In the second function, AFAICT coro is already a future. Why is it
> >> necessary to turn it into a task? In fact when I tried that in my testing, I
> >> got an assertion error -
> >>
> >> File: "C:\Python35\lib\asyncio\base_events.py", line 211, in create_task
> >> task = tasks.Task(coro, loop=self)
> >> File: "C:\Python35\lib\asyncio\tasks.py", line 70, in __init__
> >> assert coroutines.iscoroutine(coro), repr(coro)
> >> AssertionError: <Future pending ... >
> >
> > I didn't test this; it was based on the documentation, which says that
> > run_in_executor is a coroutine. Looking at the source, it's actually a
> > function that returns a future, so this may be a documentation bug.
>
> And now I'm reminded of this note in the asyncio docs:
>
> """
> Note: In this documentation, some methods are documented as
> coroutines, even if they are plain Python functions returning a
> Future. This is intentional to have a freedom of tweaking the
> implementation of these functions in the future. If such a function is
> needed to be used in a callback-style code, wrap its result with
> ensure_future().
> """
>
> IMO such methods should simply be documented as awaitables, not
> coroutines. I wonder if that's already settled, or if it's worth
> starting a discussion around.
Bah -- What a bloody mess!
And thanks for pointing this out, Ian.
Keep wondering whether my brain is atrophying, or its rocket science or...
More information about the Python-list
mailing list