[Async-sig] using asyncio in synchronous applications
Andrew Svetlov
andrew.svetlov at gmail.com
Tue Jul 11 16:25:26 EDT 2017
Hmm. After rethinking I see `set_event_loop()` is required in your design.
But better to have `run(coro())` API, it could be implemented like
def run(coro):
loop = asyncio.new_event_loop()
loop.run_until_complete(coro)
loop.close()
The implementation doesn't touch default loop but
`asyncio.get_event_loop()` call from `coro` returns a running loop instance.
On Tue, Jul 11, 2017 at 10:12 PM Chris Jerdonek <chris.jerdonek at gmail.com>
wrote:
> On Tue, Jul 11, 2017 at 10:20 AM, Andrew Svetlov
> <andrew.svetlov at gmail.com> wrote:
> > Why do you call set_event_loop() on Python 3.6 at all?
>
> Calling set_event_loop() at the end resets / sets things up for the
> next invocation. That was part of my point. Without it, I get the
> following error the next time I try to use the context manager (note
> that I've chosen a better name for the manager here):
>
> with reset_loop_after():
> loop = asyncio.get_event_loop()
> loop.run_until_complete(foo())
>
> with reset_loop_after():
> loop = asyncio.get_event_loop()
> loop.run_until_complete(foo())
>
> Traceback (most recent call last):
> ...
> result = loop.run_until_complete(future)
> File "/usr/local/lib/python3.6/asyncio/base_events.py", line
> 443, in run_until_complete
> self._check_closed()
> File "/usr/local/lib/python3.6/asyncio/base_events.py", line
> 357, in _check_closed
> raise RuntimeError('Event loop is closed')
> RuntimeError: Event loop is closed
>
> Remember that two of the three use cases I listed involve calling the
> function multiple times throughout the process's lifetime.
>
> Is there a way that doesn't require calling set_event_loop()?
>
> --Chris
>
>
> > On Tue, Jul 11, 2017, 17:56 Chris Jerdonek <chris.jerdonek at gmail.com>
> wrote:
> >>
> >> There's something I realized about "creating and destroying" ephemeral
> >> event loops if you want to create temporary event loops over time in a
> >> synchronous application.
> >>
> >> This wasn't clear to me at the beginning, but it's actually more
> >> natural to do the reverse and "destroy and create," and **at the
> >> end**:
> >>
> >> @contextmanager
> >> def run_in_loop():
> >> try:
> >> yield
> >> finally:
> >> loop = asyncio.get_event_loop()
> >> loop.close()
> >> loop = asyncio.new_event_loop()
> >> asyncio.set_event_loop(loop)
> >>
> >> The reason is that at the beginning of an application, the event loop
> >> starts out not closed. So if you start out by creating a new loop at
> >> the beginning, you'll get a warning like the following:
> >>
> >> /usr/local/lib/python3.6/asyncio/base_events.py:509:
> >> ResourceWarning: unclosed event loop <_UnixSelectorEventLoop
> >> running=False closed=False debug=False>
> >>
> >> It's like the cycle is slightly out of phase.
> >>
> >> In contrast, if you create a new loop **at the end**, you're returning
> >> the application to the neutral state it was at the beginning, namely
> >> with a non-None loop that is neither running nor closed.
> >>
> >> I can think of three use cases for the context manager above:
> >>
> >> 1) for wrapping the "main" function of an application,
> >> 2) for calling async functions from a synchronous app (even from
> >> different threads), which is what I was originally asking about, and
> >> 3) as part of a decorator around individual unit tests to guarantee
> >> loop isolation.
> >>
> >> This seems like a really simple thing, but I haven't seen the pattern
> >> above written down anywhere (e.g. in past discussions of
> >> asyncio.run()).
> >>
> >> --Chris
> >>
> >>
> >> On Mon, Jul 10, 2017 at 7:46 AM, Guido van Rossum <guido at python.org>
> >> wrote:
> >> > OK, then as long as close the connection and the loop properly it
> >> > shouldn't
> >> > be a problem, even multi-threaded. (You basically lose all advantage
> of
> >> > async, but it seems you're fine with that.)
> >> >
> >> > On Sun, Jul 9, 2017 at 9:07 PM, Chris Jerdonek
> >> > <chris.jerdonek at gmail.com>
> >> > wrote:
> >> >>
> >> >> On Sun, Jul 9, 2017 at 9:00 PM, Guido van Rossum <guido at python.org>
> >> >> wrote:
> >> >> > But the big question is, what is that library doing for you? In the
> >> >> > abstract
> >> >> > it is hard to give you a good answer. What library is it? What
> calls
> >> >> > are
> >> >> > you
> >> >> > making?
> >> >>
> >> >> It's the websockets library: https://github.com/aaugustin/websockets
> >> >>
> >> >> All I really need to do is occasionally connect briefly to a
> websocket
> >> >> server as a client from a synchronous app.
> >> >>
> >> >> Since I'm already using the library on the server-side, I thought I'd
> >> >> save myself the trouble of having to use two libraries and just use
> >> >> the same library on the client side as well.
> >> >>
> >> >> --Chris
> >> >>
> >> >>
> >> >>
> >> >>
> >> >> >
> >> >> > On Sun, Jul 9, 2017 at 8:48 PM, Chris Jerdonek
> >> >> > <chris.jerdonek at gmail.com>
> >> >> > wrote:
> >> >> >>
> >> >> >> I have a two-part question.
> >> >> >>
> >> >> >> If my application is single-threaded and synchronous (e.g. a web
> app
> >> >> >> using Gunicorn with sync workers [1]), and occasionally I need to
> >> >> >> call
> >> >> >> functions in a library that requires an event loop, is there any
> >> >> >> downside to creating and closing the loop on-the-fly only when I
> >> >> >> call
> >> >> >> the function? In other words, is creating and destroying loops
> >> >> >> cheap?
> >> >> >>
> >> >> >> Second, if I were to switch to a multi-threaded model (e.g.
> Gunicorn
> >> >> >> with async workers), is my only option to start the loop at the
> >> >> >> beginning of the process, and use loop.call_soon_threadsafe()? Or
> >> >> >> can
> >> >> >> I do what I was asking about above and create and close loops
> >> >> >> on-the-fly in different threads? Is either approach much more
> >> >> >> efficient than the other?
> >> >> >>
> >> >> >> Thanks,
> >> >> >> --Chris
> >> >> >>
> >> >> >> [1] http://docs.gunicorn.org/en/latest/design.html#sync-workers
> >> >> >> _______________________________________________
> >> >> >> Async-sig mailing list
> >> >> >> Async-sig at python.org
> >> >> >> https://mail.python.org/mailman/listinfo/async-sig
> >> >> >> Code of Conduct: https://www.python.org/psf/codeofconduct/
> >> >> >
> >> >> >
> >> >> >
> >> >> >
> >> >> > --
> >> >> > --Guido van Rossum (python.org/~guido)
> >> >
> >> >
> >> >
> >> >
> >> > --
> >> > --Guido van Rossum (python.org/~guido)
> >> _______________________________________________
> >> Async-sig mailing list
> >> Async-sig at python.org
> >> https://mail.python.org/mailman/listinfo/async-sig
> >> Code of Conduct: https://www.python.org/psf/codeofconduct/
> >
> > --
> > Thanks,
> > Andrew Svetlov
>
--
Thanks,
Andrew Svetlov
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/async-sig/attachments/20170711/3d7d057b/attachment.html>
More information about the Async-sig
mailing list