[Async-sig] using asyncio in synchronous applications

[Chris Jerdonek]

On Tue, Jul 11, 2017 at 10:20 AM, Andrew Svetlov
<andrew.svetlov at gmail.com> wrote:
> Why do you call set_event_loop() on Python 3.6 at all?

Calling set_event_loop() at the end resets / sets things up for the
next invocation. That was part of my point. Without it, I get the
following error the next time I try to use the context manager (note
that I've chosen a better name for the manager here):

    with reset_loop_after():
        loop = asyncio.get_event_loop()
        loop.run_until_complete(foo())

    with reset_loop_after():
        loop = asyncio.get_event_loop()
        loop.run_until_complete(foo())

    Traceback (most recent call last):
      ...
        result = loop.run_until_complete(future)
      File "/usr/local/lib/python3.6/asyncio/base_events.py", line
443, in run_until_complete
        self._check_closed()
      File "/usr/local/lib/python3.6/asyncio/base_events.py", line
357, in _check_closed
        raise RuntimeError('Event loop is closed')
    RuntimeError: Event loop is closed

Remember that two of the three use cases I listed involve calling the
function multiple times throughout the process's lifetime.

Is there a way that doesn't require calling set_event_loop()?

--Chris


> On Tue, Jul 11, 2017, 17:56 Chris Jerdonek <chris.jerdonek at gmail.com> wrote:
>>
>> There's something I realized about "creating and destroying" ephemeral
>> event loops if you want to create temporary event loops over time in a
>> synchronous application.
>>
>> This wasn't clear to me at the beginning, but it's actually more
>> natural to do the reverse and "destroy and create," and **at the
>> end**:
>>
>>     @contextmanager
>>     def run_in_loop():
>>         try:
>>             yield
>>         finally:
>>             loop = asyncio.get_event_loop()
>>             loop.close()
>>             loop = asyncio.new_event_loop()
>>             asyncio.set_event_loop(loop)
>>
>> The reason is that at the beginning of an application, the event loop
>> starts out not closed. So if you start out by creating a new loop at
>> the beginning, you'll get a warning like the following:
>>
>>   /usr/local/lib/python3.6/asyncio/base_events.py:509:
>> ResourceWarning: unclosed event loop <_UnixSelectorEventLoop
>> running=False closed=False debug=False>
>>
>> It's like the cycle is slightly out of phase.
>>
>> In contrast, if you create a new loop **at the end**, you're returning
>> the application to the neutral state it was at the beginning, namely
>> with a non-None loop that is neither running nor closed.
>>
>> I can think of three use cases for the context manager above:
>>
>> 1) for wrapping the "main" function of an application,
>> 2) for calling async functions from a synchronous app (even from
>> different threads), which is what I was originally asking about, and
>> 3) as part of a decorator around individual unit tests to guarantee
>> loop isolation.
>>
>> This seems like a really simple thing, but I haven't seen the pattern
>> above written down anywhere (e.g. in past discussions of
>> asyncio.run()).
>>
>> --Chris
>>
>>
>> On Mon, Jul 10, 2017 at 7:46 AM, Guido van Rossum <guido at python.org>
>> wrote:
>> > OK, then as long as close the connection and the loop properly it
>> > shouldn't
>> > be a problem, even multi-threaded. (You basically lose all advantage of
>> > async, but it seems you're fine with that.)
>> >
>> > On Sun, Jul 9, 2017 at 9:07 PM, Chris Jerdonek
>> > <chris.jerdonek at gmail.com>
>> > wrote:
>> >>
>> >> On Sun, Jul 9, 2017 at 9:00 PM, Guido van Rossum <guido at python.org>
>> >> wrote:
>> >> > But the big question is, what is that library doing for you? In the
>> >> > abstract
>> >> > it is hard to give you a good answer. What library is it? What calls
>> >> > are
>> >> > you
>> >> > making?
>> >>
>> >> It's the websockets library: https://github.com/aaugustin/websockets
>> >>
>> >> All I really need to do is occasionally connect briefly to a websocket
>> >> server as a client from a synchronous app.
>> >>
>> >> Since I'm already using the library on the server-side, I thought I'd
>> >> save myself the trouble of having to use two libraries and just use
>> >> the same library on the client side as well.
>> >>
>> >> --Chris
>> >>
>> >>
>> >>
>> >>
>> >> >
>> >> > On Sun, Jul 9, 2017 at 8:48 PM, Chris Jerdonek
>> >> > <chris.jerdonek at gmail.com>
>> >> > wrote:
>> >> >>
>> >> >> I have a two-part question.
>> >> >>
>> >> >> If my application is single-threaded and synchronous (e.g. a web app
>> >> >> using Gunicorn with sync workers [1]), and occasionally I need to
>> >> >> call
>> >> >> functions in a library that requires an event loop, is there any
>> >> >> downside to creating and closing the loop on-the-fly only when I
>> >> >> call
>> >> >> the function? In other words, is creating and destroying loops
>> >> >> cheap?
>> >> >>
>> >> >> Second, if I were to switch to a multi-threaded model (e.g. Gunicorn
>> >> >> with async workers), is my only option to start the loop at the
>> >> >> beginning of the process, and use loop.call_soon_threadsafe()? Or
>> >> >> can
>> >> >> I do what I was asking about above and create and close loops
>> >> >> on-the-fly in different threads? Is either approach much more
>> >> >> efficient than the other?
>> >> >>
>> >> >> Thanks,
>> >> >> --Chris
>> >> >>
>> >> >> [1] http://docs.gunicorn.org/en/latest/design.html#sync-workers
>> >> >> _______________________________________________
>> >> >> Async-sig mailing list
>> >> >> Async-sig at python.org
>> >> >> https://mail.python.org/mailman/listinfo/async-sig
>> >> >> Code of Conduct: https://www.python.org/psf/codeofconduct/
>> >> >
>> >> >
>> >> >
>> >> >
>> >> > --
>> >> > --Guido van Rossum (python.org/~guido)
>> >
>> >
>> >
>> >
>> > --
>> > --Guido van Rossum (python.org/~guido)
>> _______________________________________________
>> Async-sig mailing list
>> Async-sig at python.org
>> https://mail.python.org/mailman/listinfo/async-sig
>> Code of Conduct: https://www.python.org/psf/codeofconduct/
>
> --
> Thanks,
> Andrew Svetlov