ANN: miniasync, a small library build on top of asyncio for simple use cases
Hello, I have been working on miniasync, a small library build on top of asyncio to facilitate the occasional use of async code in otherwise synchronous applications. A typical use case is an otherwise synchronous application, which at some point wants to send notifications to multiple online services. miniasync makes it easier to implement this asynchronously without having to write asyncio boilerplate. http://miniasync.readthedocs.io asyncio is vast, powerful and flexible. But there is a learning curve - you need to understand the event loop api (other async languages like Javascript don't expose that), how to create it and run tasks on it, which means you need to understand about Tasks and Futures, etc. The flexibility is great, and gives Python developers a lot of scope for implementing advanced applications, but it's a put off for simple use cases. miniasync exposes a single function, "miniasync.run", which takes a list of coroutine objects, creates a local event loop, runs all the co-routines until they're complete, and returns their result in the order they were defined: import aiofiles import miniasync async def get_file_content(filename): async with aiofiles.open(filename, mode='r') as f: return await f.read() results = miniasync.run( get_file_content('file1.txt'), get_file_content('file2.txt'), ) assert results == [ '<the content of file1.txt>', '<the content of file2.txt>' ] This is similar to using a combination of gather and run_until_complete, but as a single step and without explicit reference to the loop. This also differs from gather in that: - Instead of saying you either want exceptions raised or returned, you need to list explicitly exceptions you want returned, all other exceptions are raised (explicit is better than implicit); - Unhandled exceptions cause all other tasks to be cancelled. miniasync.run always creates a new loop (so you can nest invocations of miniasync.run, and rely on each invocation only executing it's parameters). For the cases where you need the loop before running (eg. for creating a asyncio.Queue object to be shared amongst your co-routines), miniasync also exposes a context manager that creates a loop and lets you run co-routines on it: import asyncio import miniasync async def coro1(q): q.put_nowait('world') return 'hello' async def coro2(q): return await q.get() with miniasync.loop() as loop: q = asyncio.Queue() results = loop.run( coro1(q), coro2(q) ) assert results == ['hello', 'world'] You can pip install miniasync to try out, or read the docs on readthedocs (as above). For now miniasync covers what was my main issue with running simple async code. Other things I have in mind for the future is a simple interface for running multiple http requests (based on top of aiohttp). I'm keen to hear about other issues that could be simplified for simple use cases. :) Alice
Nice! At first, I thought that implementation would be trivial, but upon inspection it's actually educational! Perhaps interactive environments like ipython and jupyter-notebook could benefit from this library. Cheers, d. On 4 May 2018 at 15:45, Alice Heaton <alicemail@loveisanalogue.info> wrote:
Hello,
I have been working on miniasync, a small library build on top of asyncio to facilitate the occasional use of async code in otherwise synchronous applications.
A typical use case is an otherwise synchronous application, which at some point wants to send notifications to multiple online services. miniasync makes it easier to implement this asynchronously without having to write asyncio boilerplate.
http://miniasync.readthedocs.io
asyncio is vast, powerful and flexible. But there is a learning curve - you need to understand the event loop api (other async languages like Javascript don't expose that), how to create it and run tasks on it, which means you need to understand about Tasks and Futures, etc. The flexibility is great, and gives Python developers a lot of scope for implementing advanced applications, but it's a put off for simple use cases.
miniasync exposes a single function, "miniasync.run", which takes a list of coroutine objects, creates a local event loop, runs all the co-routines until they're complete, and returns their result in the order they were defined:
import aiofiles import miniasync
async def get_file_content(filename): async with aiofiles.open(filename, mode='r') as f: return await f.read()
results = miniasync.run( get_file_content('file1.txt'), get_file_content('file2.txt'), )
assert results == [ '<the content of file1.txt>', '<the content of file2.txt>' ]
This is similar to using a combination of gather and run_until_complete, but as a single step and without explicit reference to the loop. This also differs from gather in that:
- Instead of saying you either want exceptions raised or returned, you need to list explicitly exceptions you want returned, all other exceptions are raised (explicit is better than implicit); - Unhandled exceptions cause all other tasks to be cancelled.
miniasync.run always creates a new loop (so you can nest invocations of miniasync.run, and rely on each invocation only executing it's parameters). For the cases where you need the loop before running (eg. for creating a asyncio.Queue object to be shared amongst your co-routines), miniasync also exposes a context manager that creates a loop and lets you run co-routines on it:
import asyncio import miniasync
async def coro1(q): q.put_nowait('world') return 'hello'
async def coro2(q): return await q.get()
with miniasync.loop() as loop: q = asyncio.Queue() results = loop.run( coro1(q), coro2(q) )
assert results == ['hello', 'world']
You can pip install miniasync to try out, or read the docs on readthedocs (as above).
For now miniasync covers what was my main issue with running simple async code. Other things I have in mind for the future is a simple interface for running multiple http requests (based on top of aiohttp).
I'm keen to hear about other issues that could be simplified for simple use cases.
:) Alice _______________________________________________ Async-sig mailing list Async-sig@python.org https://mail.python.org/mailman/listinfo/async-sig Code of Conduct: https://www.python.org/psf/codeofconduct/
On 04/05/18 09:18, Dima Tisnek wrote:
Nice!
Thanks :)
At first, I thought that implementation would be trivial, but upon inspection it's actually educational!
There a number of small gotchas which are obvious once you think about them, and are not complicated per se, but can trip people when they first start using asyncio (they tripped me anyway :)). For example: once you've cancelled a Future (by calling ".cancel()" on it) you actually need to run the loop again to give the code a chance to actually do it's clean up tasks. It makes sense once you understand how asyncio works, but it's not obvious at first. miniasync aims to shield people from these things for the simple use case. :) Alice
Nice! On Fri, May 4, 2018 at 1:57 AM, Alice Heaton <alicemail@loveisanalogue.info> wrote:
On 04/05/18 09:18, Dima Tisnek wrote:
Nice!
Thanks :)
At first, I thought that implementation would be trivial, but upon inspection it's actually educational!
There a number of small gotchas which are obvious once you think about them, and are not complicated per se, but can trip people when they first start using asyncio (they tripped me anyway :)).
For example: once you've cancelled a Future (by calling ".cancel()" on it) you actually need to run the loop again to give the code a chance to actually do it's clean up tasks. It makes sense once you understand how asyncio works, but it's not obvious at first.
miniasync aims to shield people from these things for the simple use case.
:) Alice _______________________________________________ Async-sig mailing list Async-sig@python.org https://mail.python.org/mailman/listinfo/async-sig Code of Conduct: https://www.python.org/psf/codeofconduct/
-- --Guido van Rossum (python.org/~guido)
participants (3)
-
Alice Heaton
-
Dima Tisnek
-
Guido van Rossum