[Python-ideas] PEP 525: Asynchronous Generators

[Yury Selivanov]

On 2016-08-04 11:47 PM, Nickolai Novik wrote:
> Hi Yury!
>
> Thanks you for pushing this forward! As an author/contributor to 
> several packages would love to use async generator in 
> aiomysql/aiobotocore libraries. In particular want to share my use 
> case for async generators.
>
>  aiobotocore (https://github.com/aio-libs/aiobotocore) is wrapper 
> around botocore library (https://github.com/boto/botocore) brings 
> amazon services client to asyncio world. AWS api is huge, botocore 
> itself 20k lines of code, so I bit hacked it here and there to add 
> asyncio support. Code a bit ugly and hackish, but we have full amazon 
> support only with few hundreds lines of code. Since protocol separated 
> from IO, it is pretty strait forward to change requests http client 
> with aiohttp one.  But to port pagination logic, so it is possible to 
> use async for, was very challenging. Here is original pagination
> https://github.com/boto/botocore/blob/bb09e88508f5593ce4393c72e7c1edbaf6d28a6a/botocore/paginate.py#L91-L145 
> which has structure like:
>
>
> async def f():
>     var = await io_fetch()
>     while True:
>         if var > 10:
>             var += 1
>             yield var
>             var = await io_fetch()
>         else:
>             var +=2
>             yield var
>
>
> If we rewrite this generator using __aiter__ code become very ugly and 
> barely readable, since we need to track a lot of states, with async 
> generator such problem does not exist. Moreover with this PEP, 
> generators could be easily ported by just putting async/await in 
> proper places, without changing logic.
>


Thank you, Nickolai!

I've written a couple of iterators that look like the one you have in 
botocore -- I totally feel your pain.  It's really cool that you share 
this feedback.  As I sad to Andrew in this thread -- feedback from power 
asyncio users is extremely useful.

Thanks,
Yury