[Python-ideas] Asynchronous friendly iterables

[Terry Reedy]

On 8/20/2018 3:19 AM, Simon De Greve wrote:
> Hello everyone,
> 
> I'm quite new working with asyncio and thus maybe missing some things 
> about it, but wouldn't it be quite easier to have some iterables to 
> support async for loops "natively", since asyncio is now part of the 
> Stdlib?

One purpose of asynchronous programming is to pause a task that is 
waiting for input from an external system, so as to not waste CPU time. 
As other noted, this is not an issue with iterating through in-memory 
collections.

It is worth noting that 'async' and 'await' are syntax keywords for 
working with coroutines (how is underdocumented) and that the asyncio 
module is just one event-loop system that can drive coroutines.  The 
tkinter module, over 20 years old, can also.

> I've tried to work with asyncio while using discord.py, and has some 
> struggle with an "async for" loop on a dictionary, so I had to implement 
> a new dict subclass that would just reimplement items(), keys() and 
> values() functions.

Another purpose of asynchonous programming is to keep a system 
responsive, for instance, to user input by not letting a compute-bound 
task tie-up a cpu indefinitely.  This issue *can* apply to iterating 
through sufficiently large collection in, for instance, a tkinter gui 
program.  But pausing iteration after *each* iteration is usually a 
terrible waste of overhead.  So one wants to do some number of 
iterations at full speed and then pause to allow other events to be handled.

> I think that it would be a cool improvement to implement some of those 
> in some standard way. 
Occasionally pausing iteration should not be a method of the iterable or 
iterator.  We need instead wrapper functions.  One idea is to pause 
every k iterations.  For user responsiveness, we want to pause for event 
handling after a certain time has elapsed, say 50 milliseconds.  I think 
it should be possible to do this directly instead of pre-testing how 
long it takes to do 1 interation.

I know how to do the above with tkinter callback loops, but I have not 
yet worked out how to do so with async def coroutines.

-- 
Terry Jan Reedy