In my experience, it's much easier to use asyncio Queues for this. Instead of yielding, push to a queue. The consumer can then use "await queue.get()". I think the semantics of the generator become too complicated otherwise, or maybe impossible. Maybe have a look at this article: http://www.interact-sw.co.uk/iangblog/2013/11/29/async-yield-return Jonathan 2015-06-24 12:13 GMT+02:00 Andrew Svetlov <andrew.svetlov@gmail.com>:
Your idea is clean and maybe we will allow `yield` inside `async def` in Python 3.6. For PEP 492 it was too big change.
Hello,
I had a generator producing pairs of values and wanted to feed all the first members of the pairs to one consumer and all the second members to another consumer. For example:
def pairs(): for i in range(4): yield (i, i ** 2)
biconsumer(sum, list)(pairs()) -> (6, [0, 1, 4, 9])
The point is I wanted the consumers to be suspended and resumed in a coordinated manner: The first producer is invoked, it wants the first element. The coordinator implemented by biconsumer function invokes
gets the first pair and yields its first member to the first consumer. Then it wants the next element, but now it's the second consumer's turn, so
first consumer is suspended and the second consumer is invoked and fed with the second member of the first pair. Then the second producer wants the next element, but it's the first consumer's turn… and so on. In the end, when
stream of pairs is exhausted, StopIteration is thrown to both consumers and their results are combined.
The cooperative asynchronous nature of the execution reminded me asyncio and coroutines, so I thought that biconsumer may be implemented using them. However, it seems that it is imposible to write an "asynchronous generator" since the "yielding pipe" is already used for the communication with the scheduler. And even if it was possible to make an asynchronous generator, it is not clear how to feed it to a synchronous consumer like sum() or
function.
With PEP 492 the concepts of generators and coroutines were separated, so asyncronous generators may be possible in theory. An ordinary function has just the returning pipe – for returning the result to the caller. A generator has also a yielding pipe – used for yielding the values during iteration, and its return pipe is used to finish the iteration. A native coroutine has a returning pipe – to return the result to a caller just
an ordinary function, and also an async pipe – used for communication with a scheduler and execution suspension. An asynchronous generator would just have both yieling pipe and async pipe.
So my question is: was the code like the following considered? Does it make sense? Or are there not enough uses cases for such code? I found only a short mention in https://www.python.org/dev/peps/pep-0492/#coroutine-generators, so
these coroutine-generators are the same idea.
async def f(): number_string = await fetch_data() for n in number_string.split(): yield int(n)
async def g(): result = async/await? sum(f()) return result
async def h(): the_sum = await g()
As for explanation about the execution of h() by an event loop: h is a native coroutine called by the event loop, having both returning pipe and async pipe. The returning pipe leads to the end of the task, the async
On Wed, Jun 24, 2015 at 12:00 PM, Adam Bartoš <drekin@gmail.com> wrote: pairs(), the the list() like possibly pipe
is used for cummunication with the scheduler. Then, g() is called asynchronously – using the await keyword means the the access to the async pipe is given to the callee. Then g() invokes the asyncronous generator f() and gives it the access to its async pipe, so when f() is yielding values to sum, it can also yield a future to the scheduler via the async pipe and suspend the whole task.
Regards, Adam Bartoš
_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
-- Thanks, Andrew Svetlov _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/