<div dir="ltr"><div>Is there a way for a producer to say that there will be no more items put, so consumers get something like StopIteration when there are no more items left afterwards?<br><br></div>There is also the problem that one cannot easily feed a queue, asynchronous generator, or any asynchronous iterator to a simple synchronous consumer like sum() or list() or "".join(). It would be nice if there was a way to wrap them to asynchronous ones when needed – something like (async sum)(asynchronously_produced_numbers()).<br><br><div class="gmail_extra"><br><div class="gmail_quote">On Wed, Jun 24, 2015 at 1:54 PM, Jonathan Slenders <span dir="ltr"><<a href="mailto:jonathan@slenders.be" target="_blank">jonathan@slenders.be</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">In my experience, it's much easier to use asyncio Queues for this.<div>Instead of yielding, push to a queue. The consumer can then use "await queue.get()".</div><div><br></div><div>I think the semantics of the generator become too complicated otherwise, or maybe impossible.</div><div>Maybe have a look at this article: <a href="http://www.interact-sw.co.uk/iangblog/2013/11/29/async-yield-return" target="_blank">http://www.interact-sw.co.uk/iangblog/2013/11/29/async-yield-return</a></div><span class="HOEnZb"><font color="#888888"><div><br></div><div>Jonathan</div><div><br></div><div><br></div><div><br></div></font></span></div><div class="HOEnZb"><div class="h5"><div class="gmail_extra"><br><div class="gmail_quote">2015-06-24 12:13 GMT+02:00 Andrew Svetlov <span dir="ltr"><<a href="mailto:andrew.svetlov@gmail.com" target="_blank">andrew.svetlov@gmail.com</a>></span>:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Your idea is clean and maybe we will allow `yield` inside `async def`<br>
in Python 3.6.<br>
For PEP 492 it was too big change.<br>
<div><div><br>
On Wed, Jun 24, 2015 at 12:00 PM, Adam Bartoš <<a href="mailto:drekin@gmail.com" target="_blank">drekin@gmail.com</a>> wrote:<br>
> Hello,<br>
><br>
> I had a generator producing pairs of values and wanted to feed all the first<br>
> members of the pairs to one consumer and all the second members to another<br>
> consumer. For example:<br>
><br>
> def pairs():<br>
> for i in range(4):<br>
> yield (i, i ** 2)<br>
><br>
> biconsumer(sum, list)(pairs()) -> (6, [0, 1, 4, 9])<br>
><br>
> The point is I wanted the consumers to be suspended and resumed in a<br>
> coordinated manner: The first producer is invoked, it wants the first<br>
> element. The coordinator implemented by biconsumer function invokes pairs(),<br>
> gets the first pair and yields its first member to the first consumer. Then<br>
> it wants the next element, but now it's the second consumer's turn, so the<br>
> first consumer is suspended and the second consumer is invoked and fed with<br>
> the second member of the first pair. Then the second producer wants the next<br>
> element, but it's the first consumer's turn… and so on. In the end, when the<br>
> stream of pairs is exhausted, StopIteration is thrown to both consumers and<br>
> their results are combined.<br>
><br>
> The cooperative asynchronous nature of the execution reminded me asyncio and<br>
> coroutines, so I thought that biconsumer may be implemented using them.<br>
> However, it seems that it is imposible to write an "asynchronous generator"<br>
> since the "yielding pipe" is already used for the communication with the<br>
> scheduler. And even if it was possible to make an asynchronous generator, it<br>
> is not clear how to feed it to a synchronous consumer like sum() or list()<br>
> function.<br>
><br>
> With PEP 492 the concepts of generators and coroutines were separated, so<br>
> asyncronous generators may be possible in theory. An ordinary function has<br>
> just the returning pipe – for returning the result to the caller. A<br>
> generator has also a yielding pipe – used for yielding the values during<br>
> iteration, and its return pipe is used to finish the iteration. A native<br>
> coroutine has a returning pipe – to return the result to a caller just like<br>
> an ordinary function, and also an async pipe – used for communication with a<br>
> scheduler and execution suspension. An asynchronous generator would just<br>
> have both yieling pipe and async pipe.<br>
><br>
> So my question is: was the code like the following considered? Does it make<br>
> sense? Or are there not enough uses cases for such code? I found only a<br>
> short mention in<br>
> <a href="https://www.python.org/dev/peps/pep-0492/#coroutine-generators" rel="noreferrer" target="_blank">https://www.python.org/dev/peps/pep-0492/#coroutine-generators</a>, so possibly<br>
> these coroutine-generators are the same idea.<br>
><br>
> async def f():<br>
> number_string = await fetch_data()<br>
> for n in number_string.split():<br>
> yield int(n)<br>
><br>
> async def g():<br>
> result = async/await? sum(f())<br>
> return result<br>
><br>
> async def h():<br>
> the_sum = await g()<br>
><br>
> As for explanation about the execution of h() by an event loop: h is a<br>
> native coroutine called by the event loop, having both returning pipe and<br>
> async pipe. The returning pipe leads to the end of the task, the async pipe<br>
> is used for cummunication with the scheduler. Then, g() is called<br>
> asynchronously – using the await keyword means the the access to the async<br>
> pipe is given to the callee. Then g() invokes the asyncronous generator f()<br>
> and gives it the access to its async pipe, so when f() is yielding values to<br>
> sum, it can also yield a future to the scheduler via the async pipe and<br>
> suspend the whole task.<br>
><br>
> Regards, Adam Bartoš<br>
><br>
><br>
</div></div>> _______________________________________________<br>
> Python-ideas mailing list<br>
> <a href="mailto:Python-ideas@python.org" target="_blank">Python-ideas@python.org</a><br>
> <a href="https://mail.python.org/mailman/listinfo/python-ideas" rel="noreferrer" target="_blank">https://mail.python.org/mailman/listinfo/python-ideas</a><br>
> Code of Conduct: <a href="http://python.org/psf/codeofconduct/" rel="noreferrer" target="_blank">http://python.org/psf/codeofconduct/</a><br>
<span><font color="#888888"><br>
<br>
<br>
--<br>
Thanks,<br>
Andrew Svetlov<br>
_______________________________________________<br>
Python-ideas mailing list<br>
<a href="mailto:Python-ideas@python.org" target="_blank">Python-ideas@python.org</a><br>
<a href="https://mail.python.org/mailman/listinfo/python-ideas" rel="noreferrer" target="_blank">https://mail.python.org/mailman/listinfo/python-ideas</a><br>
Code of Conduct: <a href="http://python.org/psf/codeofconduct/" rel="noreferrer" target="_blank">http://python.org/psf/codeofconduct/</a></font></span></blockquote></div><br></div>
</div></div></blockquote></div><br></div></div>