Simpler syntax for async generators
Hi All, While I was reading the documentation of asyn generators, which shows the following example: async def ticker(delay, to): """Yield numbers from 0 to *to* every *delay* seconds.""" for i in range(to): yield i await asyncio.sleep(delay) It's came to my mind that it could could be simpler. We could use just async yield and all the other stuff could be done in the background. Like if we use the yield in function, that function becomes a generator example: def my_async_generator(): for i in range(5): async yield i What do you think about this? Would this be feasible? BR, __george__
On Sat, 14 Sep 2019 at 16:26, George Fischhof <george@fischhof.hu> wrote: place to another. Therefore, change just for change sake, is pointless. In fact, more harm than pointless. -1
-- Gustavo J. A. M. Carneiro Gambit Research "The universe is always one step beyond logic." -- Frank Herbert
Which all other stuff? Are you saying that you wouldn't need to use "async def", if there was an "async yield" in the function? But that doesn't solve many problems, because you can create async functions that don't have a yield in them -- ie, they are async, but not generator functions, the two are orthogonal. -CHB -- Christopher Barker, PhD Python Language Consulting - Teaching - Scientific Software Development - Desktop GUI and Web Development - wxPython, numpy, scipy, Cython
Christopher Barker <pythonchb@gmail.com> ezt írta (időpont: 2019. szept. 14., Szo, 21:58):
Yes, that is why I wrote simpler async generator: With this syntax the use / programmer do not have to write async def, nor await, because after the yield the await should come automatically. I got my value, then do my job with the value, in the background the next value is prepared, and next time, when I need the next value, it will be prepared, or there will be a wait until it is prepared. So async yield would mean: give me the value, and start creating the next one, I will come back. __george__
George Fischhof wrote:
With this syntax the use / programmer do not have to write async def, nor await, because after the yield the await should come automatically.
I don't see how you can eliminate the await in your example generator. Can you show us how it would look in its entirety under your proposal? -- Greg
On Mon, Sep 16, 2019 at 11:36:45AM +1200, Greg Ewing wrote:
I think the proposal is for two things: 1. `async yield i` to be syntactic sugar for `yield i; await asyncio.sleep(delay)`. 2. The interpreter should infer that a plain `def` function is actually an `async def` from the presence of `async yield`, similar to the way the interpreter infers that a plain `def` function is actually a generator from the presence of a `yield`. # Original async def ticker(delay, to): """Yield numbers from 0 to to every delay seconds.""" for i in range(to): yield i await asyncio.sleep(delay) # Re-written def ticker(delay, to): """Yield numbers from 0 to to every delay seconds.""" for i in range(to): async yield i So you save one line and perhaps 25-30 key presses when writing the function, at the cost of making it much less clear what's going on when reading the function. How does it known how long to sleep? What if you're not using asyncio, but some other framework? -- Steven
Steven D'Aprano <steve@pearwood.info> ezt írta (időpont: 2019. szept. 16., H 2:55):
Hi Steven, yes exactly I would like this. Regarding the waiting and frameworks: The solution should be independent from waiting and frameworks: Let's see what happens now (sync, not async) User has a generator, he wants values. Uses a for cycle to process all values from generator: Asks for first value - wait for the value to be computed Processes the value he got Asks for next value - wait for the value to be computed And processes again, and again The async version: User has a generator, he wants values, he created an async version, because he wants the values faster Uses a for cycle to process all values from generator: Asks for first value - wait for the value to be computed Processes the value he got - in the meantime, in the background the computation of next value is in progress Asks for next value - it is possible that the values is already computed so there is only maximum a smaller wait for the value And processes again, and again So my idea above the syntax is the following: When one value is yielded, the async generator starts computing in the background, it means it has to execute commands until next yield is reached (the next yield can be same one in a cycle or even a different yield statement .) And stops when reaches the next yield. When the next request comes for the next value, it gives the computed value if it is ready, or computes until the value is ready. And gives the value that time. Actually I can imagine Python as being more and more async to be faster. And therefore async could be the default working method. Practically the user is not interested in how the value is computed, he just wants faster programs. The only exception when async can be bad is when the computation of next value and processing the got value requires the same resource. So when the default is async, the user will get faster program, by default. And he should only sign whe he does not want async, because of same resource usage. BR, __george__
On Mon, Sep 16, 2019 at 7:13 PM George Fischhof <george@fischhof.hu> wrote:
Okay, well that's the problem. An async function won't yield the values faster, and it will not allow two things to happen simultaneously. It just allows I/O and other such operations to happen asynchronously. If you want the values faster, consider instead a second thread or process and a queue; in place of the generator, just read from the queue, which will block until you have something to read. Meanwhile, your other thread/process is generating values and placing them on the queue. What you're asking for is not just a syntactic change - it's a significant semantic change that introduces concurrency in ways that async functions do not currently have. Basically you'd get all the problems of threading, without the advantages of threading. ChrisA
George Fischoff wrote:
So when the default is async, the user will get faster program, by default.
Chris Angelico wrote:
I think the OP might be somewhat conflating parallelism and concurrency. For anyone not entirely clear on the differences between the two, I would highly recommend the following video for an in-depth explanation: https://www.youtube.com/watch?v=cN_DpYBzKso. The usage of concurrent programming does not increase the speed of any given program by default, and is not suited to every situation. On Mon, Sep 16, 2019 at 6:01 AM Chris Angelico <rosuav@gmail.com> wrote:
The interpreter should infer that a plain `def` function is actually an `async def` from the presence of `async yield`, similar to the way
Steven D'Aprano wrote: the interpreter infers that a plain `def` function is actually a generator from the presence of a `yield`. I would have to strongly disagree with this part of the author's proposal especially. The whole purpose of the `async def` declaration was to help ensure that a normal subroutine function was not mistakenly used as a coroutine. Allowing the interpreter to infer that a regularly declared function is actually a subroutine would ultimately go against this intention by making the distinction between subroutines and coroutines far less explicit. Steven D'Aprano wrote:
What if you're not using asyncio, but some other framework?
This point is also especially important. The "async/await" syntax was meant to essentially be its own separate API of sorts, to allow for the usage of entirely distinct async frameworks such as curio. Asyncio is dependent on the "async/await" syntax, but not the other way around. I'm not certain that I understand how these changes could be applied without causing compatibility issues. On Sun, Sep 15, 2019 at 8:52 PM Steven D'Aprano <steve@pearwood.info> wrote:
On Sat, 14 Sep 2019 at 16:26, George Fischhof <george@fischhof.hu> wrote: place to another. Therefore, change just for change sake, is pointless. In fact, more harm than pointless. -1
-- Gustavo J. A. M. Carneiro Gambit Research "The universe is always one step beyond logic." -- Frank Herbert
Which all other stuff? Are you saying that you wouldn't need to use "async def", if there was an "async yield" in the function? But that doesn't solve many problems, because you can create async functions that don't have a yield in them -- ie, they are async, but not generator functions, the two are orthogonal. -CHB -- Christopher Barker, PhD Python Language Consulting - Teaching - Scientific Software Development - Desktop GUI and Web Development - wxPython, numpy, scipy, Cython
Christopher Barker <pythonchb@gmail.com> ezt írta (időpont: 2019. szept. 14., Szo, 21:58):
Yes, that is why I wrote simpler async generator: With this syntax the use / programmer do not have to write async def, nor await, because after the yield the await should come automatically. I got my value, then do my job with the value, in the background the next value is prepared, and next time, when I need the next value, it will be prepared, or there will be a wait until it is prepared. So async yield would mean: give me the value, and start creating the next one, I will come back. __george__
George Fischhof wrote:
With this syntax the use / programmer do not have to write async def, nor await, because after the yield the await should come automatically.
I don't see how you can eliminate the await in your example generator. Can you show us how it would look in its entirety under your proposal? -- Greg
On Mon, Sep 16, 2019 at 11:36:45AM +1200, Greg Ewing wrote:
I think the proposal is for two things: 1. `async yield i` to be syntactic sugar for `yield i; await asyncio.sleep(delay)`. 2. The interpreter should infer that a plain `def` function is actually an `async def` from the presence of `async yield`, similar to the way the interpreter infers that a plain `def` function is actually a generator from the presence of a `yield`. # Original async def ticker(delay, to): """Yield numbers from 0 to to every delay seconds.""" for i in range(to): yield i await asyncio.sleep(delay) # Re-written def ticker(delay, to): """Yield numbers from 0 to to every delay seconds.""" for i in range(to): async yield i So you save one line and perhaps 25-30 key presses when writing the function, at the cost of making it much less clear what's going on when reading the function. How does it known how long to sleep? What if you're not using asyncio, but some other framework? -- Steven
Steven D'Aprano <steve@pearwood.info> ezt írta (időpont: 2019. szept. 16., H 2:55):
Hi Steven, yes exactly I would like this. Regarding the waiting and frameworks: The solution should be independent from waiting and frameworks: Let's see what happens now (sync, not async) User has a generator, he wants values. Uses a for cycle to process all values from generator: Asks for first value - wait for the value to be computed Processes the value he got Asks for next value - wait for the value to be computed And processes again, and again The async version: User has a generator, he wants values, he created an async version, because he wants the values faster Uses a for cycle to process all values from generator: Asks for first value - wait for the value to be computed Processes the value he got - in the meantime, in the background the computation of next value is in progress Asks for next value - it is possible that the values is already computed so there is only maximum a smaller wait for the value And processes again, and again So my idea above the syntax is the following: When one value is yielded, the async generator starts computing in the background, it means it has to execute commands until next yield is reached (the next yield can be same one in a cycle or even a different yield statement .) And stops when reaches the next yield. When the next request comes for the next value, it gives the computed value if it is ready, or computes until the value is ready. And gives the value that time. Actually I can imagine Python as being more and more async to be faster. And therefore async could be the default working method. Practically the user is not interested in how the value is computed, he just wants faster programs. The only exception when async can be bad is when the computation of next value and processing the got value requires the same resource. So when the default is async, the user will get faster program, by default. And he should only sign whe he does not want async, because of same resource usage. BR, __george__
On Mon, Sep 16, 2019 at 7:13 PM George Fischhof <george@fischhof.hu> wrote:
Okay, well that's the problem. An async function won't yield the values faster, and it will not allow two things to happen simultaneously. It just allows I/O and other such operations to happen asynchronously. If you want the values faster, consider instead a second thread or process and a queue; in place of the generator, just read from the queue, which will block until you have something to read. Meanwhile, your other thread/process is generating values and placing them on the queue. What you're asking for is not just a syntactic change - it's a significant semantic change that introduces concurrency in ways that async functions do not currently have. Basically you'd get all the problems of threading, without the advantages of threading. ChrisA
George Fischoff wrote:
So when the default is async, the user will get faster program, by default.
Chris Angelico wrote:
I think the OP might be somewhat conflating parallelism and concurrency. For anyone not entirely clear on the differences between the two, I would highly recommend the following video for an in-depth explanation: https://www.youtube.com/watch?v=cN_DpYBzKso. The usage of concurrent programming does not increase the speed of any given program by default, and is not suited to every situation. On Mon, Sep 16, 2019 at 6:01 AM Chris Angelico <rosuav@gmail.com> wrote:
The interpreter should infer that a plain `def` function is actually an `async def` from the presence of `async yield`, similar to the way
Steven D'Aprano wrote: the interpreter infers that a plain `def` function is actually a generator from the presence of a `yield`. I would have to strongly disagree with this part of the author's proposal especially. The whole purpose of the `async def` declaration was to help ensure that a normal subroutine function was not mistakenly used as a coroutine. Allowing the interpreter to infer that a regularly declared function is actually a subroutine would ultimately go against this intention by making the distinction between subroutines and coroutines far less explicit. Steven D'Aprano wrote:
What if you're not using asyncio, but some other framework?
This point is also especially important. The "async/await" syntax was meant to essentially be its own separate API of sorts, to allow for the usage of entirely distinct async frameworks such as curio. Asyncio is dependent on the "async/await" syntax, but not the other way around. I'm not certain that I understand how these changes could be applied without causing compatibility issues. On Sun, Sep 15, 2019 at 8:52 PM Steven D'Aprano <steve@pearwood.info> wrote:
participants (8)
-
Anders Hovmöller -
Chris Angelico -
Christopher Barker -
George Fischhof -
Greg Ewing -
Gustavo Carneiro -
Kyle Stanley -
Steven D'Aprano