[asyncio] Suggestion for a major PEP
Hello, I copy paste the main idea from an article I have written: contextual async <https://pythonc.home.blog/2018/12/14/premier-article-de-blog/> " Imagine you have some code written for monothread. And you want to include your code in a multithread environment. Do you need to adapt all your code which is what you do when you want to migrate to async code ? The answer is no. Functionnally these constraints are not justified neither technically Do we have the tools to do this ? Yes because thanks to boost::context we can switch context between tasks. When a task suspends, it just calls a function (the event loop or reactor) to potentially switch to another task. Just like threads switch contexts… Async/Await logic has introduced a symetric relation wich introduces unnecessary contraints. We should just the same logic as thread logic. " Read the examples in the article I have developped a prototype in C++ and everything works perfectly. My opinion is that sooner or later, it will have to switch to this logic because chaining async/aswait is a huge contraints and does not make sense in my opinion. Maybe I am missing something, Feel free to give me your feedback. Regards, Chris
If you want this style of concurrency, you don't need to write a PEP, just 'pip install gevent' :-) But unfortunately you're years too late to argue for making asyncio work this way. This was discussed extensively at the time, and the decision to use special syntax was made intentionally, and after studying existing systems like gevent that made the other choice. This section of the trio docs explain why explicit async/await syntax makes life easier for developers: https://trio.readthedocs.io/en/latest/reference-core.html#checkpoints It's also awkward but very doable to support both sync and async mode with a single code base: https://github.com/python-trio/unasync/ In fact, when doing this, the async/await syntax isn't really the hard part – the hard part is that different libraries have very different networking APIs. E.g., the stdlib socket API and the stdlib asyncio API are totally different. -n On Sun, Dec 16, 2018 at 12:21 AM Christophe Bailly <chbailly@gmail.com> wrote:
-- Nathaniel J. Smith -- https://vorpus.org
Hello, Thanks for your answer. The advantage of this method is that you still follow the logic of async/await. In fact you make it even easier,. It is just a different implementation but with fewer constraints. So my suggestion is to keep this logic because it is a very good logic !!!. , But we should remove this unjusfied chaining of async/await methods. To be clear, this is an async /await logic, except you have an async on one end and an await at the other end and you remove everything in between ! I could have written my examples with async/await, I have used the future syntax but it is the same. I can rewrite my examples if you prefer, I will just use other keywords. That is my opinon, we differ on this but I think there is something really wrong when you add unjustified syntax. You suggest gevent but where do you see async await in gevent ?
I perfectly understand that it is late to implement this, but we could take also into account the real limitations that are difficult to overcome with asyncio. I think I do not need to post links, many will undertand the constraints I am talking about. Regards, Chris On Sun, 16 Dec 2018 at 10:16, Nathaniel Smith <njs@pobox.com> wrote:
On Sun, Dec 16, 2018 at 09:21:14AM +0100, Christophe Bailly wrote:
Async/Await logic has introduced a symetric relation wich introduces unnecessary contraints. We should just the same logic as thread logic.
I'm not an expert on async, but whenever I hear people saying "we should use (something just like) threads" I'm reminded of something that Jamie Zawinski could have (but didn't) say: Some people, when confronted with a problem, think "I know, I'll use threads." Nothwey htwo pavroble ems. I know, that's a sound-bite, not a reasoned argument. But if your intention is to make async code just like threads, how do you avoid the well-known perils of threading? The point of async code is to make context switches explicit, rather than implicit like threading. So at first glance, it seems like you are suggesting we take the major benefit of async (explicitness) and replace it with the major disadvantage of other concurrency models (implicitness). -- Steve
Christophe Bailly wrote:
All of your examples there are C++. It's not clear how any of this relates to Python.
Do we have the tools to do this ? Yes because thanks to boost::context we can switch context between tasks.
How does it work? Is it pulling some kind of stack switching trick? If so, I would be skeptical about the portability and reliability. Also, if it requires the Python interpreter to become C++ or depend on a C++ runtime, it's not going to be accepted. -- Greg
If you want this style of concurrency, you don't need to write a PEP, just 'pip install gevent' :-) But unfortunately you're years too late to argue for making asyncio work this way. This was discussed extensively at the time, and the decision to use special syntax was made intentionally, and after studying existing systems like gevent that made the other choice. This section of the trio docs explain why explicit async/await syntax makes life easier for developers: https://trio.readthedocs.io/en/latest/reference-core.html#checkpoints It's also awkward but very doable to support both sync and async mode with a single code base: https://github.com/python-trio/unasync/ In fact, when doing this, the async/await syntax isn't really the hard part – the hard part is that different libraries have very different networking APIs. E.g., the stdlib socket API and the stdlib asyncio API are totally different. -n On Sun, Dec 16, 2018 at 12:21 AM Christophe Bailly <chbailly@gmail.com> wrote:
-- Nathaniel J. Smith -- https://vorpus.org
Hello, Thanks for your answer. The advantage of this method is that you still follow the logic of async/await. In fact you make it even easier,. It is just a different implementation but with fewer constraints. So my suggestion is to keep this logic because it is a very good logic !!!. , But we should remove this unjusfied chaining of async/await methods. To be clear, this is an async /await logic, except you have an async on one end and an await at the other end and you remove everything in between ! I could have written my examples with async/await, I have used the future syntax but it is the same. I can rewrite my examples if you prefer, I will just use other keywords. That is my opinon, we differ on this but I think there is something really wrong when you add unjustified syntax. You suggest gevent but where do you see async await in gevent ?
I perfectly understand that it is late to implement this, but we could take also into account the real limitations that are difficult to overcome with asyncio. I think I do not need to post links, many will undertand the constraints I am talking about. Regards, Chris On Sun, 16 Dec 2018 at 10:16, Nathaniel Smith <njs@pobox.com> wrote:
On Sun, Dec 16, 2018 at 09:21:14AM +0100, Christophe Bailly wrote:
Async/Await logic has introduced a symetric relation wich introduces unnecessary contraints. We should just the same logic as thread logic.
I'm not an expert on async, but whenever I hear people saying "we should use (something just like) threads" I'm reminded of something that Jamie Zawinski could have (but didn't) say: Some people, when confronted with a problem, think "I know, I'll use threads." Nothwey htwo pavroble ems. I know, that's a sound-bite, not a reasoned argument. But if your intention is to make async code just like threads, how do you avoid the well-known perils of threading? The point of async code is to make context switches explicit, rather than implicit like threading. So at first glance, it seems like you are suggesting we take the major benefit of async (explicitness) and replace it with the major disadvantage of other concurrency models (implicitness). -- Steve
Christophe Bailly wrote:
All of your examples there are C++. It's not clear how any of this relates to Python.
Do we have the tools to do this ? Yes because thanks to boost::context we can switch context between tasks.
How does it work? Is it pulling some kind of stack switching trick? If so, I would be skeptical about the portability and reliability. Also, if it requires the Python interpreter to become C++ or depend on a C++ runtime, it's not going to be accepted. -- Greg
participants (4)
-
Christophe Bailly -
Greg Ewing -
Nathaniel Smith -
Steven D'Aprano