+1 for PEP, nothing more to add from technical point of view. An extra step to the right direction, at least to me. Thank you Yury for that :-) About side conversation on sync/async split world, except to force coroutines pattern usage like in Go, I don't see how we can become more implicit. Even if the zen of Python recommands to prefer an explicit approach, I see more explicit/implicit as a balance you must adjust between Simplicity/Flexibility than a binary choice. To me, the success of Python as language is also because you have a good balance between theses approaches, and the last move from "yield from" to "await" illustrates that: Hide the internal mechanisms of implementation, but keep the explicit way to declare that. Like Andrew Svetlov, I don't believe a lot in the implicit approach of Gevent, because very quickly, you need to understand the extra tools, like synchronization primitives. The fact to know if you need to prefix with "await" or not the call of the functions is the tree that hides the forest. With the async pattern, it's impossible to hide everything and everything will work automagically: You must understand a little bit what's happening, or it will be very complicated to debug. To me, you can hide everything only if you are really sure it will work 100% of time without human intervention, like with autonomous Google cars. However, it might be interesting to have an async "linter", that should list all blocking I/O code in async coroutines, to help new comers to find this type of bugs. But with the dynamic nature of Python, I don't know if it's realistic to try to implement that. To me, it should be a better answer than to try to remove all sync/async code differences. Moreover, I see the need of async libs as an extra opportunity to challenge and simplify the Python toolbox. For now, with aiohttp, you have an unified API for HTTP in general, contrary in sync world with requests and flask for example. At least to me, a client and a server are only two sides of the same piece. More true with p2p protocols. As discussed several times, the next level might be more code reuse like suggested by David Beazley with SansIO, split protocol and I/O handling: https://twitter.com/dabeaz/status/761599925444550656?lang=fr https://github.com/brettcannon/sans-io I don't know yet if the benefit to share more code between implementations will be more important than the potential complexity code increase. The only point I'm sure for now: I'm preparing the pop-corn to watch the next episodes: curious to see what are the next ideas/implementations will emerge ;-) At least to me, it's more interesting than follow a TV serie, thank you for that :-) Have a nice week. Ludovic Gasc (GMLudo) http://www.gmludo.eu/ On 29 Jul 2016 20:50, "Yarko Tymciurak" <yarkot1@gmail.com> wrote:
On Friday, July 29, 2016, Yury Selivanov <yselivanov@gmail.com> wrote:
Comments inlined:
On Jul 29, 2016, at 2:20 PM, Yarko Tymciurak <yarkot1@gmail.com> wrote:
Hmm... I think we need to think about a future where, programmatically, there's little-to no distinction between async and synchronous functions. Pushing this down deeper in the system is the way to go. For one, it will serve simple multi-core use once gilectomy is completed (it, or something effectively equivalent will complete). For another, this is the path to reducing the functionally "useless" rewrite efforts of libraries (e.g. github.com/aio-libs), which somehow resemble all the efforts of migrating libraries from 2-3 (loosely). The resistance and unexpected time that 2-3 migration experienced won't readily be mimicked in async tasks - too much effort to get computer and I/O bound benefits? Maintain two versions of needed libraries, or jump languages is what will increasingly happen in the distributed (and more so IOT) world.
When and *if* gilectomy is completed (or another project to remove the GIL), we will be able to do this:
1) Run existing async/await applications as is, but instead of running a process per core, we will be able to run a single process with many threads. Likely one asyncio (or other) event loop per thread. This is very speculative, but possible in theory.
2) Run existing blocking IO applications in several threads in one process. This is something that only sounds like an easy thing to do, I suspect that a lot of code will break (or dead lock) when the GIL is removed. Even if everything works perfectly well, threads aren’t answer to all problems — try to manage 1000s of them.
Long story short, even if we had no GIL at all, having async/await (and non-blocking IO) would make sense. And if you have async/await, with GIL or without, you will inevitably have different APIs and different IO low-level libs that drive them.
There are ways to lessen the pain. For instance, I like Cory’s approach with hyper - implement protocols separately from IO, so that it’s easy to port them to various sync and async frameworks.
Time to think about paving the way to async-as first class citizen
world.
That's probably too much for this PEP, but the topic (a- prefixing) is
a good canary for the bigger picture we need to start mulling over.
So in this context (and in general w/ async) asking the question "can
we make it so it doesn't matter?" is a good one to always be asking - it will get is there.
Unfortunately there is no way to use the same APIs for both async/await and synchronous world. At least for CPython builtin types they have to have different names.
I’m fine to discuss the ‘a’ prefix, but I’m a bit afraid that focusing on it too much will distract us from the PEP and details of it that really matter.
Yury
To keep it simple, try thinking like this (and yes, Yury, apologies - this is now a side discussion, and not about this pep): everything in CPython is async, and if you don't want async, you don't need to know about, you run a single async task and don't need to know more...
Can we get there? That would be cool...
- Yarko
_______________________________________________ Async-sig mailing list Async-sig@python.org https://mail.python.org/mailman/listinfo/async-sig Code of Conduct: https://www.python.org/psf/codeofconduct/