[Tutor] What exactly does "await" do? (David)(Mats Wichmann)(Alan Gauld)

[ThreeBlindQuarks]

Alphonsus,

It looks like you copied paragraphs from a textbook so are we supposed to debate that?

I think that various ways that have been developed to do asynchronous programming are probably not beginner topics. They represent very different ways of looking at problems and implementing solutions.

Think of it more simply. The simplest Python programs do things a step at a time in a rather deterministic way and that is fine for solving lots of problems albeit not always the most efficient way.

Python added concepts where a piece of code does not need to finish all at once. Instead of using something like a list comprehension that makes a list of the first million primes and stores them in memory before your program can use the ones it wants, they made several kind of generator objects that are called and yield one result and are put into stasis. Your program can use that result and if it wants more, the generator starts up again where it left off. If after a dozen primes, your program is done, the remaining 999_987 are never even calculated. 

Python thus allows you to link up multiple such generators in ways that produce results just in time as many parts of the program run only as needed. But they do not run at the same time. Each part that calls another is suspended while the other runs and then continues when it receives what the other has supplied.

Python then allows various ways to multitask that have been around for a while. You can start additional full-blown programs or variations on lightweight threads and these can allow some level of independence as the operating system, or a scheduler within python, regularly stops running code and schedules another and eventually restarts the current code. On machines with many CPU, some ways allow things to really run simultaneously and others still run within the same process sharing the same CPU and more. But you have little control on when a process is suspended or animated.

In this context, we have the concept of what it means to wait. Programs that make various requests such as to read a file or do something over a network or even access memory not in the cache, get suspended until the request is ready to be handled. But what if you want to download many images (or anything else) from out there or write it out to somewhere or use some form of interprocess communication and so on? Perhaps your main program has things to do that don't care if the previous operation is finished yet. As an example, you may want to read in a file and not wait but order the next file and not wait and so on. Maybe you want to also have some display monitor the progress of the downloads and show what percent has completed? So you want to start such a process too. Finally, when lots of pieces of code are busily doing what you wish, in some interleaved way, you know you need to wait till one (or all) are done before continuing.

The above may barely catch the flavor of what asynchronous means. Python grafted many features on way late in the game and even some other languages that built it in early keep evolving ways to do more.

What does await do? It depends. 

Sometimes you use it directly as you describe so that the caller waits but sometimes it is more subtle and the waiting is done by some other routine. An example is when you call a routine that is itself a generator that monitors your many asynchronous offspring and returns a token of the first one that finishes. The main body of your program (or ANY function like this) can use a sort of FOR loop on the generator and get back one at a time. But it does not get back a result from the asynchronous task but just a sort of object representing it. It can then await on that object and it immediately returns with the result from that object. This is not really waiting but a reuse of the keyword to mean they are ready to inspect the results.

Writing asynchronously can be very hard to do properly and takes some serious effort to learn. I repeat, not really beginner stuff.

Then again, in languages like JavaScript/NodeJS some people think it is the normal way to do things!




Sent with Proton Mail secure email.

------- Original Message -------
On Friday, February 17th, 2023 at 9:43 PM, Alphonsus <phonokoye at gmail.com> wrote:


> Good day to you Mr David, Mr Mats and Mr. Alan and also to everyone on
> this mailing list, I do appreciate your response to my questions which
> have opened up my understanding on the subject, I also apologize for
> my delayed response. With this mail message I hope to summarize my
> idea of the subject from your messages and other sources on the web.
> Please call me out when my statements are wrong. Thank you in advance.
> 
> asyncio is python's implementation of Asynchronous IO which in turn is
> a form of IO where other tasks may continue before the IO transmission
> has ended. This implementation makes use of generators, coroutines,
> Task and Future objects.
> Coroutines are further divided into simple and native coroutines,
> simple coroutines are generators and native coroutines are blocks of
> code enclosed in an async def header.
> 
> The await keyword is used to suspend the execution of the surrounding
> coroutine and begin the execution of the awaited coroutine. (by
> coroutine, I speak of the native coroutine) The await function in
> itself does not specify that the coroutine will run concurrently with
> some other coroutine and coroutines can only run concurrently with
> coroutines, but we have to explicitly specify that the coroutines will
> run concurrently by using asyncio.gather or creating tasks using
> asyncio.create_task and then awaiting the tasks. But all these are
> just high level stuff.
> 
> The low level stuff involves the asyncio event loop, Future objects
> and some concept called a Transport. All of these may not be used
> frequently but are important for greater control over an asyncio
> program. These parts of asyncio are probably the most tricky parts. It
> is important to understand these parts though, so as to make better
> programming decisions. I also do not understand the low level approach
> to IO in python. Can I get references that teach these?
> _______________________________________________
> Tutor maillist - Tutor at python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor