[Python-ideas] Learning from the shell in supporting asyncio background calls

[Oscar Benjamin]

On 10 July 2015 at 11:49, Nick Coghlan <ncoghlan at gmail.com> wrote:
> run_in_background is akin to invoking a shell command with a trailing
> "&" - it puts the operation into the background, leaving the current
> thread to move on to the next operation (or wait for input at the
> REPL). When coroutines are scheduled, they won't start running until
> you start a foreground task, while callables delegated to the default
> executor will start running immediately.
>
> To actually get the *results* of that task, you have to run it in the
> foreground of the current thread using run_in_foreground - this is
> akin to bringing a background process to the foreground of a shell
> session using "fg".
>
> To relate this idea back to some of the examples Sven was discussing,
> here's how translating some old serialised synchronous code to use
> those APIs might look in practice:
>
>     # Serial synchronous data loading
>     def load_and_process_data():
>         data1 = load_remote_data_set1()
>         data2 = load_remote_data_set2()
>         return process_data(data1, data2)
>
>     # Parallel asynchronous data loading
>     def load_and_process_data():
>         future1 = asyncio.run_in_background(load_remote_data_set1_async())
>         future2 = asyncio.run_in_background(load_remote_data_set2_async())
>         data1 = asyncio.run_in_foreground(future1)
>         data2 = asyncio.run_in_foreground(future2)
>         return process_data(data1, data2)

Why is that better than something like:

     data1, data2 = asyncio.run([future1, future2])

IIUC your proposal is that run_in_background adds the tasks to an
implicit global variable. Then the first call to run_in_foreground
runs both tasks returning when future1 is ready. At that point it
suspends future2 if incomplete? Then the second call to
run_in_foreground returns immediately if future2 is ready or otherwise
runs that task until complete?

--
Oscar