[Python-ideas] Learning from the shell in supporting asyncio background calls

[Nick Coghlan]

On 11 July 2015 at 15:04, Nick Coghlan <ncoghlan at gmail.com> wrote:
> On 10 July 2015 at 21:51, Guido van Rossum <guido at python.org> wrote:
>> As I wrote on the issue, I'm -1 on this proposal. Not only does this API
>> encourage beginners to ignore the essential difference between synchronous
>> functions meant to run in a thread (using synchronous I/O and pre-emptive
>> CPU scheduling) and asyncio coroutines/tasks (which use overlapped I/O and
>> require explicit scheduling), it also encourages avoiding the "await"
>> primitive (formerly "yield from") in favor of a function call which cannot
>> be used from within a coroutine/task.
>
> My apologies for the confusion - the revised proposal focuses on
> coroutines, not threads. With the benefit of hindight, leaving the
> implementation details out of the python-ideas post was clearly a
> mistake, as my previous posts had been more focused on threads. I've
> added the full implementation details in my reply to Oscar, which will
> hopefully make the revised proposal clearer.

I wrote a second post about this foreground/background task idea,
which presents a hopefully more compelling example: setting up two TCP
echo servers from the interactive prompt, and then interacting with
them using asynchronous clients, including an example using
run_in_background, run_in_foreground and asyncio.wait to run parallel
client commands.

This is all done using the main thread in the REPL, and takes
advantage of the fact that it's all running in the same thread to
dynamically allocate the server ports and pass that information to the
demonstration clients. I believe this particular example effectively
demonstrates the power of asyncio to dramatically simplify the testing
of both network clients and network servers, as you don't need to mess
about with synchronising across threads or processes.

The full post is at
http://www.curiousefficiency.org/posts/2015/07/asyncio-tcp-echo-server.html,
but I'll include the examples of usage inline.

The code for setting up the servers and retrieving their chosen ports
looks like:

    >>> make_server = asyncio.start_server(handle_tcp_echo, '127.0.0.1')
    >>> server = run_in_foreground(make_server)
    >>> port = server.sockets[0].getsockname()[1]

    >>> make_server2 = asyncio.start_server(handle_tcp_echo, '127.0.0.1')
    >>> server2 = run_in_foreground(make_server2)
    >>> port2 = server2.sockets[0].getsockname()[1]

This is an effectively synchronous operation, so it could be readily
encapsulated in a normal function call for invocation from a test
suite to set up local test servers, and report the port number to
connect to.

The code for running parallel clients looks like:

    >>> echo1 = run_in_background(tcp_echo_client('Hello World!', port))
    >>> echo2 = run_in_background(tcp_echo_client('Hello World!', port2))
    >>> run_in_foreground(asyncio.wait([echo1, echo2]))
    >>> echo1.result()
    'Hello World!'
    >>> echo2.result()
    'Hello World!'

While I don't go into it in the post, blocking clients could also be
tested in much the same way, by using run_in_background's callable
support to run them as call-and-response operations through the
default executor, while running the asynchronous server components in
the main thread.

Regards,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia