asyncio - run coroutine in the background

Frank Millman frank at chagford.com
Tue Feb 16 10:21:04 EST 2016


"Kevin Conway"  wrote in message 
news:CAKF=+dhXZ=yax8STAWr_gjX3Tg8yUjPRJG-7yM2_bRV2Kxm3Jg at mail.gmail.com...
>
> > My background task does take a long time to run - about 10 seconds - but
> > most of that time is spent waiting for database responses, which is 
> > handled
> > in another thread.
>
> Something else to look into is an asyncio driver for your database
> connections. Threads aren't inherently harmful, but using them to achieve
> async networking when running asyncio is a definite code smell since that
> is precisely the problem asyncio is supposed to solve for.
>

Maybe I have not explained very well. I am not using threads to achieve 
async networking. I am using asyncio in a client server environment, and it 
works very well. If a client request involves a database query, I use a 
thread to perform that so that it does not slow down the other users. I 
usually want the originating client to block until I have a response, so I 
use 'await'. However, occasionally the request takes some time, and it is 
not necessary for the client to wait for the response, so I want to unblock 
the client straight away, run the task in the background, and then notify 
the client when the task is complete. This is where your suggestion of 
'ensure_future' does the job perfectly.

I would love to drive the database asynchronously, but of the three 
databases I use, only psycopg2 seems to have asyncio support. As my 
home-grown solution (using queues) seems to be working well so far, I am 
sticking with that until I start to experience responsiveness issues. If 
that happens, my first line of attack will be to switch from threads to 
processes.

Hope this makes sense.

Frank




More information about the Python-list mailing list