On Fri, Jun 9, 2017 at 11:51 AM Cory Benfield <cory@lukasa.co.uk> wrote:

My concern with multiple loops boils down to the fact that urllib3 supports being used in a multithreaded context where each thread can independently make forward progress on one request. To establish that with a synchronous codebase you either need one event loop per thread or you need to spawn a background thread on startup that owns the only event loop in the process.

Yeah, one event loop per thread is probably the way to go for integration with synchronous codebases. A dedicated event loop thread may perform better but libraries that spawn threads are problematic.

Generally speaking I’ve not had positive results with libraries spawning their own threads in Python. In my experience this has tended to lead to programs that deadlock mysteriously or that fail to terminate in the face of a Ctrl+C. So I tend to prefer to have users spawn their own threads, which would make me want a “one-event-loop-per-thread” model: hence, needing a loop parameter to pass around prior to 3.6.

You can avoid the loop parameter on older versions of asyncio (at least as long as the default event loop policy is used) by manually setting your event loop as current before calling run_until_complete (and resetting it afterwards).

Tornado's run_sync() method is equivalent to asyncio's run_until_complete(), and Tornado supports multiple IOLoops in this way. We use this to expose a synchronous version of our AsyncHTTPClient: https://github.com/tornadoweb/tornado/blob/62e47215ce12aee83f951758c96775a43e80475b/tornado/httpclient.py#L54

-Ben

I admit that my concerns here regarding libraries spawning their own threads may be overblown: after my series of negative experiences I basically never went back to that model, and it may be that the problems were more user-error than anything else. However, I feel comfortable saying that libraries spawning their own Python threads is definitely subtle and hard to get right, at the very least.

Cory
_______________________________________________
Async-sig mailing list
Async-sig@python.org
https://mail.python.org/mailman/listinfo/async-sig
Code of Conduct: https://www.python.org/psf/codeofconduct/