[Async-sig] async/sync library reusage

[Ben Darnell]

On Fri, Jun 9, 2017 at 11:51 AM Cory Benfield <cory at lukasa.co.uk> wrote:

>
>
> My concern with multiple loops boils down to the fact that urllib3
> supports being used in a multithreaded context where each thread can
> independently make forward progress on one request. To establish that with
> a synchronous codebase you either need one event loop per thread or you
> need to spawn a background thread on startup that owns the only event loop
> in the process.
>

Yeah, one event loop per thread is probably the way to go for integration
with synchronous codebases. A dedicated event loop thread may perform
better but libraries that spawn threads are problematic.


>
> Generally speaking I’ve not had positive results with libraries spawning
> their own threads in Python. In my experience this has tended to lead to
> programs that deadlock mysteriously or that fail to terminate in the face
> of a Ctrl+C. So I tend to prefer to have users spawn their own threads,
> which would make me want a “one-event-loop-per-thread” model: hence,
> needing a loop parameter to pass around prior to 3.6.
>

You can avoid the loop parameter on older versions of asyncio (at least as
long as the default event loop policy is used) by manually setting your
event loop as current before calling run_until_complete (and resetting it
afterwards).

Tornado's run_sync() method is equivalent to asyncio's
run_until_complete(), and Tornado supports multiple IOLoops in this way. We
use this to expose a synchronous version of our AsyncHTTPClient:
https://github.com/tornadoweb/tornado/blob/62e47215ce12aee83f951758c96775a43e80475b/tornado/httpclient.py#L54

-Ben


>
> I admit that my concerns here regarding libraries spawning their own
> threads may be overblown: after my series of negative experiences I
> basically never went back to that model, and it may be that the problems
> were more user-error than anything else. However, I feel comfortable saying
> that libraries spawning their own Python threads is definitely subtle and
> hard to get right, at the very least.
>
> Cory
> _______________________________________________
> Async-sig mailing list
> Async-sig at python.org
> https://mail.python.org/mailman/listinfo/async-sig
> Code of Conduct: https://www.python.org/psf/codeofconduct/
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/async-sig/attachments/20170609/97f6b50c/attachment.html>