async_stagger: Happy Eyeballs implementation in pure asyncio
Repo: https://github.com/twisteroidambassador/async_stagger Docs: http://async-stagger.readthedocs.io/en/latest/ Provides near drop-in replacements for open_connection() and create_connection() using Happy Eyeballs. Also exposes the underlying scheduling logic where you can plug in your own coroutines to run. I basically ported trio's implementation to asyncio, and it turned out to be not too difficult. Cheers, twistero
Yury, This looks like good work.Would it make sense to add this to asyncio in 3.8? --Guido On Sun, May 13, 2018 at 1:40 AM, twisteroid ambassador < twisteroid.ambassador@gmail.com> wrote:
Repo: https://github.com/twisteroidambassador/async_stagger Docs: http://async-stagger.readthedocs.io/en/latest/
Provides near drop-in replacements for open_connection() and create_connection() using Happy Eyeballs. Also exposes the underlying scheduling logic where you can plug in your own coroutines to run.
I basically ported trio's implementation to asyncio, and it turned out to be not too difficult.
Cheers,
twistero _______________________________________________ Async-sig mailing list Async-sig@python.org https://mail.python.org/mailman/listinfo/async-sig Code of Conduct: https://www.python.org/psf/codeofconduct/
-- --Guido van Rossum (python.org/~guido)
On May 13, 2018, 9:35 AM -0400, Guido van Rossum <guido@python.org>, wrote:
Yury,
This looks like good work.Would it make sense to add this to asyncio in 3.8?
Yes, it is solid. I'd like to see this in asyncio; specifically, I suggest to add a keyword-only argument to loop.create_connection & asyncio.open_connection to use happy eyeballs (off by default). Exposing the "staggered_race()" helper function might also be a good idea, I'm just not super happy with the name. Twistero, would you be interested in submitting a PR? Yury
Sure, I should be able to massage the code into asyncio. Probably will need substantial help on writing any tests, though. On Tue, May 15, 2018 at 2:29 AM, Yury Selivanov <yselivanov@gmail.com> wrote:
On May 13, 2018, 9:35 AM -0400, Guido van Rossum <guido@python.org>, wrote:
Yury,
This looks like good work.Would it make sense to add this to asyncio in 3.8?
Yes, it is solid. I'd like to see this in asyncio; specifically, I suggest to add a keyword-only argument to loop.create_connection & asyncio.open_connection to use happy eyeballs (off by default). Exposing the "staggered_race()" helper function might also be a good idea, I'm just not super happy with the name.
Twistero, would you be interested in submitting a PR?
Yury
Just created an Issue for this: https://bugs.python.org/issue33530 On Tue, May 15, 2018 at 2:07 PM, twisteroid ambassador <twisteroid.ambassador@gmail.com> wrote:
Sure, I should be able to massage the code into asyncio. Probably will need substantial help on writing any tests, though.
On Tue, May 15, 2018 at 2:29 AM, Yury Selivanov <yselivanov@gmail.com> wrote:
On May 13, 2018, 9:35 AM -0400, Guido van Rossum <guido@python.org>, wrote:
Yury,
This looks like good work.Would it make sense to add this to asyncio in 3.8?
Yes, it is solid. I'd like to see this in asyncio; specifically, I suggest to add a keyword-only argument to loop.create_connection & asyncio.open_connection to use happy eyeballs (off by default). Exposing the "staggered_race()" helper function might also be a good idea, I'm just not super happy with the name.
Twistero, would you be interested in submitting a PR?
Yury
Hi, Great idea, I didn't know that it was already implemented by Trio and I didn't know that there were two RFCs to specify how to make this. In the case of the connection that strategy can bring many benefits in networks that are dynamic, perhaps the case of the ELBs in AWS which scale up or shrink the number of load balances that are in front of your application taking into account the traffic that they have to handle. That change gets reflected in the DNS answer, indeed AWS suggests to the clients to save the DNS resolutions in the cache client for not more than 60 seconds, obviously because the IP addresses will be adaptative to the number of load balancers at some specific time. Having a "big" connection timeout the client experience till reaching an available destination is proportional to the number of hosts iterated and failed for host in hosts: try: return await connect(host, timeout=1) except TimeoutError: logging.warning("Host not available, trying the next one") So definitely the proposal of the Happy Eyeballs implementation improves the latency needed to get a healthy connection. Thanks for making it visible! On Sun, May 13, 2018 at 7:40 AM, twisteroid ambassador <twisteroid.ambassador@gmail.com> wrote:
Repo: https://github.com/twisteroidambassador/async_stagger Docs: http://async-stagger.readthedocs.io/en/latest/
Provides near drop-in replacements for open_connection() and create_connection() using Happy Eyeballs. Also exposes the underlying scheduling logic where you can plug in your own coroutines to run.
I basically ported trio's implementation to asyncio, and it turned out to be not too difficult.
Cheers,
twistero _______________________________________________ Async-sig mailing list Async-sig@python.org https://mail.python.org/mailman/listinfo/async-sig Code of Conduct: https://www.python.org/psf/codeofconduct/
-- --pau
On Sun, May 13, 2018 at 1:40 AM, twisteroid ambassador <twisteroid.ambassador@gmail.com> wrote:
Repo: https://github.com/twisteroidambassador/async_stagger Docs: http://async-stagger.readthedocs.io/en/latest/
Provides near drop-in replacements for open_connection() and create_connection() using Happy Eyeballs. Also exposes the underlying scheduling logic where you can plug in your own coroutines to run.
I basically ported trio's implementation to asyncio, and it turned out to be not too difficult.
A few people in trio's chat channel have been experimenting with strategies for implementing it in gevent, and folks might find the strategies they've been coming up with interesting too: https://gist.github.com/davidkhess/bc213e643db2581ee830a1e706e85f8f https://gist.github.com/ssanderson/f625716602a4bd7c8ead0dd4befad8ea There's some discussion starting here, and continuing through today: https://gitter.im/python-trio/general?at=5afa458bb84be71db908becd -n -- Nathaniel J. Smith -- https://vorpus.org
participants (5)
-
Guido van Rossum -
Nathaniel Smith -
Pau Freixes -
twisteroid ambassador -
Yury Selivanov