Mailman 3 - Async-sig

bioopenssl: ssl protocol for asyncio but using pyOpenSSL
by Daniel Holth 12 Jul '19

12 Jul '19

This is my first asyncio project. I will probably not finish it, but let me know if anyone's interested in picking up where I left off. It is `asyncio.sslproto`, but with all calls to `ssl` replaced with equivalent calls to pyOpenSSL's `OpenSSL.SSL` module. It passes some of the tests, I think it is basically working. This is useful because the built in `ssl` module is missing some features compared to pyopenssl. However the pyopenssl version would require additional work to do critical things like certificate and hostname checking. https://bitbucket.org/dholth/bioopenssl/src/default/ Thanks, Daniel Holth

1 1

Preventing `yield` inside certain context managers
by Nathaniel Smith 02 Apr '19

02 Apr '19

This proposal might be of interest to folks here: https://discuss.python.org/t/preventing-yield-inside-certain-context-manage… I posted the main text there instead of here because it's a change to the core interpreter, but the motivation is async APIs. Probably best to post replies there, to keep the discussion consolidated. -n -- Nathaniel J. Smith -- https://vorpus.org

1 0

Inadvertent layering of synchronous code as frameworks adopt asyncio
by Daniel Nugent 27 Mar '19

27 Mar '19

Hello, I was hoping that the Async SIG might have some suggestions on how to deal with this sort of issue: More frameworks are adopting asyncio as time marches on. A notable example of this is Jupyter and the Python kernels it supports (please see announcement here: blog.jupyter.org/ipython-7-0-async-repl-a35ce050f7f7). This was enabled by a change in Tornado version 5.0 to support the asyncio event loop. The problem is that this makes any code which inadvertently ran an asyncio event loop (that is, calls through a blocking API provided by a library implemented in asyncio) fail. The Jupyter developers seem to feel that this is a deficiency in the asyncio event loop model and suggest all users encountering such a problem adopt the patch module nest_asyncio (github.com/jupyter/notebook/issues/3397#issuecomment-419474214). However, it is my understanding that the Python team strongly feels this is not the correct path: bugs.python.org/issue33523bugs.python.org/issue29558bugs.python.org/issue22… I have been trying to figure out the right way to work around this issue such that a library implemented with asyncio that provides a synchronous API will not cause a problem and have come up short thus far. I was considering investigating the janus sync/async queue as a way of facilitating communication between the different modes, but I am not sure that the scenario I describe reflects the intended usage. That is, an outer asyncio driven program fragment calls into middle synchronous code, which calls to inner asynchronous code. It seems that janus is mostly intended to facilitate communication between a single outer asynchronous layer and an inner synchronous layer. However, the documentation is a little sparse so I may just not understand it yet. I don't believe I'm the only person struggling to figure out how to deal with this sort of situation, so I think this would be useful for the community to figure out a solid answer to. For example, I found this blog post which outlines the same sort of problem and suggests that they elected to use nest_asyncio threespeedlogic.com/python-tworoutines.html If anyone could provide guidance on how to go forward, I would appreciate it. I would also like to understand the decision making around not allowing event loop nesting/reentrancy as seen in the bugs.python.orgissues I referenced so that I may explain the tradeoffs of possibly adopting the nest_asyncio patch module (for the sake of argument, lets ignore the possible issues with non-standard event loops) better to my peers. Thank you, -Dan Nugent

7 21

Reliably make unhandled exceptions crash the event loop
by Josh Quigley 27 Feb '19

27 Feb '19

Hi, I have been trying to make unhandled exceptions reliably crash the event loop (eg replicated behaviour of boost::asio, for those familiar with that C++ library). I'm aiming to have any exception bubble up from run_forever or run_until_complete style functions. I had thought I had a perfectly acceptable solution and then hit a strange case that threw my understanding of the way the loop worked. In 'test_b' below, the only difference is we keep a reference to the created task. This test hangs - the exception is raised, but the custom exception handler is never called. I'd be very interested to understand exactly why this happens. I'd also appreciate any feedback on the best way to reliably crash the event loop on unhandled exceptions (my next attempt will be to replace AbstractEventLoop.call_exception and see what happens). # example.py import asyncio class CustomException(RuntimeError): pass class UnhandledExceptionError(RuntimeError): pass def run_until_unhandled_exception(*, loop=None): """Run the event until there is an unhandled error in a callback This function sets the exception handler on the loop """ loop = loop if loop is not None else asyncio.get_event_loop() ex = [] def handler(loop, context): print('handler') loop.default_exception_handler(context) loop.stop() ex.append(context.get('exception')) loop.set_exception_handler(handler) loop.run_forever() if len(ex) > 0: raise UnhandledExceptionError('Unhandled exception in loop') from ex[0] async def fail_after(delay): await asyncio.sleep(delay) print('raise CustomException(...)') raise CustomException(f'fail_after(delay={delay})') async def finish_after(delay): await asyncio.sleep(delay) return delay def test_a(event_loop): event_loop.create_task(fail_after(0.01)) run_until_unhandled_exception(loop=event_loop) def test_b(event_loop): task = event_loop.create_task(fail_after(0.01)) run_until_unhandled_exception(loop=event_loop) def run_test(test): try: test(asyncio.get_event_loop()) except Exception as ex: print(ex) if __name__ == '__main__': run_test(test_a) run_test(test_b) # This hangs

3 4

killing tasks that won't cancel
by Chris Jerdonek 20 Feb '19

20 Feb '19

I have an asyncio question. In Python 3.7, is there a way to reliably end a task after having already tried calling cancel() on it and waiting for it to end? In Python 3.6, I did this with task.set_exception(), but in 3.7 that method was removed. --Chris

4 11

set_blocking_signal_threshold equivalent for asyncio?
by Yusuke Tsutsumi 13 Feb '19

13 Feb '19

Hi, In tornado, there's a really nice feature called set_blocking_signal_threshold, which sets a signal that fires if a coroutine has been running for too long without returning control back to the main loop: https://www.tornadoweb.org/en/stable/ioloop.html?highlight=signal#tornado.i… In tornado this will then log the traceback of the coroutine in question. This has been a very valuable tool when a developer accidentally introduces code that blocks the event loop for way too long of a time. Is there an equivalent in asyncio? I have a sketch in my head of how to implement that, but wanted to see if it existed somewhere first. Thanks! -Yusuke

2 3

AI_V4MAPPED on Windows
by Jan Seeger 04 Feb '19

04 Feb '19

Greetings! I'm working with aiocoap, which uses the AI_V4MAPPED flag to use IPv4-mapped addresses for dual-stack support. When trying to run on Windows, creating a connection fails, because the socket option IPV6_V6ONLY is set to true per default on Windows, whereas the value is configurable on Linux. I've attached a file that should reproduce the error. A possible fix would be calling socket.setsockopt(socket.IPPROTO_IPV6, socket.IPV6_V6ONLY, False) when V4-mapped addresses have been requested (this bug can also appear on Linux when /proc/sys/net/ipv6/bindv6only contains 1). If you require any more information, feel free to contact me! Best Regards, Jan Seeger PS: I am not a subscriber of this list, so please leave my address on any replies you send.

2 1

await question
by Pradip Caulagi 11 Dec '18

11 Dec '18

I was wondering if every use of 'await' should return the control to event loop? So in this example - https://gist.github.com/caulagi/3edea8cf734495f2592528a48f99e1d2 - I was hoping I would see 'A', 'B' to be mixed, but I see only 'A' followed by 'B'. What am I missing? I am using Python 3.7.1. How is my example different from https://docs.python.org/3.7/library/asyncio-task.html#asyncio.run? Thanks.

3 4

concurrency mistake
by saurabh singh 28 Aug '18

28 Aug '18

my question is 1st one is concurrent but 2nd one is not, how and please correct me, what i miss and what should i know more thank you import asyncio # 1st code async def say(what, when): await asyncio.sleep(when) print(what) loop = asyncio.get_event_loop() loop.create_task(say('first hello', 2)) loop.create_task(say('second hello', 1)) loop.run_forever() loop.close() ''' result >>> second hello >>> first hello ''' # 2nd code async def say(what, when): await asyncio.sleep(when) print(what) async def main(loop): yield from loop.create_task(say('first hello', 2)) yield from loop.create_task(say('second hello', 1)) print('close') loop = asyncio.get_event_loop() loop.run_until_complete(main(loop)) loop.close() ''' result >>> first hello >>> second hello '''

4 3

new library: sniffio – Sniff out which async library your code is running under
by Nathaniel Smith 21 Aug '18

21 Aug '18

Hi all, A number of people are working on packages that support multiple async backends (e.g., asyncio + trio, or trio + curio, or trio + twisted, ...). So then the question arises... how can I figure out which async library my user is actually using? Answer: install sniffio, and then call sniffio.current_async_library(), and it tells you. Well, right now it only works for trio and asyncio, but if you maintain an async library and you want to make it easier for packages to detect you, then it's easy to add support – see the manual. We considered various clever things, but ultimately decided that the best approach was to use a ContextVar and make it the coroutine runner's responsibility to advertise which async flavor it uses. In particular, this approach works even for hybrid programs that are using multiple coroutine runners in the same loop, like a Twisted program with asyncio-flavored and twisted-flavored coroutines in the same thread, or a Trio program using trio-asyncio to run both asyncio-flavored and trio-flavored coroutines in the same thread. Github: https://github.com/python-trio/sniffio Manual: https://sniffio.readthedocs.io/ PyPI: https://pypi.org/p/sniffio -n -- Nathaniel J. Smith -- https://vorpus.org

5 14