On Jul 6, 2016, at 9:44 PM, Nathaniel Smith <njs@pobox.com> wrote:
On Wed, Jul 6, 2016 at 6:17 PM, Yury Selivanov <yselivanov@gmail.com> wrote:
...does it actually work to re-enter a main loop from inside a __del__ callback? It seems like you could get into really nasty states with multiple nested __del__ calls, or if a single sweep detects multiple pieces of garbage with __del__ methods, then some of those __del__ calls could be delayed indefinitely while the first __del__ runs. Is the cycle collector even re-entrant?
We can have a flag on the async gen object to make sure that we run the finalizer only once. The finalizer will likely schedule async_gen.aclose() coroutine which will ensure a strong ref to the gen until it is closed. This can actually work.. ;)
Hmm, if the strategy is to schedule the work to happen outside of the actual __del__ call, then I think this is back to assuming that all coroutine runners are immortal and always running. Is that an assumption you're comfortable with?
No need to require coroutine runners to be immortal. If a finalizer finds out that its event loop is closed, it does nothing. The interpreter will then issue a ResourceWarning if the generator wasn’t properly closed. This is how a finalizer for asyncio event loop might look like: def _finalize_gen(self, gen): if not self.is_closed(): self.create_task(gen.aclose()) And this is how asyncio loop might set it up: class AsyncioEventLoop: def run_forever(): ... old_finalizer = sys.get_async_generator_finalizer() sys.set_async_generator_finalizer(self._finalize_gen) try: while True: self._run_once() … finally: … sys.set_async_generator_finalizer(old_finalizer) Why do I think that it’s OK that some async generators might end up not being properly closed? Because this can already happen to coroutines: async def coro1(): try: print('try') await asyncio.sleep(1) finally: await asyncio.sleep(0) print('finally') async def coro2(): await asyncio.sleep(0) loop = asyncio.get_event_loop() loop.create_task(coro1()) loop.run_until_complete(coro2()) In other words, an event loop might stop or be interrupted, and there is no way to guarantee that all coroutines will be finalized properly in that case. To address Glyph’s point about many event loops in one process: a finalizer set with set_async_generator_finalizer() (which should be thread specific, same as set_coroutine_wrapper) can actually be assigned to the async generator when it’s instantiated. Yury