Async-sig February 2017

Download

async-sig@python.org

3 participants
2 discussions

Debugging destroyed, pending tasks
by Mark E. Haase 03 Feb '17

03 Feb '17

I'm still climbing up the asyncio learning curve. I'm building an application on top of asyncio, and one of my goals is that when the application is interrupted (e.g. SIGINT), it should carefully clean up all running tasks before closing the event loop. Some tasks should be cancelled right away, but other tasks should be allowed to complete. As an example of the latter, I may have several file download tasks that I want to finish before closing the event loop. (If I get a 2nd SIGINT, then I close the loop immediately and exit.) I have a hard time debugging the "Task was destroyed..." errors. For example: Task was destroyed but it is pending! task: <Task pending coro=<sleep() done, defined at /usr/lib/python3.5/asyncio/tasks.py:502> wait_for=<Future pending cb=[Task._wakeup()]> cb=[_wait.<locals>._on_completion() at /usr/lib/python3.5/asyncio/tasks.py:414]> I'm pretty sure that I understand why I get this error: this is a task (e.g. created with asyncio.ensure_future()) that was never awaited. To prevent this error, I should await the task (or call its result() method, right?). But how do I figure out where the task was created? I have PYTHONASYNCIODEBUG=1 in my environment, and I see the "defined at" text, which is helpful if the coroutine is in my own code, but for asyncio.sleep(), it's not very helpful. Is it possible to see where this task was created, e.g. the line containing asyncio.ensure_future(asyncio.sleep(...)) ? If not, is this a capability that could be added to debug mode? E.g. ensure_future would add its own stack trace to the task object so that it could be retrieved later when a pending task is destroyed. In my current scenario, I have few enough tasks that there's only one place I think this particular error could come from, but as my application grows, this will get harder and harder to debug. Thoughts?

2 2

Re: [Async-sig] Some thoughts on asynchronous API design in a post-async/await world
by Dima Tisnek 03 Feb '17

03 Feb '17

Hi list, sorry for being a bit late, but I only discovered that post recently... I've a couple concerns: 1. update now 3.6 is out, the post should be rewritten or new one made, mainly because this post if what inquisitive minds find... 2. asyncio for whom? I've presented asyncio tutorial of sourts at a local Python meetup. I've noticed that asyncio documentation is pretty hard to understand. There are many concepts and a ton of caveats. I now finally understand why, and I think it comes down to asyncio being 2 distinct things: a. asyncio is a standard library meant to be used as a framework by other, higher-level, 3rd party libraries that are still written by very smart developers, and b. asyncio is a way for average joe developers to write async code. I used to think I was in cat.a., but clearly I'm more of a cat.b. dev :( I really with there was decent advice how to use asyncio/aiohttp/pytest-asyncio/curio? etc. Something along the lines of: * if you want fire-and-forget (e.g. async logging), do xyz1 * if you want a worker task, do xyz2 * if you want a synchronisation point, do xyz3 * if you want ... I'd love these to be reflected in the API that Python advertises. For example, I'd rather think of a Future() as a thing that provides a result, meaning it's commonly shared by multiple tasks, meaning it would by default be shielded. Should a Future() die if it's garbage-collected, like a generator or should it conitnue? These are areas where Python should provide best practice. Task cancellation (just like thread cancellation) is by definition(?) arcane, and I'd argue average developer should be advised against... I for one, still can't make up my mind what's better -- to cancel bottom or top frame when task being cancelled awaits something that awaits something else, etc... Should provision to block cancellation even exist? What would be the best quorum for these discussions? Cheers, d. P.S. I'm trying to avoid "coroutine" nomenclature on purpose

1 0