Debugging destroyed, pending tasks

I'm still climbing up the asyncio learning curve. I'm building an application on top of asyncio, and one of my goals is that when the application is interrupted (e.g. SIGINT), it should carefully clean up all running tasks before closing the event loop. Some tasks should be cancelled right away, but other tasks should be allowed to complete. As an example of the latter, I may have several file download tasks that I want to finish before closing the event loop. (If I get a 2nd SIGINT, then I close the loop immediately and exit.) I have a hard time debugging the "Task was destroyed..." errors. For example: Task was destroyed but it is pending! task: <Task pending coro=<sleep() done, defined at /usr/lib/python3.5/asyncio/tasks.py:502> wait_for=<Future pending cb=[Task._wakeup()]> cb=[_wait.<locals>._on_completion() at /usr/lib/python3.5/asyncio/tasks.py:414]> I'm pretty sure that I understand why I get this error: this is a task (e.g. created with asyncio.ensure_future()) that was never awaited. To prevent this error, I should await the task (or call its result() method, right?). But how do I figure out where the task was created? I have PYTHONASYNCIODEBUG=1 in my environment, and I see the "defined at" text, which is helpful if the coroutine is in my own code, but for asyncio.sleep(), it's not very helpful. Is it possible to see where this task was created, e.g. the line containing asyncio.ensure_future(asyncio.sleep(...)) ? If not, is this a capability that could be added to debug mode? E.g. ensure_future would add its own stack trace to the task object so that it could be retrieved later when a pending task is destroyed. In my current scenario, I have few enough tasks that there's only one place I think this particular error could come from, but as my application grows, this will get harder and harder to debug. Thoughts?

On Feb 2, 2017, at 2:40 PM, Mark E. Haase <mehaase@gmail.com> wrote:
Is it possible to see where this task was created, e.g. the line containing asyncio.ensure_future(asyncio.sleep(...)) ? If not, is this a capability that could be added to debug mode? E.g. ensure_future would add its own stack trace to the task object so that it could be retrieved later when a pending task is destroyed.
Yeah, this is doable. I suggest you to open an issue on bugs.python.org. Yury

Thanks, Yury, for the quick reply, but I just realized my question was based on my own mistake. Asyncio already does exactly what I want, but I have a subprocess that wasn't inheriting PYTHONASYNCIODEBUG=1 from its parent. Once I've exported this environment variable to the subprocess, I am now seeing stack traces that show where pending tasks were created. Sorry everybody! Please ignore this thread... On Thu, Feb 2, 2017 at 2:53 PM, Yury Selivanov <yselivanov@gmail.com> wrote:
On Feb 2, 2017, at 2:40 PM, Mark E. Haase <mehaase@gmail.com> wrote:
Is it possible to see where this task was created, e.g. the line containing asyncio.ensure_future(asyncio.sleep(...)) ? If not, is this a capability that could be added to debug mode? E.g. ensure_future would add its own stack trace to the task object so that it could be retrieved later when a pending task is destroyed.
Yeah, this is doable. I suggest you to open an issue on bugs.python.org.
Yury
participants (2)
-
Mark E. Haase
-
Yury Selivanov