[Async-sig] Simplifying stack traces for tasks?
Mark E. Haase
mehaase at gmail.com
Fri Apr 13 11:55:38 EDT 2018
Thanks for the feedback! I put this aside for a while but I'm coming back
to it now and cleaning it up.
The approach used in this first post was obviously very clumsy. In my
latest version I am using module instance directly (as shown in Nathaniel's
reply) and using the qualified package name (as suggested by Roger). I
created an explicit blacklist (incomplete--still needs more testing) of
functions to hide in my custom backtraces and refactored a bit so I can
write tests for it. Code below.
One interesting thing I learned while working on this is that the
backtraces change depending on the asyncio debug mode, because in debug
mode couroutines are wrapped in CoroWrapper[1], which adds a frame
everytime a coroutine sends, throws, etc. So I am now thinking that my
custom excepthook is probably most useful in debug mode, but probably not
good to enable in production.
I'm working on a more general asyncio task group library that will include
this excepthook. I'll release the whole thing on PyPI when it's done.
[1]
https://github.com/python/cpython/blob/23ab5ee667a9b29014f6f7f01797c611f63ff743/Lib/asyncio/coroutines.py#L25
---
def _async_excepthook(type_, exc, tb):
'''
An ``excepthook`` that hides event loop internals and displays
task group information.
:param type type_: the exception type
:param Exception exc: the exception itself
:param tb tb: a traceback of the exception
'''
print(_async_excepthook_format(type_, exc, tb))
def _async_excepthook_format(type_, exc, tb):
'''
This helper function is used for testing.
:param type type_: the exception type
:param Exception exc: the exception itself
:param tracetack tb: a traceback of the exception
:return: the formatted traceback as a string
'''
format_str = ''
cause_exc = None
cause_str = None
if exc.__cause__ is not None:
cause_exc = exc.__cause__
cause_str = 'The above exception was the direct cause ' \
'of the following exception:'
elif exc.__context__ is not None and not exc.__suppress_context__:
cause_exc = exc.__context__
cause_str = '\nDuring handling of the above exception, ' \
'another exception occurred:'
if cause_exc:
format_str += _async_excepthook_format(type(cause_exc), cause_exc,
cause_exc.__traceback__)
if cause_str:
format_str += '\n{}\n\n'.format(cause_str)
format_str += 'Async Traceback (most recent call last):\n'
# Need file, line, function, text
for frame, line_no in traceback.walk_tb(tb):
if _async_excepthook_exclude(frame):
format_str += ' ---\n'
else:
code = frame.f_code
filename = code.co_filename
line = linecache.getline(filename, line_no).strip()
format_str += ' File "{}", line {}, in {}\n' \
.format(filename, line_no, code.co_name)
format_str += ' {}\n'.format(line)
format_str += '{}: {}'.format(type_.__name__, exc)
return format_str
_ASYNC_EXCEPTHOOK_BLACKLIST = {
'asyncio.base_events': ('_run_once', 'call_later', 'call_soon'),
'asyncio.coroutines': ('__next__', 'send', 'throw'),
'asyncio.events': ('__init__', '_run'),
'asyncio.tasks': ('_step', '_wakeup'),
'traceback': ('extract', 'extract_stack'),
}
def _async_excepthook_exclude(frame):
''' Return True if ``frame`` should be excluded from tracebacks. '''
module = frame.f_globals['__name__']
function = frame.f_code.co_name
return module in _ASYNC_EXCEPTHOOK_BLACKLIST and \
function in _ASYNC_EXCEPTHOOK_BLACKLIST[module]
On Tue, Nov 14, 2017 at 7:15 PM, Nathaniel Smith <njs at pobox.com> wrote:
> On Tue, Nov 14, 2017 at 2:00 PM, Roger Pate <rogerpate at gmail.com> wrote:
> > On Tue, Nov 14, 2017 at 9:54 AM, Mark E. Haase <mehaase at gmail.com>
> wrote:
> > ...
> >> print('Async Traceback (most recent call last):')
> >> for frame in traceback.extract_tb(tb):
> >> head, tail = os.path.split(frame.filename)
> >> if (head.endswith('asyncio') or tail == 'traceback.py') and
> \
> >> frame.name.startswith('_'):
> > ...
> >> The meat of it is towards the bottom, "if head.endswith('asyncio')..."
> There
> >> are a lot of debatable details and this implementation is pretty hacky
> and
> >> clumsy, but I have found it valuable in my own usage, and I haven't yet
> >> missed the omitted stack frames.
> >
> > It would be better to determine if the qualified module name is
> > "traceback" or starts with "asyncio." (or topmost package is
> > "asyncio", etc.) rather than allow false positives for
> > random_package.asyncio.module._any_function or
> > random_package.traceback._any_function. I don't see an easy way to
> > get the module object at this point in your hook; however:
>
> You can't get the module from the cooked data that extract_tb returns,
> but it's there in the tb object itself. This walks the traceback and
> prints each frame's module:
>
> current = tb
> while current is not None:
> print("Next module", current.tb_frame.f_globals.get("__name__"))
> current = current.tb_next
>
> -n
>
> --
> Nathaniel J. Smith -- https://vorpus.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/async-sig/attachments/20180413/5a8de716/attachment.html>
More information about the Async-sig
mailing list