Thanks for the feedback! I put this aside for a while but I'm coming back to it now and cleaning it up. The approach used in this first post was obviously very clumsy. In my latest version I am using module instance directly (as shown in Nathaniel's reply) and using the qualified package name (as suggested by Roger). I created an explicit blacklist (incomplete--still needs more testing) of functions to hide in my custom backtraces and refactored a bit so I can write tests for it. Code below. One interesting thing I learned while working on this is that the backtraces change depending on the asyncio debug mode, because in debug mode couroutines are wrapped in CoroWrapper[1], which adds a frame everytime a coroutine sends, throws, etc. So I am now thinking that my custom excepthook is probably most useful in debug mode, but probably not good to enable in production. I'm working on a more general asyncio task group library that will include this excepthook. I'll release the whole thing on PyPI when it's done. [1] https://github.com/python/cpython/blob/23ab5ee667a9b29014f6f7f01797c611f63ff... --- def _async_excepthook(type_, exc, tb): ''' An ``excepthook`` that hides event loop internals and displays task group information. :param type type_: the exception type :param Exception exc: the exception itself :param tb tb: a traceback of the exception ''' print(_async_excepthook_format(type_, exc, tb)) def _async_excepthook_format(type_, exc, tb): ''' This helper function is used for testing. :param type type_: the exception type :param Exception exc: the exception itself :param tracetack tb: a traceback of the exception :return: the formatted traceback as a string ''' format_str = '' cause_exc = None cause_str = None if exc.__cause__ is not None: cause_exc = exc.__cause__ cause_str = 'The above exception was the direct cause ' \ 'of the following exception:' elif exc.__context__ is not None and not exc.__suppress_context__: cause_exc = exc.__context__ cause_str = '\nDuring handling of the above exception, ' \ 'another exception occurred:' if cause_exc: format_str += _async_excepthook_format(type(cause_exc), cause_exc, cause_exc.__traceback__) if cause_str: format_str += '\n{}\n\n'.format(cause_str) format_str += 'Async Traceback (most recent call last):\n' # Need file, line, function, text for frame, line_no in traceback.walk_tb(tb): if _async_excepthook_exclude(frame): format_str += ' ---\n' else: code = frame.f_code filename = code.co_filename line = linecache.getline(filename, line_no).strip() format_str += ' File "{}", line {}, in {}\n' \ .format(filename, line_no, code.co_name) format_str += ' {}\n'.format(line) format_str += '{}: {}'.format(type_.__name__, exc) return format_str _ASYNC_EXCEPTHOOK_BLACKLIST = { 'asyncio.base_events': ('_run_once', 'call_later', 'call_soon'), 'asyncio.coroutines': ('__next__', 'send', 'throw'), 'asyncio.events': ('__init__', '_run'), 'asyncio.tasks': ('_step', '_wakeup'), 'traceback': ('extract', 'extract_stack'), } def _async_excepthook_exclude(frame): ''' Return True if ``frame`` should be excluded from tracebacks. ''' module = frame.f_globals['__name__'] function = frame.f_code.co_name return module in _ASYNC_EXCEPTHOOK_BLACKLIST and \ function in _ASYNC_EXCEPTHOOK_BLACKLIST[module] On Tue, Nov 14, 2017 at 7:15 PM, Nathaniel Smith <njs@pobox.com> wrote:
On Tue, Nov 14, 2017 at 2:00 PM, Roger Pate <rogerpate@gmail.com> wrote:
On Tue, Nov 14, 2017 at 9:54 AM, Mark E. Haase <mehaase@gmail.com> wrote: ...
print('Async Traceback (most recent call last):') for frame in traceback.extract_tb(tb): head, tail = os.path.split(frame.filename) if (head.endswith('asyncio') or tail == 'traceback.py') and
\
frame.name.startswith('_'):
...
The meat of it is towards the bottom, "if head.endswith('asyncio')..." There are a lot of debatable details and this implementation is pretty hacky and clumsy, but I have found it valuable in my own usage, and I haven't yet missed the omitted stack frames.
It would be better to determine if the qualified module name is "traceback" or starts with "asyncio." (or topmost package is "asyncio", etc.) rather than allow false positives for random_package.asyncio.module._any_function or random_package.traceback._any_function. I don't see an easy way to get the module object at this point in your hook; however:
You can't get the module from the cooked data that extract_tb returns, but it's there in the tb object itself. This walks the traceback and prints each frame's module:
current = tb while current is not None: print("Next module", current.tb_frame.f_globals.get("__name__")) current = current.tb_next
-n
-- Nathaniel J. Smith -- https://vorpus.org