Thanks for the feedback! I put this aside for a while but I'm coming back to it now and cleaning it up.

The approach used in this first post was obviously very clumsy. In my latest version I am using module instance directly (as shown in Nathaniel's reply) and using the qualified package name (as suggested by Roger). I created an explicit blacklist (incomplete--still needs more testing) of functions to hide in my custom backtraces and refactored a bit so I can write tests for it. Code below.

One interesting thing I learned while working on this is that the backtraces change depending on the asyncio debug mode, because in debug mode couroutines are wrapped in CoroWrapper[1], which adds a frame everytime a coroutine sends, throws, etc. So I am now thinking that my custom excepthook is probably most useful in debug mode, but probably not good to enable in production.

I'm working on a more general asyncio task group library that will include this excepthook. I'll release the whole thing on PyPI when it's done.

[1] https://github.com/python/cpython/blob/23ab5ee667a9b29014f6f7f01797c611f63ff743/Lib/asyncio/coroutines.py#L25

---


def _async_excepthook(type_, exc, tb):
    '''
    An ``excepthook`` that hides event loop internals and displays
    task group information.

    :param type type_: the exception type
    :param Exception exc: the exception itself
    :param tb tb: a traceback of the exception
    '''
    print(_async_excepthook_format(type_, exc, tb))


def _async_excepthook_format(type_, exc, tb):
    '''
    This helper function is used for testing.

    :param type type_: the exception type
    :param Exception exc: the exception itself
    :param tracetack tb: a traceback of the exception
    :return: the formatted traceback as a string
    '''
    format_str = ''
    cause_exc = None
    cause_str = None

    if exc.__cause__ is not None:
        cause_exc = exc.__cause__
        cause_str = 'The above exception was the direct cause ' \
                    'of the following exception:'
    elif exc.__context__ is not None and not exc.__suppress_context__:
        cause_exc = exc.__context__
        cause_str = '\nDuring handling of the above exception, ' \
                    'another exception occurred:'

    if cause_exc:
        format_str += _async_excepthook_format(type(cause_exc), cause_exc,
            cause_exc.__traceback__)

    if cause_str:
        format_str += '\n{}\n\n'.format(cause_str)

    format_str += 'Async Traceback (most recent call last):\n'

    # Need file, line, function, text
    for frame, line_no in traceback.walk_tb(tb):
        if _async_excepthook_exclude(frame):
            format_str += '  ---\n'
        else:
            code = frame.f_code
            filename = code.co_filename
            line = linecache.getline(filename, line_no).strip()
            format_str += '  File "{}", line {}, in {}\n' \
                .format(filename, line_no, code.co_name)
            format_str += '    {}\n'.format(line)

    format_str += '{}: {}'.format(type_.__name__, exc)
    return format_str


_ASYNC_EXCEPTHOOK_BLACKLIST = {
    'asyncio.base_events': ('_run_once', 'call_later', 'call_soon'),
    'asyncio.coroutines': ('__next__', 'send', 'throw'),
    'asyncio.events': ('__init__', '_run'),
    'asyncio.tasks': ('_step', '_wakeup'),
    'traceback': ('extract', 'extract_stack'),
}

def _async_excepthook_exclude(frame):
    ''' Return True if ``frame`` should be excluded from tracebacks. '''
    module = frame.f_globals['__name__']
    function = frame.f_code.co_name
    return module in _ASYNC_EXCEPTHOOK_BLACKLIST and \
         function in _ASYNC_EXCEPTHOOK_BLACKLIST[module]


On Tue, Nov 14, 2017 at 7:15 PM, Nathaniel Smith <njs@pobox.com> wrote:
On Tue, Nov 14, 2017 at 2:00 PM, Roger Pate <rogerpate@gmail.com> wrote:
> On Tue, Nov 14, 2017 at 9:54 AM, Mark E. Haase <mehaase@gmail.com> wrote:
> ...
>>         print('Async Traceback (most recent call last):')
>>         for frame in traceback.extract_tb(tb):
>>             head, tail = os.path.split(frame.filename)
>>             if (head.endswith('asyncio') or tail == 'traceback.py') and \
>>                 frame.name.startswith('_'):
> ...
>> The meat of it is towards the bottom, "if head.endswith('asyncio')..."There
>> are a lot of debatable details and this implementation is pretty hacky and
>> clumsy, but I have found it valuable in my own usage, and I haven't yet
>> missed the omitted stack frames.
>
> It would be better to determine if the qualified module name is
> "traceback" or starts with "asyncio." (or topmost package is
> "asyncio", etc.) rather than allow false positives for
> random_package.asyncio.module._any_function or
> random_package.traceback._any_function.  I don't see an easy way to
> get the module object at this point in your hook; however:

You can't get the module from the cooked data that extract_tb returns,
but it's there in the tb object itself. This walks the traceback and
prints each frame's module:

current = tb
while current is not None:
    print("Next module", current.tb_frame.f_globals.get("__name__"))
    current = current.tb_next

-n

--
Nathaniel J. Smith -- https://vorpus.org