[Python-Dev] Handle errors in cleanup code

Nathaniel Smith njs at pobox.com
Tue Jun 13 04:43:54 EDT 2017

On Tue, Jun 13, 2017 at 12:10 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:

> reporting failures from concurrent.futures.wait:
> https://pythonhosted.org/futures/#concurrent.futures.wait

Yeah, and asyncio.gather is another example in the stdlib. Or there's
twisted's DeferredList. Trio is unusual in effectively forcing all tasks to
be run under a gather(), but the basic concept isn't unique at all.

> Figuring out how to *display* an exception tree coherently is going to
> be a pain (it's already problematic with just the linked list), but if
> we can at least model exception trees consistently, then we'd be able
> to share that display logic, even if the scenarios resulting in
> MultiErrors varied.

It's true, it was a pain :-). And I'm sure it can be refined. But I think
this is a reasonable first cut:

Basically the algorithm there is:

if there's a __cause__ or __context__:
  print it (recursively using this algorithm)
  print the description line ("this exception was the direct cause" or
whatever is appropriate)
print the traceback and str() attached to this exception itself
for each embedded exception:
  print "Details of embedded exception {i}:"
  with extra indentation:
     print the embedded exception (recursively using this algorithm)
(+ some memoization to avoid infinite recursion on loopy structures)

Of course really complicated trainwrecks that send exception shrapnel
flying everywhere can still be complicated to read, but ... that's why we
make the big bucks, I guess. (My fingers initially typoed that as "that's
why we make the big bugs". Just throwing that out there.)

Currently that code has a hard-coded assumption that the only kind of
exception-container is MultiError, but I guess it wouldn't be too hard to
extend that into some kind of protocol that your asymmetric CleanupError
could also participate in? Like: an abstract exception container has 0 or 1
(predecessor exception, description) pairs [i.e., __cause__ or
__context__], plus 0 or more (embedded exception, description) pairs, and
then the printing code just walks the resulting tree using the above

If this all works out at the library level, then one can start to imagine
ways that the interpreter could potentially get benefits from participating:

- right now, if an exception that already has a __cause__ or __context__ is
re-raised, then implicit exception chaining will overwrite the old
__cause__ or __context__. Instead, it could check to see if there's already
a non-None value there, and if so do exc.__context__ =
MultiError([exc.__context__, new_exc]).

- standardizing the protocol for walking over an exception container would
let us avoid fighting over who owns sys.excepthook, and make it easier for
third-party exception printers (like IPython, pytest, etc.) to play along.

- MultiError.catch is a bit awkward. If we were going all-in on this then
one can imagine some construct like

    ... do stuff ...
    raise MultiError([ValueError(1), MultiError([TypeError(),
multiexcept ValueError as exc:
    print("caught a ValueError", exc)
multiexcept TypeError:
    print("caught a TypeError and raising RuntimeError")
    raise RuntimeError

which prints

  caught a ValueError 1
  caught a TypeError and raising RuntimeError
  caught a ValueError 2

and then raises a RuntimeError with its __context__ pointing to the

...but of course that's preeeeetty speculative at this point; definitely
more python-ideas territory.


Nathaniel J. Smith -- https://vorpus.org <http://vorpus.org>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20170613/ec4ee64b/attachment.html>

More information about the Python-Dev mailing list