[Python-Dev] Handle errors in cleanup code

Nick Coghlan ncoghlan at gmail.com
Tue Jun 13 03:10:57 EDT 2017


On 13 June 2017 at 14:10, Nathaniel Smith <njs at pobox.com> wrote:
> On Mon, Jun 12, 2017 at 1:07 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>> Since I don't see anything in the discussion so far that *requires*
>> changes to the standard library (aside from "we may want to use this
>> ourselves"), the right place to thrash out the design details is so
>> probably contextlib2: https://github.com/jazzband/contextlib2
>>
>> That's where contextlib.ExitStack was born, and I prefer using it to
>> iterate on context management design concepts, since we can push
>> updates out faster, and if we make bad choices anywhere along the way,
>> they can just sit around in contextlib2, rather than polluting the
>> standard library indefinitely.
>
> I'd also be open to extracting MultiError into a standalone library
> that trio and contextlib2 both consume, if there was interest in going
> that way.

I think that would make sense, as it occurred to me while reading your
post that a construct like MultiError may also be useful when
reporting failures from concurrent.futures.wait:
https://pythonhosted.org/futures/#concurrent.futures.wait

At the moment, it isn't at all clear how best to throw an exception
that reports *all* of the raised exceptions in a concurrent map call,
rather than just the first failure.

So the recurring element we have that's common across all 3 scenarios
is the notion of "peer exceptions", where we're unwinding the stack
for multiple reasons, and we want to keep track of all of them.

The "failed resource cleanup" case then becomes an asymmetric variant
of that, where there's one primary exception (the one that triggered
the cleanup), and one or more secondary failures. Given MultiError,
that could be modelled as a two-tiered tree:

    class CleanupError(MultiError): pass

    CleanupError([
        original_exc,
        MultiError([*cleanup_errors]),
    ])

And if there was no original exception and the resource cleanup failed
unprovoked, you'd indicated that by having just a single child rather
than two:

    CleanupError([
        MultiError([*cleanup_errors]),
    ])

Figuring out how to *display* an exception tree coherently is going to
be a pain (it's already problematic with just the linked list), but if
we can at least model exception trees consistently, then we'd be able
to share that display logic, even if the scenarios resulting in
MultiErrors varied.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia


More information about the Python-Dev mailing list