[Python-Dev] Handle errors in cleanup code

Brett Cannon brett at python.org
Mon Jun 12 13:02:31 EDT 2017


On Mon, 12 Jun 2017 at 01:08 Nick Coghlan <ncoghlan at gmail.com> wrote:

> On 12 June 2017 at 17:31, Martin (gzlist) via Python-Dev
> <python-dev at python.org> wrote:
> > On 12/06/2017, Serhiy Storchaka <storchaka at gmail.com> wrote:
> >>
> >> But if an error is raised when execute undo_something(), it replaces the
> >> original exception which become chaining as the __context__ attribute.
> >> The problem is that this can change the type of the exception. If
> >> do_something_other() raises SystemExit and undo_something() raises
> >> KeyError, the final exception has type KeyError.
> >
> > For Python 2.7, I've used the following idiom, which always masks
> > errors from the undo:
> >
> >       do_something()
> >       try:
> >           do_something_other()
> >       except:
> >           try:
> >               undo_something()
> >           finally:
> >               raise
> >
> > Unfortunately, that breaks down on Python 3, as the new exception
> > propogates with the original as context..
>
> Relevant tracker issue for that problem:
> https://bugs.python.org/issue18861
>
> >> Does it mean that we should rewrite every chunk of code similar to the
> >> above? And if such cumbersome code is necessary and become common, maybe
> >> it needs syntax support in the language? Or changing the semantic of
> >> exceptions raised in error handlers and finally blocks?
> >
> > What I want is a way to chain exceptions in the other direction,
> > raising the original error, but attaching a related one. Unfortunately
> > neither __cause__ or __context__ really help.
>
> Aye, agreed. The key challenge we have is that we're trying to
> represent the exception state as a linked list, when what we really
> have once we start taking cleanup errors into account is an exception
> *tree*.
>
> The thing that finally clicked for me in this thread is that if we add
> contextlib.ResourceSet, and have it add a new attribute to the
> original exception (e.g. "__cleanup_errors__") with a list of
> exceptions raised while attempt to cleanup after the earlier
> exception, then that lets us actually start modelling that tree at
> runtime.
>

You might want to go back and read Jake's LWN article for this year's
language summit as Nathaniel Smith said he wanted some sort of
multi-exception type which would go along with this idea.

-Brett


>
> Once we understand the *behaviour* we want, *then* we can consider
> whether we might want to add a context manager to have any raised
> exceptions be attached to the currently active exception as cleanup
> errors rather than as new exceptions in their own right.
>
> For example:
>
>     do_something()
>     try:
>         do_something_other()
>     except BaseException as exc:
>         with contextlib.cleanup(exc) as reraise:
>             # Exceptions raised in here would be added to
>             # exc.__cleanup_errors__, rather than being
>             # propagated normally
>             undo_something()
>         reraise()
>
> The need for the "as reraise:"/"reraise()" trick arises from the need
> to special case the situation where the original exception inherits
> from Exception, but one of the raised exceptions *doesn't* - we want
> SystemExit/KeyboardInterrupt/etc to take precedence in that case, and
> a bare raise statement won't handle that for us (it *could* in a
> hypothetical future version of Python that's natively
> `__cleanup_errors__` aware, but that's not going to be useful for
> existing versions).
>
> Since I don't see anything in the discussion so far that *requires*
> changes to the standard library (aside from "we may want to use this
> ourselves"), the right place to thrash out the design details is
> probably contextlib2: https://github.com/jazzband/contextlib2
>
> That's where contextlib.ExitStack was born, and I prefer using it to
> iterate on context management design concepts, since we can push
> updates out faster, and if we make bad choices anywhere along the way,
> they can just sit around in contextlib2, rather than polluting the
> standard library indefinitely.
>
> Cheers,
> Nick.
>
> P.S. trio's MultiError is also likely worth looking into in this context
>
> --
> Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/brett%40python.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20170612/b9d0d9b4/attachment.html>


More information about the Python-Dev mailing list