Avoiding nested for try..finally: atexit for functions?
Hello, I often have code of the form: def my_fun(): allocate_res1() try: # do stuff allocate_res2() try: # do stuff allocate_res3() try: # do stuff finally: cleanup_res3() finally: cleanup_res2() finally: cleanup_res1() return With increasing number of managed resources, the indentation becomes really annoying, there is lots of line noise, and I don't like the fact that the cleanup is so far away from the allocation. I would much rather have something like this: def my_fun(): allocate_res1() atreturn.register(cleanup_res1) # do stuff allocate_res2() atreturn.register(cleanup_res2) # do stuff allocate_res3() atreturn.register(cleanup_res3) # do stuff return Has the idea of implementing such "on return" handlers ever come up? Maybe there is some tricky way to do this with function decorators? Best, -Nikolaus -- »Time flies like an arrow, fruit flies like a Banana.« PGP fingerprint: 5B93 61F8 4EA2 E279 ABF6 02CF A9AD B7F8 AE4E 425C
On Tue, Oct 18, 2011 at 7:14 PM, Nikolaus Rath <Nikolaus@rath.org> wrote:
Hello,
I often have code of the form:
def my_fun(): allocate_res1() try: # do stuff allocate_res2() try: # do stuff allocate_res3() try: # do stuff finally: cleanup_res3() finally: cleanup_res2() finally: cleanup_res1()
return
With increasing number of managed resources, the indentation becomes really annoying, there is lots of line noise, and I don't like the fact that the cleanup is so far away from the allocation.
Use the `with` statement and context managers. They were added for this exact situation. See http://www.python.org/dev/peps/pep-0343/ Resulting code will resemble: def func(): with alloc() as res1, alloc() as res2, alloc() as res3: # do stuff Cheers, Chris
On Wed, Oct 19, 2011 at 1:14 PM, Chris Rebert <pyideas@rebertia.com> wrote:
On Tue, Oct 18, 2011 at 7:14 PM, Nikolaus Rath <Nikolaus@rath.org> wrote:
Hello,
I often have code of the form:
def my_fun(): allocate_res1() try: # do stuff allocate_res2() try: # do stuff allocate_res3() try: # do stuff finally: cleanup_res3() finally: cleanup_res2() finally: cleanup_res1()
return
With increasing number of managed resources, the indentation becomes really annoying, there is lots of line noise, and I don't like the fact that the cleanup is so far away from the allocation.
Use the `with` statement and context managers. They were added for this exact situation. See http://www.python.org/dev/peps/pep-0343/
Resulting code will resemble:
def func(): with alloc() as res1, alloc() as res2, alloc() as res3: # do stuff
Or, to more closely mirror the original example: # Define these wherever the current resources are defined @contextlib.contextmanager def cm1(): res1 = allocate_res1() try: yield res1 finally: cleanup_res1() @contextlib.contextmanager def cm2(): res2 = allocate_res2() try: yield res2 finally: cleanup_res2() @contextlib.contextmanager def cm3(): res3 = allocate_res3() try: yield res2 finally: cleanup_res3() def func(): with cm1() as res1: # do stuff with cm2() as res2: # do stuff with cm3() as res3: # do stuff Any time a with statement's body consists solely of another with statement you can collapse them into one line as Chris did in his example. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
Nick Coghlan <ncoghlan-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> writes:
On Wed, Oct 19, 2011 at 1:14 PM, Chris Rebert <pyideas-QkDgq5C4a+JWk0Htik3J/w@public.gmane.org> wrote:
On Tue, Oct 18, 2011 at 7:14 PM, Nikolaus Rath <Nikolaus-BTH8mxji4b0@public.gmane.org> wrote:
Hello,
I often have code of the form:
def my_fun(): allocate_res1() try: # do stuff allocate_res2() try: # do stuff allocate_res3() try: # do stuff finally: cleanup_res3() finally: cleanup_res2() finally: cleanup_res1()
return
With increasing number of managed resources, the indentation becomes really annoying, there is lots of line noise, and I don't like the fact that the cleanup is so far away from the allocation.
Use the `with` statement and context managers. They were added for this exact situation. See http://www.python.org/dev/peps/pep-0343/
Resulting code will resemble:
def func(): with alloc() as res1, alloc() as res2, alloc() as res3: # do stuff
Or, to more closely mirror the original example:
# Define these wherever the current resources are defined @contextlib.contextmanager def cm1(): res1 = allocate_res1() try: yield res1 finally: cleanup_res1()
@contextlib.contextmanager def cm2(): res2 = allocate_res2() try: yield res2 finally: cleanup_res2()
@contextlib.contextmanager def cm3(): res3 = allocate_res3() try: yield res2 finally: cleanup_res3()
def func(): with cm1() as res1: # do stuff with cm2() as res2: # do stuff with cm3() as res3: # do stuff
Indeed, that works. But I do you really consider this code nicer than the original one? I think a simple line count answers the question :-). Best, -Nikolaus -- »Time flies like an arrow, fruit flies like a Banana.« PGP fingerprint: 5B93 61F8 4EA2 E279 ABF6 02CF A9AD B7F8 AE4E 425C
Chris Rebert <pyideas-QkDgq5C4a+JWk0Htik3J/w@public.gmane.org> writes:
On Tue, Oct 18, 2011 at 7:14 PM, Nikolaus Rath <Nikolaus@rath.org> wrote:
Hello,
I often have code of the form:
def my_fun(): allocate_res1() try: # do stuff allocate_res2() try: # do stuff allocate_res3() try: # do stuff finally: cleanup_res3() finally: cleanup_res2() finally: cleanup_res1()
return
With increasing number of managed resources, the indentation becomes really annoying, there is lots of line noise, and I don't like the fact that the cleanup is so far away from the allocation.
Use the `with` statement and context managers. They were added for this exact situation. See http://www.python.org/dev/peps/pep-0343/
Resulting code will resemble:
def func(): with alloc() as res1, alloc() as res2, alloc() as res3: # do stuff
I think they're not for exactly this situation for two reasons: 1. This requires the alloc() functions to be context managers. If they're not, then I need to code a wrapping context manager as well. It's probably possible to write a generic wrapper that works for any cleanup function, but the result is not going to look very nice. 2. If I don't want to allocate the resources all at the same time, the indentation mess is still the same: def func(): with alloc() as res1: # do stuff with alloc() as res2: # do stuff with alloc() as res3: # do stuff Best, -Nikolaus -- »Time flies like an arrow, fruit flies like a Banana.« PGP fingerprint: 5B93 61F8 4EA2 E279 ABF6 02CF A9AD B7F8 AE4E 425C
On Tue, Oct 18, 2011 at 10:14:56PM -0400, Nikolaus Rath wrote: [...]
I would much rather have something like this:
def my_fun(): allocate_res1() atreturn.register(cleanup_res1) # do stuff allocate_res2() atreturn.register(cleanup_res2) # do stuff allocate_res3() atreturn.register(cleanup_res3) # do stuff return
Has the idea of implementing such "on return" handlers ever come up? Maybe there is some tricky way to do this with function decorators?
The "with" statement is a good answer. If for some reason you need to be compatible with version of Python so old it doesn't have it, then try the bzrlib.cleanup module in bzr. It implements the sort of API you describe above. And there are times when an API like that might be nicer to use anyway, e.g. when you have conditional allocations like: def foo(): res1 = allocate_res1() add_cleanup(res1.release) ... if cond: res2 = allocate_res2() add_cleanup(res2.release) res3 = allocate_res3() add_cleanup(res3.release) ... do_stuff() And avoiding the cascading indents of multiple with statements can be nice too. -Andrew.
Andrew Bennetts <andrew-CCXH/aKPDeEYtQj7fl1lsA@public.gmane.org> writes:
On Tue, Oct 18, 2011 at 10:14:56PM -0400, Nikolaus Rath wrote: [...]
I would much rather have something like this:
def my_fun(): allocate_res1() atreturn.register(cleanup_res1) # do stuff allocate_res2() atreturn.register(cleanup_res2) # do stuff allocate_res3() atreturn.register(cleanup_res3) # do stuff return
Has the idea of implementing such "on return" handlers ever come up? Maybe there is some tricky way to do this with function decorators?
The "with" statement is a good answer. If for some reason you need to be compatible with version of Python so old it doesn't have it, then try the bzrlib.cleanup module in bzr. It implements the sort of API you describe above.
Yes, that's sort of what I was thinking about. I think the API is still more convoluted than necessary, but that's probably because it works with Python 2.4. Having thought about this a bit more, I think it should be possible to use 'with' rather than decorators to implement something like this: with CleanupManager() as mngr: allocate_res1() mngr.register(cleanup_res1) # do stuff allocate_res2() mngr.register(cleanup_res2) # do stuff allocate_res3() mngr.register(cleanup_res3) # do stuff The mngr object would just run all the registered functions when the block is exited. Thoughts? Best, -Nikolaus -- »Time flies like an arrow, fruit flies like a Banana.« PGP fingerprint: 5B93 61F8 4EA2 E279 ABF6 02CF A9AD B7F8 AE4E 425C
On 19 October 2011 14:54, Nikolaus Rath <Nikolaus@rath.org> wrote:
The "with" statement is a good answer. If for some reason you need to be compatible with version of Python so old it doesn't have it, then try the bzrlib.cleanup module in bzr. It implements the sort of API you describe above.
Yes, that's sort of what I was thinking about. I think the API is still more convoluted than necessary, but that's probably because it works with Python 2.4.
Having thought about this a bit more, I think it should be possible to use 'with' rather than decorators to implement something like this:
with CleanupManager() as mngr: allocate_res1() mngr.register(cleanup_res1) # do stuff allocate_res2() mngr.register(cleanup_res2) # do stuff allocate_res3() mngr.register(cleanup_res3) # do stuff
The mngr object would just run all the registered functions when the block is exited.
That's probably better than my decorator suggestion, because it allows you to limit the scope precisely, rather than just being function-scope. The CleanupManager class might make a good addition to contextlib, in actual fact... Paul.
Paul Moore <p.f.moore-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> writes:
On 19 October 2011 14:54, Nikolaus Rath <Nikolaus-BTH8mxji4b0@public.gmane.org> wrote:
The "with" statement is a good answer. If for some reason you need to be compatible with version of Python so old it doesn't have it, then try the bzrlib.cleanup module in bzr. It implements the sort of API you describe above.
Yes, that's sort of what I was thinking about. I think the API is still more convoluted than necessary, but that's probably because it works with Python 2.4.
Having thought about this a bit more, I think it should be possible to use 'with' rather than decorators to implement something like this:
with CleanupManager() as mngr: allocate_res1() mngr.register(cleanup_res1) # do stuff allocate_res2() mngr.register(cleanup_res2) # do stuff allocate_res3() mngr.register(cleanup_res3) # do stuff
The mngr object would just run all the registered functions when the block is exited.
That's probably better than my decorator suggestion, because it allows you to limit the scope precisely, rather than just being function-scope. The CleanupManager class might make a good addition to contextlib, in actual fact...
What would be the best way to handle errors during cleanup? Personally I would log them with logging.exception and discard them, but using the logging module is probably not a good option for contextlib, or is it? Best, -Nikolaus -- »Time flies like an arrow, fruit flies like a Banana.« PGP fingerprint: 5B93 61F8 4EA2 E279 ABF6 02CF A9AD B7F8 AE4E 425C
Nikolaus Rath dixit (2011-10-19, 10:43):
with CleanupManager() as mngr: allocate_res1() mngr.register(cleanup_res1) # do stuff allocate_res2() mngr.register(cleanup_res2) # do stuff allocate_res3() mngr.register(cleanup_res3) # do stuff
The mngr object would just run all the registered functions when the block is exited. [snip] What would be the best way to handle errors during cleanup? Personally I would log them with logging.exception and discard them, but using the logging module is probably not a good option for contextlib, or is it?
I'd suggest something like the following: class CleanupManager: _default_error_handler = lambda exc_type, exc_value, tb: False def __init__(self, error_handler=_default_error_handler): self.error_handler = error_handler self.cleanup_callbacks = [] def register(self, callback): self.cleanup_callbacks.append(callback) def __enter__(self): return self def __exit__(self, exc_type, exc_value, tb): try: if exc_value is not None: # if returns True, exception will be suppressed... return self.error_handler(exc_type, exc_value, tb) finally: # ...except something wrong happen when using callbacks self._next_callback() def _next_callback(self): if self.cleanup_callbacks: callback = self.cleanup_callbacks.pop() try: callback() finally: # all callbacks to be used + all errors to be reported self._next_callback() Then a user can specify any error callback they need, e.g.: >>> def error_printer(exc_type, exc_value, tb): ... print(exc_type.__name__, exc_value) ... return True ... >>> with CleanupManager(error_printer) as cm: ... cm.register(lambda: print(1)) ... cm.register(lambda: print(2)) ... raise ValueError('spam') ... ValueError spam 2 1 Please also note that all cleanup callbacks will be used and, at the same time, no exception will remain unnoticed -- that within the with-block (handled with the error handler), but also that from the error handler as well as those from all cleanup handlers (as long as we talk about Py3.x, with its cool exception chaining feature): >>> erroneous_error_handler = (lambda exc_tp, exc, tb: ''/3) >>> with CleanupManager(erroneous_error_handler) as cm: ... cm.register(lambda: 1/0) # error (division by 0) ... cm.register(lambda: 44+'') # error (bad operand type) ... raise ValueError('spam') # error ... Traceback (most recent call last): File "<stdin>", line 4, in <module> ValueError: spam During handling of the above exception, another exception occurred: Traceback (most recent call last): File "cleanup_manager.py", line 19, in __exit__ return self.error_handler(exc_type, exc_value, tb) File "<stdin>", line 1, in <lambda> TypeError: unsupported operand type(s) for /: 'str' and 'int' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "cleanup_manager.py", line 28, in _next_callback callback() File "<stdin>", line 3, in <lambda> TypeError: unsupported operand type(s) for +: 'int' and 'str' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "<stdin>", line 4, in <module> File "cleanup_manager.py", line 22, in __exit__ self._next_callback() File "cleanup_manager.py", line 31, in _next_callback self._next_callback() File "cleanup_manager.py", line 28, in _next_callback callback() File "<stdin>", line 2, in <lambda> ZeroDivisionError: division by zero Cheers. *j
Jan Kaliszewski <zuo-WMSfXiZwWcGxgjU+5Knr6g@public.gmane.org> writes:
Nikolaus Rath dixit (2011-10-19, 10:43):
with CleanupManager() as mngr: allocate_res1() mngr.register(cleanup_res1) # do stuff allocate_res2() mngr.register(cleanup_res2) # do stuff allocate_res3() mngr.register(cleanup_res3) # do stuff
The mngr object would just run all the registered functions when the block is exited. [snip] What would be the best way to handle errors during cleanup? Personally I would log them with logging.exception and discard them, but using the logging module is probably not a good option for contextlib, or is it?
I'd suggest something like the following:
class CleanupManager:
_default_error_handler = lambda exc_type, exc_value, tb: False
def __init__(self, error_handler=_default_error_handler): self.error_handler = error_handler self.cleanup_callbacks = []
def register(self, callback): self.cleanup_callbacks.append(callback)
def __enter__(self): return self
def __exit__(self, exc_type, exc_value, tb): try: if exc_value is not None: # if returns True, exception will be suppressed... return self.error_handler(exc_type, exc_value, tb) finally: # ...except something wrong happen when using callbacks self._next_callback()
def _next_callback(self): if self.cleanup_callbacks: callback = self.cleanup_callbacks.pop() try: callback() finally: # all callbacks to be used + all errors to be reported self._next_callback()
[...]
Please also note that all cleanup callbacks will be used and, at the same time, no exception will remain unnoticed -- that within the with-block (handled with the error handler), but also that from the error handler as well as those from all cleanup handlers (as long as we talk about Py3.x, with its cool exception chaining feature):
Wow, that's really neat! I was in Python 2.x mode and would have tried to iterate over the callbacks, not knowing what to do with any exceptions from them. That said, do you have a suggestion for Python 2.7 as well? Maybe something like a cleanup error handler? class CleanupManager: _default_error_handler = lambda exc_type, exc_value, tb: False _default_cleanup_error_handler = lambda exc_type, exc_value, tb: True def __init__(self, error_handler=_default_error_handler, cleanup_error_handler=_default_cleanup_error_handler): self.error_handler = error_handler self.cleanup_error_handler = cleanup_error_handler self.cleanup_callbacks = [] def register(self, callback): self.cleanup_callbacks.append(callback) def __enter__(self): return self def __exit__(self, exc_type, exc_value, tb): try: if exc_value is not None: # if returns True, exception will be suppressed... return self.error_handler(exc_type, exc_value, tb) finally: # Saves the exception that we are going to raise at the # end (if any) exc_info = None for cb in self.cleanup_callbacks: try: cb() except: # If returns true, ignore exceptions during cleanup if self.cleanup_error_handler(*sys.exc_info()): pass else: # Only first exception gets propagated if not exc_info: exc_info = sys.exc_info() if exc_info: raise exc_info[0], exc_info[1], exc_info[2] Best, -Nikolaus -- »Time flies like an arrow, fruit flies like a Banana.« PGP fingerprint: 5B93 61F8 4EA2 E279 ABF6 02CF A9AD B7F8 AE4E 425C
Nikolaus Rath dixit (2011-10-19, 22:19):
That said, do you have a suggestion for Python 2.7 as well? Maybe something like a cleanup error handler?
As Sven noted, the error handler proposed by me is redundant. Such ones would be redundant too, IMHO. The appropriate place to catch cleanup errors would be a cleanup callback itself. And -- according to another Sven's idea -- callbacks could be registered together with arguments, e.g.: def closing_callback(stream): try: stream.close() except Exception: log.critical('Hm, something wrong...', exc_info=True) with CleanupMagager() as cm: xfile = open(x) cm.register(closing_callback, xfile) ... An improved (and at the same time simplified) implementation (being also a recipe for Python 2.x, though this list is about ideas for Py3.x): class CleanupManager(object): def __init__(self, initial_callbacks=()): self.cleanup_callbacks = list(initial_callbacks) def register(self, callback, *args, **kwargs): self.cleanup_callbacks.append((callback, args, kwargs)) def __enter__(self): return self def __exit__(self, exc_type, exc, tb): self._next_callback() def _next_callback(self): if self.cleanup_callbacks: callback, args, kwargs = self.cleanup_callbacks.pop() try: callback(*args, **kwargs) finally: # all cleanup callbacks to be used # Py3.x: all errors to be reported self._next_callback() I hope it implements well what you explained... I'm not sure if it is worth to be added to the standard library (in the case of your primary example I'd rather prefer that try-finally nested structure) -- though in some cases it may become really useful: with CleanupMagager() as cm: ... cm.register(foo) ... if cond: cm.register(bar) else: cm.register(spam) for x in y: cm.register(baz, x) ... Cheers. *j
On 10/21/2011 08:22 PM, Jan Kaliszewski wrote:
An improved (and at the same time simplified) implementation (being also a recipe for Python 2.x, though this list is about ideas for Py3.x):
class CleanupManager(object):
def __init__(self, initial_callbacks=()): self.cleanup_callbacks = list(initial_callbacks)
def register(self, callback, *args, **kwargs): self.cleanup_callbacks.append((callback, args, kwargs))
def __enter__(self): return self
def __exit__(self, exc_type, exc, tb): self._next_callback()
def _next_callback(self): if self.cleanup_callbacks: callback, args, kwargs = self.cleanup_callbacks.pop() try: callback(*args, **kwargs) finally: # all cleanup callbacks to be used # Py3.x: all errors to be reported self._next_callback()
I hope it implements well what you explained... I'm not sure if it is worth to be added to the standard library (in the case of your primary example I'd rather prefer that try-finally nested structure) -- though in some cases it may become really useful:
It implements almost exactly what I need. I will use it in a slightly modified form so that exceptions in the cleanup handlers are logged and discarded, so that they original exception is preserved (can't switch to Python 3 before pycryptopp becomes Py3 compatible). Who decides if it's going into stdlib? I'm of course in favor, but I feel that my opinion may not count that much and, in addition to that, be highly biased :-). Thanks, -Nikolaus -- »Time flies like an arrow, fruit flies like a Banana.« PGP fingerprint: 5B93 61F8 4EA2 E279 ABF6 02CF A9AD B7F8 AE4E 425C
On 10/21/2011 08:22 PM, Jan Kaliszewski wrote:
An improved (and at the same time simplified) implementation (being also a recipe for Python 2.x, though this list is about ideas for Py3.x):
class CleanupManager(object):
def __init__(self, initial_callbacks=()): self.cleanup_callbacks = list(initial_callbacks)
def register(self, callback, *args, **kwargs): self.cleanup_callbacks.append((callback, args, kwargs))
def __enter__(self): return self
def __exit__(self, exc_type, exc, tb): self._next_callback()
def _next_callback(self): if self.cleanup_callbacks: callback, args, kwargs = self.cleanup_callbacks.pop() try: callback(*args, **kwargs) finally: # all cleanup callbacks to be used # Py3.x: all errors to be reported self._next_callback()
I hope it implements well what you explained... I'm not sure if it is worth to be added to the standard library (in the case of your primary example I'd rather prefer that try-finally nested structure) -- though in some cases it may become really useful:
with CleanupMagager() as cm: ... cm.register(foo) ... if cond: cm.register(bar) else: cm.register(spam) for x in y: cm.register(baz, x)
I've opened a wishlist bug for this at http://bugs.python.org/issue13585. Best, -Nikolaus -- »Time flies like an arrow, fruit flies like a Banana.« PGP fingerprint: 5B93 61F8 4EA2 E279 ABF6 02CF A9AD B7F8 AE4E 425C
Jan Kaliszewski schrieb am Do, 20. Okt 2011, um 02:19:22 +0200:
class CleanupManager:
_default_error_handler = lambda exc_type, exc_value, tb: False
def __init__(self, error_handler=_default_error_handler): self.error_handler = error_handler self.cleanup_callbacks = []
def register(self, callback): self.cleanup_callbacks.append(callback)
def __enter__(self): return self
def __exit__(self, exc_type, exc_value, tb): try: if exc_value is not None: # if returns True, exception will be suppressed... return self.error_handler(exc_type, exc_value, tb) finally: # ...except something wrong happen when using callbacks self._next_callback()
def _next_callback(self): if self.cleanup_callbacks: callback = self.cleanup_callbacks.pop() try: callback() finally: # all callbacks to be used + all errors to be reported self._next_callback()
Why introduce an error handler at all? If you want to handle exceptions inside the with statement, an explicit try-except block would be both more convenient (you don't need an extra function definition) and more flexible (you can use - say - 'except ValueError:' to just catch some exceptions). To give an example, I'd really prefer with CleanupManager() as cleanup: try: # do stuff except ValueError: # handle exception over def handle_value_error(exc_type, exc_value, exc_tb): if issubclass(exc_type, ValueError): # handle exception return True return False with CleanupManager(handle_value_error) as cleanup: # do stuff Instead of an error handler, I'd rather accept the first (or more) callbacks as constructor parameters -- there wouldn't be any point in starting the with block if you weren't about to add a callback in the very next statement, so we could as well accept it as a parameter to the constructor.
Please also note that all cleanup callbacks will be used and, at the same time, no exception will remain unnoticed -- that within the with-block (handled with the error handler), but also that from the error handler as well as those from all cleanup handlers (as long as we talk about Py3.x, with its cool exception chaining feature):
>>> erroneous_error_handler = (lambda exc_tp, exc, tb: ''/3) >>> with CleanupManager(erroneous_error_handler) as cm: ... cm.register(lambda: 1/0) # error (division by 0) ... cm.register(lambda: 44+'') # error (bad operand type) ... raise ValueError('spam') # error ... Traceback (most recent call last): File "<stdin>", line 4, in <module> ValueError: spam
During handling of the above exception, another exception occurred:
Traceback (most recent call last): File "cleanup_manager.py", line 19, in __exit__ return self.error_handler(exc_type, exc_value, tb) File "<stdin>", line 1, in <lambda> TypeError: unsupported operand type(s) for /: 'str' and 'int'
During handling of the above exception, another exception occurred:
Traceback (most recent call last): File "cleanup_manager.py", line 28, in _next_callback callback() File "<stdin>", line 3, in <lambda> TypeError: unsupported operand type(s) for +: 'int' and 'str'
During handling of the above exception, another exception occurred:
Traceback (most recent call last): File "<stdin>", line 4, in <module> File "cleanup_manager.py", line 22, in __exit__ self._next_callback() File "cleanup_manager.py", line 31, in _next_callback self._next_callback() File "cleanup_manager.py", line 28, in _next_callback callback() File "<stdin>", line 2, in <lambda> ZeroDivisionError: division by zero
Note that the only exception you could easily catch in this example is the ZeroDivisionError. try: erroneous_error_handler = (lambda exc_tp, exc, tb: ''/3) with CleanupManager(erroneous_error_handler) as cm: cm.register(lambda: 1/0) # error (division by 0) cm.register(lambda: 44+'') # error (bad operand type) raise ValueError('spam') # error except ValueError as e: print(e) won't catch the ValueError. Cheers, Sven
Sven Marnach dixit (2011-10-20, 16:11):
Why introduce an error handler at all? If you want to handle exceptions inside the with statement, an explicit try-except block would be both more convenient (you don't need an extra function definition) and more flexible (you can use - say - 'except ValueError:' to just catch some exceptions). To give an example, I'd really prefer
with CleanupManager() as cleanup: try: # do stuff except ValueError: # handle exception
You are right, that error handler is redundant. Except that, my code mimics the nested structure of try-finally clauses from the primary OP's example. Cheers. *j
Jan Kaliszewski schrieb am Do, 20. Okt 2011, um 02:19:22 +0200:
def register(self, callback): self.cleanup_callbacks.append(callback)
We should probably accept additional *args and **kwargs which will be passed on to the callback, similar to atexit.register(). And possibly add an unregister() method in analogy to atexit.unregister() as well? Cheers, Sven
On 19 October 2011 03:14, Nikolaus Rath <Nikolaus@rath.org> wrote:
Hello,
I often have code of the form:
def my_fun(): allocate_res1() try: # do stuff allocate_res2() try: # do stuff allocate_res3() try: # do stuff finally: cleanup_res3() finally: cleanup_res2() finally: cleanup_res1()
return
With increasing number of managed resources, the indentation becomes really annoying, there is lots of line noise, and I don't like the fact that the cleanup is so far away from the allocation.
I would much rather have something like this:
def my_fun(): allocate_res1() atreturn.register(cleanup_res1) # do stuff allocate_res2() atreturn.register(cleanup_res2) # do stuff allocate_res3() atreturn.register(cleanup_res3) # do stuff return
Has the idea of implementing such "on return" handlers ever come up? Maybe there is some tricky way to do this with function decorators?
Here's a "tricky way with decorators" :-) :
def withexit(f): ... ex = [] ... def atex(g): ex.append(g) ... def wrapper(): ... f() ... for g in ex: g() ... wrapper.atex = atex ... return wrapper ... def p1(): print "one" ... def p2(): print "two" ... @withexit ... def ff(): ... print 1 ... ff.atex(p1) ... print 2 ... ff.atex(p2) ... print 3 ... ff() 1 2 3 one two
Paul.
On 2011-10-19 04:14, Nikolaus Rath wrote:
I would much rather have something like this:
def my_fun(): allocate_res1() atreturn.register(cleanup_res1) # do stuff allocate_res2() atreturn.register(cleanup_res2) # do stuff allocate_res3() atreturn.register(cleanup_res3) # do stuff return
Has the idea of implementing such "on return" handlers ever come up? Maybe there is some tricky way to do this with function decorators?
How about a not-so-tricky solution using context managers? Something like (untested): import contextlib @contextlib.contextmanager def atwithexit(): handlers = [] try: yield handlers.append finally: for h in reversed(handlers): h() def my_fun(): with atwithexit() as atreturn: allocate_res1() atreturn(cleanup_res1) # do stuff allocate_res2() atreturn(cleanup_res2) # do stuff allocate_res3() atreturn(cleanup_res3) # do stuff return HTH - Jacob
participants (8)
-
Andrew Bennetts -
Chris Rebert -
Jacob Holm -
Jan Kaliszewski -
Nick Coghlan -
Nikolaus Rath -
Paul Moore -
Sven Marnach