This pep sounded kinda scary to me, so I wanted to try it out.
The reference implementation appears to have some bugs in it (around reraise star) , so it's not entirely clear what the expected behavior is supposed to be, but by checking out ffc493b5 I got some OK results.
I had a look at the tempfile example given under the Motivations section, and wanted to see how this would work (as it's the area of code I'm most familiar with).
Would the proposed tempfile change look something like this? (functionally, if not stylistically):
--- a/Lib/tempfile.py
+++ b/Lib/tempfile.py
@@ -819,8 +819,14 @@ def __repr__(self):
def __enter__(self):
return
self.name - def __exit__(self, exc, value, tb):
- self.cleanup()
+ def __exit__(self, exc_cls, exc_value, tb):
+ try:
+ self.cleanup()
+ except Exception as clean_exc:
+ if exc_value is not None:
+ raise ExceptionGroup('Exception occurred during cleanup', [exc_value, clean_exc])
+ else:
+ raise
def cleanup(self):
if self._finalizer.detach():
If so, then the following code fails to catch the ZeroDivisionError (there is an uncaught exception raised):
import tempfile, os, pathlib
def do_some_stuff():
with tempfile.TemporaryDirectory() as td:
os.rmdir(td)
pathlib.Path(td).write_text("Surprise!")
1/0
if __name__ == '__main__':
try:
do_some_stuff()
except Exception:
print("Something went wrong")
else:
print("No error")
The output I get:
"""
Traceback (most recent call last):
File "/home/sstagg/tmp/fuzztest/cpython/td.py", line 7, in do_some_stuff
1/0
ZeroDivisionError: division by zero
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/sstagg/tmp/fuzztest/cpython/Lib/tempfile.py", line 824, in __exit__
self.cleanup()
File "/home/sstagg/tmp/fuzztest/cpython/Lib/tempfile.py", line 833, in cleanup
self._rmtree(
self.name)
File "/home/sstagg/tmp/fuzztest/cpython/Lib/tempfile.py", line 809, in _rmtree
_shutil.rmtree(name, onerror=onerror)
File "/home/sstagg/tmp/fuzztest/cpython/Lib/shutil.py", line 718, in rmtree
_rmtree_safe_fd(fd, path, onerror)
File "/home/sstagg/tmp/fuzztest/cpython/Lib/shutil.py", line 631, in _rmtree_safe_fd
onerror(os.scandir, path, sys.exc_info())
File "/home/sstagg/tmp/fuzztest/cpython/Lib/shutil.py", line 627, in _rmtree_safe_fd
with os.scandir(topfd) as scandir_it:
NotADirectoryError: [Errno 20] Not a directory: '/tmp/tmpeedn6r0n'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/sstagg/tmp/fuzztest/cpython/td.py", line 12, in <module>
do_some_stuff()
File "/home/sstagg/tmp/fuzztest/cpython/td.py", line 7, in do_some_stuff
1/0
File "/home/sstagg/tmp/fuzztest/cpython/Lib/tempfile.py", line 827, in __exit__
raise ExceptionGroup('Exception occurred during cleanup', [exc_value, clean_exc])
ExceptionGroup: Exception occurred during cleanup
"""
===
So, to catch and handle errors raised from TemporaryDirectory safely, the try-except has to be wrapped in a try-*except block?:
if __name__ == '__main__':
try:
try:
do_some_stuff()
except Exception:
print("Fail Site 1")
except *NotADirectoryError:
print("Fail Site 2")
except *Exception:
print("Fail Site 3")
In this situation, Sites 2 and 3 are called, but if there is no problem during cleanup then Site 1 is called?
If, instead, the ExceptionGroup is explicitly handled:
if __name__ == '__main__':
try:
do_some_stuff()
except (Exception, ExceptionGroup):
print("Fail Site 1")
Then this actually works quite nicely for the 'except Exception' scenario, but is much more complex if you want to catch and handle specific types of exceptions:
if __name__ == '__main__':
try:
do_some_stuff()
except ExceptionGroup as exc_group:
zd_errors, others = exc_group(lambda e: isinstance(e, ZeroDivisionError))
if zd_errors:
print("Fail Site 1")
if others:
raise others
except ZeroDivisionError:
print("Fail Site 2")
If the idea is that tempfile.TemporaryDirectory *always* raises an ExceptionGroup (even if there was no cleanup exception) then any code that calls anything that might eventually call TemporaryDirectory will have to be aware that a different type of exception could appear that normal handling doesn't catch (or for /every/ usage of TemporaryDirectory be immediately wrapped in try-*except handling code).
It feels like ever letting an ExceptionGroup unwind more than 1 or two stack frames is super dangerous, as it directly requires all code up the stack to consider this situation and handle it separately from 'normal' exceptions.
It's quite possible that I've completely mis-understood, or skipped over something critical here. If so, apologies!
Steve