On Fri, Oct 1, 2021 at 12:03 AM Soni L. <fakedme+py@gmail.com> wrote:
But what you're talking about doesn't have this clear distinction, other than in *your own definitions*. You have deemed that, in some areas, a certain exception should be turned into a RuntimeError; but in other areas, it shouldn't. To me, that sounds like a job for a context manager, not a function-level declaration.
But generators *are* iterators. By definition. In fact this had to be a breaking change *because there was code in the wild that relied on it*!
There was code in the wild that assumed it. Every change breaks someone's workflow.
Imagine if that code could be changed to be:
def gen() with StopIteration: try: yield next(foo) except StopIteration: raise
and have the StopIteration propagate as a StopIteration instead of RuntimeError! (altho supporting this *specific* use-case would probably be painful given that this is mostly a purely syntactic transformation.)
Not an improvement. Most definitely not an improvement. The whole point of generator functions is that they early-terminate by returning, same as every other function does. def gen(): try: yield next(foo) except StopIteration: pass Or "return" instead of "pass" if you have other code after that and you want to not continue. StopIteration is part of the API of next(), but it is not part of the API of gen(), which simply has "yield" and "return"; thus the "except StopIteration" is part of handling the call to next().
With a source transformation, really. that is:
def foo() with exceptions: something raise ...
always transforms into:
def foo(): set_to_True_to_pass_through_instead_of_wrapping_in_RuntimeError = False try: something set_to_True_to_pass_through_instead_of_wrapping_in_RuntimeError = True raise ... except exceptions as exc: if set_to_True_to_pass_through_instead_of_wrapping_in_RuntimeError: raise else: raise RuntimeError from exc
that is: the "with exceptions" becomes "except exceptions", and every "raise" gains an "set_to_True_to_pass_through_instead_of_wrapping_in_RuntimeError = True" immediately before it (mostly - kinda glossing over the whole "the expression of the raise doesn't get to, itself, raise its own exceptions", but anyway).
How would this be handled? def foo() with RuntimeError: try: foo() except RuntimeError as e: raise What about this? def foo() with KeyError: things = {} try: return things["foo"] except KeyError: return things["bar"] Or this? def foo() with KeyError: things = {} try: things["foo"] finally: things["bar"] I'm not nitpicking your code transformation here, btw; I'm asking what your intuition is about this kind of code. If "def ... with Exception:" is supposed to suppress exceptions, what happens if they get caught?
For example this: (real code)
def get_property_values(self, prop): try: factory = self.get_supported_properties()[prop] except KeyError as exc: raise PropertyError from exc iterator = factory(self._obj) try: first = next(iterator) except StopIteration: return (x for x in ()) except abdl.exceptions.ValidationError as exc: raise LookupError from exc except LookupError as exc: raise RuntimeError from exc # don't accidentally swallow bugs in the iterator return itertools.chain([first], iterator)
There's a lot of exception transformation happening here, and I don't understand, without context, the full purpose of it all. But the fact that you're raising LookupError directly seems a tad odd (it's a parent class for IndexError and KeyError), and it seems like a custom exception type would solve all of this. I *think* what's happening here is that you transform ValidationError into LookupError, but only if it happens on the very first yielded result?? Then how about: class PropertyNotFoundError(LookupError): pass def get_property_values(self, prop): try: factory = self.get_supported_properties()[prop] except KeyError as exc: raise PropertyError from exc iterator = factory(self._obj) try: yield next(iterator) except StopIteration: pass except abdl.exceptions.ValidationError as exc: raise PropertyNotFoundError from exc yield from iterator If the factory raises some other LookupError, then that's not a PropertyNotFoundError. And if it *does* raise PropertyNotFoundError, then that is clearly deliberate, and it's a signal that the property is, in fact, not found. Your code, as currently written (and including the rewrite), is *still* capable of leaking a faulty exception - just as long as the factory returns one good value first. So I'm not sure what you actually gain by this transformation.
In this case, not only does it clean stuff up, it also solves potential maintainability issues. Without this feature, this would need a bunch more blocks to get the correct exception hygiene.
I'm still not sure what you mean by "correct exception hygiene". What you seem to be saying is "transform these exceptions into these others, if they occur in these places", which as mentioned is a really good job for a context manager (or a plain try/except).
So what you're saying is that the raise statement will always raise an exception, but that any exception raised from any other function won't. Okay. So you basically want exceptions to... not be exceptions. You want to use exceptions as if they're return values.
Why not just use return values?
Because they're "unpythonth". Yes they're a thing in Rust and Rust is the inspiration for this idea (and a good part of the reason we're rewriting our code in Rust) but we do think it's possible to have a pythonic solution to a pythonic problem. We've seen how many exceptions are accidentally swallowed by python web frameworks and how much of a debugging nightmare it can make. That's why we write code that guards against unexpected exceptions. Like the (real code) we pasted here above. (It's saved us plenty of trouble *in practice*, so this is a real issue.)
I don't understand. You're using return value functionality and behaviour, but you feel obliged to use exceptions for some reason, and so you want exceptions to stop bubbling. But if you want the error condition to not bubble, don't use bubbling exception handling. There's nothing wrong with using return values in Python. Can't you use a sentinel object (the way NotImplemented can be returned from certain dunder functions) to do this job? It seems FAR cleaner than warping your API around the need to use exceptions. ChrisA