Hello, On Sat, 27 Mar 2021 14:29:21 +0000 Irit Katriel <iritkatriel@googlemail.com> wrote:
Hi Paul,
IIUC, you are saying that exception group should not be a builtin type because it is (1) complex (2) special-purposed. Instead, you propose that we make exception handling pluggable.
Yes, I wanted to mention that alternative possibility. I have to admit that I haven't read the entire discussion of the PEP, but the messages I read oftentimes were over-optimistic about the feature, rather than considering its issues/possible alternatives.
(1) I agree, it is somewhat complex - it took us several iterations to get from the idea of "a container of exceptions" to something that works correctly with split()/subgroup(), maintaining the integrity of __traceback__/__cause__/__context__ information, allowing subclassing and having a relatively simple API. To me this is a reason to provide it as a builtin rather than a reason not to (I think we should add this point to the PEP- thanks).
It definitely feels like a lot of effort went into devising and polishing ExceptionGroup's and except*, thanks. But I'm not sure if you gentlemen come up with the "ultimate" way to deal with multiple errors, which deserves being codified on the language level (vs be added as pluggable means indeed). That's why I asked if there's any prior art of a similar solution(s) in other languages and/or systems - that would definitely sooth concerns and add credibility. (And I'm not saying that Python can't lead there - just that more sustainable way would be to try and make it more generic/explicit rather than fix on a particular novel solution right away.)
You wrote:
So yes, maybe being well aware that you're handling exactly Trio's (or asyncio's, or context manager's) error isn't bad thing actually.
Maybe, in which case you can create a subclass of ExceptionGroup to mark the groups coming from your library. But also maybe not, in which case you don't have to.
(2) special-purposed? How so? When I asked for a practical advantage of your pluggable solution I meant an example of a use case that our proposal doesn't accommodate well and where a pluggable one does better. The only example you suggested so far was the TemporaryDirectory one, which I don't find compelling:
That's the impression I've got from reading the "Motivation" section. It cites Trio and multiple errors from async tasks as the guiding usecase. It also mentions a couple of cases in the stdlib, where my personal impression is that a simpler mechanism would suffice. Then it also cites some 3rd-party libs and their maintainers, who say such thing *might* be useful. But it's going to take practice to see if the actual thing added is useful. What if other projects turn out to want something slightly different (e.g. simpler?). Oh, they won't have much choice (to complain), as it's already shipped to them, not even in the stdlib, but in the language! Bottom line: this seems like a Trio's special-purpose feature, with good wishes of becoming the de facto standard. I find that somewhat ironical, as Trio's original motivation was to explore alternative ways of doing async framework in Python. And now it hardcodes its way of dealing with multiple errors in the language (no more exploration and alternatives).
def chained(e, excs): while e: if isinstance(e, excs): return e e = e.__cause__ # Simplified, should consider __context__ too
try: tempfile.TemporaryDirectory(...) except *chained(OSError) as e: print(e)
What if the user code raised an OSError too, so now that exception group has more than one?
I don't see how that's different from PEP654's ExceptionGroup's case. It's just in ExceptionGroup, there would be 2 (unrelated?) OSError's, while using normal chaining rules, there's at least context info which is preserved.
Even without that problem, why should I prefer this to our proposal?
My idea was to show an alternative not mentioned in PEP654's "Rejected Ideas" section.
I am interested to know (1) what limitations you see in the ExceptionGroup type we are proposing (in what sense is it special-purposed?
Just as a specific example, what if my application indeed needs ExceptionGroup's (and not just more flexible matching of existing exception chains), but I want it to be a bit simpler: a) to have a flat structure of contained exceptions; b) want to actually emphasize the users that they should not rely on the ordering, so want it to be a set. My understanding is that ExceptionGroup is carefully designed to allow subclassing it and override behavior, and the above should be possible by subclassing. But usually, a simpler base class is subclassed to have more advanced/complex behavior. So again, current ExceptionGroup feels kind of specialized with its careful attention to hierarchy and multi-story "3D" tracebacks. It will be good if you actually found the sweet spot. But what if not, and it will be a UX chore (Python's chore, nor a particular framework's chore)?
what purpose is it unsuitable for?) and (2) an example where the pluggable solution works better.
From my PoV, a solution which doesn't add particular complex behavior into the language core, but allows to plug it in, and keep whole thing more explicit, is something that "works better". Again, I wanted to mention that point, as between the initial PEP version and the latest one, I don't see that alternative to have been mentioned in the PEP (which might mean that it wasn't mentioned enough by other reviewers).
Irit
On Sat, Mar 27, 2021 at 12:42 PM Paul Sokolovsky <pmiscml@gmail.com> wrote:
Hello,
On Sat, 27 Mar 2021 10:55:40 +0000 Irit Katriel <iritkatriel@googlemail.com> wrote:
One of the motivations for introducing ExceptionGroup as a builtin is so that we won't have a different custom version in each library that needs it. So if you are writing a library the needs to raise multiple exceptions, and then you decide to call Trio, you don't need to translate Trio's MultiError into your own exception group type, because everybody uses the builtin.
Looking from a different angle shows a different perspective:
1. Trio devised an interesting concept of "nurseries" to deal with multiple tasks in asyncio-like programming.
2. It was all nice and beautiful until ... it came to error handling. Note that it's a rather typical situation - one can write nice, clean, beautiful code, which is not adequate in real-world scenarios, particularly because of error handling concerns. Oftentimes, such initial code/concepts are discarded, and more robust (though maybe not as beautiful) concepts/code is used.
3. But that's not what happened in the Trio case. The concept of nurseries got pushed forward, until it became clear that it requires changes on the programming language level.
4. That's how PEP654 was born, which, besides async scheduling of multiple tasks, bring *somewhat similar* usecases of e.g. raising exceptions thru context managers' own exceptions.
Note that where errors and exceptions lead us is in questions on how to handle them. And beyond a couple of well known patterns ("dump and crash" or "dump and continue with next iteration"), error handling is very adhoc to a particular application and particular error(s). Seen from that angle, Trio wants to vendor-lock the entire language into its particular (experimental) way of handling multiple errors.
So yes, maybe being well aware that you're handling exactly Trio's (or asyncio's, or context manager's) error isn't bad thing actually. And if it's clear that multiple asyncio frameworks are interested in sharing common exception base class for such usecases, then it could be introduced on the "asyncio" package level, or maybe even better, as a module similar to "concurrent.futures".
And your users don't need to learn how your particular exception group works because they know that you are using the builtin one.
I see the aesthetic value of your suggestion, but does it have practical advantages in light of the above?
The concern is that it codifies pretty complex and special-purpose things on the language level. And it seems that the whole concept is rather experimental and "original design". We can compare that with another sufficiently complex feature which landed recently: pattern matching. At all phases of the design and discussion of that feature, one of the guiding principles was: "Many other languages implement it, so we know it's generally useful, and have design space more or less charted, and can vary things to find local optimum for Python".
Contrary to that, the original PEP654 didn't refer to (cross-language, cross-framework) prior art in handling multiple errors, and I don't see that changed in the latest version. So I'm sorry, but it seems like NIH feature of a specific 3rd-party framework being promoted to a whole language's way of doing things.
Under such circumstance, I guess it would be good idea to try to decouple behavior of that feature from the languages core, and make aspects of behavior more explicit (following "explicit is better than implicit" principle), and allow to vary/evolve it without changing the core language.
I tried to draft a scheme aspiring to allow that. (Which would definitely need more work to achieve parity with functionality in PEP654, because again, it tries to codify rather complex and "magic" behavior. Where complex and magic behavior in exception handling is itself of concern, so making it more explicit may be a good idea. (A good "how other languages deal with it") review would mention that Go, Rust, Zig, etc. don't have and condemn exception handling at all (normal simple classy exceptions, not magic we discuss here!)).
Thanks, Paul
-- Best regards, Paul mailto:pmiscml@gmail.com