On 28Feb2021 23:56, Irit Katriel <iritkatriel@googlemail.com> wrote:
If you go long, I go longer :)
:-)
On Sun, Feb 28, 2021 at 10:51 PM Cameron Simpson <cs@cskk.id.au> wrote:
On 28Feb2021 10:40, Irit Katriel <iritkatriel@googlemail.com> wrote:
split() and subgroup() take care to preserve the correct metadata on all the internal nodes, and if you just use them you only make safe operations. This is why I am hesitating to add iteration utilities to the API. Like we did, people will naturally try that first, and it's not the safest API.
Wouldn't it be safe if the ExceptionGroup were immutable, as you plan? Or have you another hazard in mind?
Making them immutable won't help the metadata issue. split() and subgroup() copy the (context, cause traceback) from the original ExceptionGroups (root and internal nodes of the tree) to the result trees. If you DIY creating new ExceptionGroups you need to take care of that.
Ah, right. Yes. The overflows into my request for a factory method to construct a "like" ExceptionGroup with a new exception tree lower down.
But all that said, being able to iterable the subexceptions seems a natural way to manage that:
unhandled = [] try: ......... except *OSError as eg: for e in eg: if an ok exception: handle it else: unhandled.append(e) if unhandled: raise ExceptionGroup("unhandled", unhandled)
You just lost the metadata of eg. It has no context, cause and its traceback begins here.
Aye. Hence a wish, again lower down, for some reference to the source ExceptionGroup and therefore a handy factory for making a new group with the right metadata.
And the exceptions contained in it, if they came from a deeper tree that you flattened into the list, now look like their traceback jumps straight to here from the place they were actually first inserted into an ExceptionGroup. This may well be an impossible code path.
Perhaps so. But it doesn't detract from how useful it is to iterate over the inner exceptions. I see this as an argument for making it possible to obtain the correct metadata, not against iteration itself. Even if the iteration yielded some proxy or wrapper for the inner exception instead of the naked exception itself.
Here's an example: [... flattened ExceptionGroup with uninformative tracebacks ...]
import traceback def flatten(exc): ... if isinstance(exc, ExceptionGroup): ... for e in exc.errors: ... yield from flatten(e) ... else: ... yield exc [...] traceback.print_exception(flat_h()) Traceback (most recent call last): File "<stdin>", line 3, in flat_h ExceptionGroup: flat_h
Traceback (most recent call last): File "<stdin>", line 3, in f ValueError: 42
traceback.print_exception(h()) prints a reasonable traceback - h() called g() called f().
But according to traceback.print_exception(flat_h()), flat_h() called f().
You can preserve the metadata (and the nested structure with all its metadata) if you replace the last line with: raise eg.subgroup(lambda e: e in unhandled)
Ok. That works for me as my desired factory. Verbose maybe, but workable. Um. It presumes exception equality will do - that feels slightly error prone (including some similar but not unhandled exceptions). But I can write something more pointed based on id().
And for the part before that, iteration, Guido's pattern showed that you can roll it into the subgroup callback.
Aye.
There are some immediate shortcomings above. In particular, I have no way of referencing the original ExceptionGroup without surprisingly cumbersome:
try: ......... except ExceptionGroup as eg0: unhandled = [] eg, os_eg = eg0.split(OSError) if os_eg: for e in os_eg: if an ok exception: handle it else: unhandled.append(e) if eg: eg, value_eg = eg.split(ValueError) if value_eg: for e in value_eg: if some_ValueError_we_understand: handle it else: unhandled.append(e) if eg: unhandled.append(eg) if unhandled: raise ExceptionGroup("unhandled", unhandled) from eg0
This is where except* can help:
try: ......... except except *OSError as eg: unhandled = [] handled, unhandled = eg.split(lambda e: e is an ok exception) # with side effect to handle e if unhandled: raise unhandled except *ValueError as eg: handled, unhandled = eg.split(lambda e: e is a value error we understand) # with side effect to handle e if unhandled:
Alas, that raises within each branch. I want to gather all the unhandled exceptions together into single ExceptionGroup. Using split gets me a bunch of groups, were I to defer the raise to after all the checking (as I want to in my pattern). So I've have to combine multiple groups back together. unhandled_groups = [] try: ......... except except *OSError as eg: handled, unhandled = eg.split(lambda e: e is an ok exception) # with side effect to handle e if unhandled: unhandled_groups.append(unhandled) except *ValueError as eg: handled, unhandled = eg.split(lambda e: e is a value error we understand) # with side effect to handle e if unhandled: unhandled_groups.append(unhandled) if unhandled_groups: # combine them here? new_eg = eg0.subgroup(lambda e: e in all-the-es-from-all-the-unhandled-groups) That last line needs eg0, the original ExceptionGroup (unavailable AFAICT), _and_ something to test for a naked exception being one of the unhandled ones. I _could_ gather the latter as a side effect of my split() lambda, but that makes things even more elaborate. A closure, even! If I could just iterate over the nested naked exceptions _none_ of this complexity would be required.
I have the following concerns with the pattern above:
There's no way to make a _new_ ExceptionGroup with the same __cause__ and __context__ and message as the original group: not that I can't assign to these, but I'd need to enuerate them; what if the exceptions grew new attributes I wanted to replicate?
As I said, split() and subgroup() do that for you.
They do if I'm only concerned with the particular exception subclass (eg OSError) i.e. my work is complete within the single except* clause. If I use except*, but later want to build an overarching "unhandled exceptions group", I have nothing to build on.
This cries out for another factory method like .subgroup but which makes a new ExceptionGroup from an original group, containing a new sequence of exceptions but the same message and coontext. Example:
unhandled_eg = eg0.with_exceptions(unhandled)
Why same message and context but not same cause and traceback?
Insufficient verbiage. I meant same message and context and cause. The point being that it is exceptly such a missing out of some attribute that I'd like to avoid by not having to enumerate the preserved attributes - the factory should do that, knowing the ExceptionGroup internals. Anyway, as you point out, .subgroup does this.
I don't see a documented way to access the group's message.
In the PEP: "The ExceptionGroup class exposes these parameters in the fields message and errors".
Hmm. Missed that. Thanks. I did find the PEP hard to read in some places. I think it could well do with saying "eg" instead of "e" throughout where "e" is an ExceptionGroup - I had a distinct tendency to want to read "e" as one of the naked exceptions and not the group.
I'm quite unhappy about .subgroup (and presumably .split) returning None when the group is empty. The requires me to have the gratuitous "if eg:" and "if value_eg:" if-statements in the example above.
If, instead, ExceptionGroups were like any other container I could just test if they were empty:
if eg:
_and_ even if they were empty, iterate over them. Who cares if the loop iterates zero times? [...] Anyway, I'm strongly of the opinion that ExceptionGroups should look like containers, be iterable, be truthy/falsey based on empty/nonempty and that .split and .subgroup should return empty subgroups instead of None.
This would be true if iteration was a useful pattern for working with ExceptionGroup, but I still think subgroup/split is a better tool in most cases.
I think I want to iterate over these things - it is useful to me. I want an ExceptionGroup to look like a container. I accept that subgroup makes a new group with metadata intact, but often that is not what _I_ care about, particularly when I'm just winnowing some special known cases. I cannot see that returning empty groups instead of None does any harm at all, and brings benefits in streamlining testing and processing. I guess if we don't get iteration and containerness I'll just have to subclass ExceptionGroup for my own use, giving it iterations and truthiness. It is hard to override subgroup and split to return empty ExceptionGroups though, without hacking with internals. Let's turn this on its head: - what specific harm comes from giving EGs container truthiness for size testing? - what specific harm comes from returning an empty EG from split on no match instead of None? - what specific harm comes from supporting iteration with a caveat about metadata in the docstring, and maybe a recommendation of subgroup? - if I wanted to override subgroup and split to not return None, is that even possible with the current ? i.e. can I make a clean metadata preserved EG from an empty list? For example: eg2 = eg.subgroup(lambda e: False) Does that get me None, or an empty group? If the latter, I can roll my own subclass with my desired features. If not, I can't AFAICT. EGs _have_ a .errors attribute which has all these aspects, why not expand it to the class as a whole? You seem very happen to implement 80% of what I want using callbacks (lambda e: ...), but I find explicit iteration much easier to read. I rarely use filter() for example, and often prefer a generator expression of list comprehension. Cheers, Cameron Simpson <cs@cskk.id.au>