Thanks for the addendum--I suspected a mix of performance and the difficulty of handling potential failures at that point would be the ultimate answer to why the exception is already cut out of the loop there. Not that the dumber __raised__() version wouldn't have the same problem, but its behavior in case of failure could be straightforward. An idea I had last night led to some progress this morning. BaseException's with_traceback(tb) method already sets some thematic precedent for users being able to cause new exceptions while setting values within the raise process, (though unfortunately it looks like the traceback it sets gets overwritten by the new one set in C during the raise.) I realize that this is just a thematic precedent, since they'd be caused at very different places. For now I've replaced my previous workaround with a with_cause(c) method which is at least a step in the right direction for handling my specific case in a way that feels roughly congruent with existing idioms, makes sense in that context, and cleans up code using the exceptions; __cause__ still gets overwritten by the C code, but this doesn't cause trouble since we know what this object will be at raise time. This wouldn't suffice if I needed access to __traceback__ at this point. RE the question in your first response on patch benchmarking: I'm new to the list and have no previous contributions, but there's a first time for everything; if the information on doing so is readily available I should be fine, but otherwise I'd need some direction. On Fri, Apr 10, 2015 at 5:07 PM, Andrew Barnert wrote:

On Apr 10, 2015, at 14:39, Andrew Barnert < abarnert@yahoo.com.dmarc.invalid> wrote:

On Apr 10, 2015, at 12:54, Travis Everett wrote:

Hi all,

I have a specialized use case for exceptions which need to analyze their __cause__ property before they're fully initialized. The property isn't set until after init, and (forgive my ignorance of Python's source!) appears to be done at the C level in a way that it can't be observed with a property setter or __setattr__. I currently create a method to do this post-init work as a workaround, and push the burden of making sure it gets called on the code using the exceptions--but this makes the exceptions idiomatic and error-prone.

A magic method called once all of the special exception properties are already set would suffice. The exception could define something like __raised__(self) to react to the new values of those properties.

A more comprehensive approach might be a __raise__(self, cause, context, traceback...) method which has full control over setting the properties, but I assume there are good reasons why the usual channels (setters and __setattr__) are cut out of the loop.

I'd assume the only reasons are to make the code a bit simpler and a bit more efficient.

Actually, on second though, it's not _quite_ that simple.

First, settings args, __traceback__ etc. are no longer hookable; it seems like you'd want those to be as well, for consistency? So that's a few more functions to change.

Second, the API functions for setting cause and context (and the private API function for automatically setting context) can't possibly fail anywhere, so they have no return value. Changing them to go through SetAttr means they now can (your __setattr__ could raise anything it wants--or even creating the 1 object to store in __suppress_context__ could fail). Should we just swallow and ignore any such exceptions when called from the C error API? (C code that uses the exception type API or the object API will of course continue to get exceptions, as will Python code, it's just the error machinery that would ignore errors from munging exceptions to be raised.)

Adding some new __raise__ or __raised__ special-purpose protocol doesn't seem like it would be any simpler or more efficient than just changing the existing APIs to access the attributes via SetAttr, which would give you what you want without being a special case.

Also notice that there's an open PEP ( https://www.python.org/dev/peps/pep-0490/) to make the C API for raising exceptions automatically set the context instead of forcing it to be done manually (with a private API function), as a few builtin and stdlib functions do. So it seems safer to change things in a way that would work equally well with the current APIs, with that change, and with all reasonable alternatives to that change--and again, using SetAttr seems like the simplest way to make sure of that.

But either way, it's a relatively local and simple change in Objects/exceptions.c.

I suspect the big thing you'd need to get this accepted is to show that the performance cost is negligible. Do you know how to run appropriate benchmarks and present the results if someone else writes the patch?

I was encouraged to bring discussion here (to see if there's any traction) after opening an enhancement request ( http://bugs.python.org/issue23902).

Cheers, Travis

_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

There are certainly still questions about appropriate behavior. I think the least-surprising result of a failure in e.__raised__ is to raise with context, but I realize I'm naive about the implementation-level implications of doing so. I thought twice about using that word, but by calling __raised__ straightforward, I meant that a failure in the __raise__ version creates more ambiguity in regard to what those properties will contain. Does it keep only the properties successfully set before failure and attempt no recovery? Does it try to make lemonade by keeping the successful properties, but set default values for the others? Does it overwrite all values with defaults as if the magic method wasn't defined? I'd say the first is least surprising because it lacks magic, but it means a failure can break the cause/context chain; these all seem quite surprising in their own ways. On Sat, Apr 11, 2015 at 8:15 PM, Andrew Barnert wrote:

On Apr 11, 2015, at 08:40, Travis Everett wrote:

Thanks for the addendum--I suspected a mix of performance and the difficulty of handling potential failures at that point would be the ultimate answer to why the exception is already cut out of the loop there. Not that the dumber __raised__() version wouldn't have the same problem, but its behavior in case of failure could be straightforward.

Is it straightforward? When e.__raised__ raises, do you want to just swallow it and raise e anyway, or raise the exception raised by e.__raised__ with e as a context, or lose e and just raise the exception raised by e.__raised__? (Or you could just punt like C++ and say that if an exception type itself raises an exception in a place that's hard to deal with, the program terminates. :) Remember the possibility that what e.__raised__ raised may be a MemoryError, which always makes things extra fun.

You have pretty much the same choices in setting the __cause__ or __context__; some of the answers may turn out to be harder to implement there than with __raised__, but instead of just assuming that means the ideal version of your proposal won't work, it's probably better to decide what behavior is actually right, and then see if it turns out to be impossible.

So... Which one do you think is right in each case?

An idea I had last night led to some progress this morning. BaseException's with_traceback(tb) method already sets some thematic precedent for users being able to cause new exceptions while setting values within the raise process, (though unfortunately it looks like the traceback it sets gets overwritten by the new one set in C during the raise.) I realize that this is just a thematic precedent, since they'd be caused at very different places.

For now I've replaced my previous workaround with a with_cause(c) method which is at least a step in the right direction for handling my specific case in a way that feels roughly congruent with existing idioms, makes sense in that context, and cleans up code using the exceptions; __cause__ still gets overwritten by the C code, but this doesn't cause trouble since we know what this object will be at raise time. This wouldn't suffice if I needed access to __traceback__ at this point.

RE the question in your first response on patch benchmarking: I'm new to the list and have no previous contributions, but there's a first time for everything; if the information on doing so is readily available I should be fine, but otherwise I'd need some direction.

On Fri, Apr 10, 2015 at 5:07 PM, Andrew Barnert wrote:

...

On Apr 10, 2015, at 14:39, Andrew Barnert < abarnert@yahoo.com.dmarc.invalid> wrote:

On Apr 10, 2015, at 12:54, Travis Everett wrote:

Hi all,

I have a specialized use case for exceptions which need to analyze their __cause__ property before they're fully initialized. The property isn't set until after init, and (forgive my ignorance of Python's source!) appears to be done at the C level in a way that it can't be observed with a property setter or __setattr__. I currently create a method to do this post-init work as a workaround, and push the burden of making sure it gets called on the code using the exceptions--but this makes the exceptions idiomatic and error-prone.

A magic method called once all of the special exception properties are already set would suffice. The exception could define something like __raised__(self) to react to the new values of those properties.

A more comprehensive approach might be a __raise__(self, cause, context, traceback...) method which has full control over setting the properties, but I assume there are good reasons why the usual channels (setters and __setattr__) are cut out of the loop.

I'd assume the only reasons are to make the code a bit simpler and a bit more efficient.

Actually, on second though, it's not _quite_ that simple.

First, settings args, __traceback__ etc. are no longer hookable; it seems like you'd want those to be as well, for consistency? So that's a few more functions to change.

Second, the API functions for setting cause and context (and the private API function for automatically setting context) can't possibly fail anywhere, so they have no return value. Changing them to go through SetAttr means they now can (your __setattr__ could raise anything it wants--or even creating the 1 object to store in __suppress_context__ could fail). Should we just swallow and ignore any such exceptions when called from the C error API? (C code that uses the exception type API or the object API will of course continue to get exceptions, as will Python code, it's just the error machinery that would ignore errors from munging exceptions to be raised.)

Adding some new __raise__ or __raised__ special-purpose protocol doesn't seem like it would be any simpler or more efficient than just changing the existing APIs to access the attributes via SetAttr, which would give you what you want without being a special case.

Also notice that there's an open PEP ( https://www.python.org/dev/peps/pep-0490/) to make the C API for raising exceptions automatically set the context instead of forcing it to be done manually (with a private API function), as a few builtin and stdlib functions do. So it seems safer to change things in a way that would work equally well with the current APIs, with that change, and with all reasonable alternatives to that change--and again, using SetAttr seems like the simplest way to make sure of that.

But either way, it's a relatively local and simple change in Objects/exceptions.c.

I suspect the big thing you'd need to get this accepted is to show that the performance cost is negligible. Do you know how to run appropriate benchmarks and present the results if someone else writes the patch?

I was encouraged to bring discussion here (to see if there's any traction) after opening an enhancement request ( http://bugs.python.org/issue23902).

Cheers, Travis

_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/