[C++-sig] Proposal to improve Python exception handling in Boost.Python

[David Abrahams]

Frank Benkstein wrote:
> Hi,
> 
> David Abrahams wrote:
>> Frank Benkstein wrote:
>>> David Abrahams wrote:
>> I don't know if you're aware of this, but Boost.Python is currently
>> incompatible with the use of Py_Finalize
> 
> We currently call Py_Initialize/Py_Finalize in our applications multiple
> times while using Boost.Python and didn't notice any problems other than
> ensuring that the error indicator is not set, i.e. calling
> PyErr_Clear/PyErr_Print.
> 
> Is there a document somewhere summarizing what's expected to fail and
> what must be done to make it fully work?

file:///tmp/todo.html#pyfinalize-safety is the best I can come up with.

>> All I was saying was that that single function probably ought to include
>> the PyErr_Occurred check
> 
> If don't understand how that should work.  Do want to call this function
> after every call to the C API?  The way I see it every function that
> calls the C API has to check for errors itself.  If it detects one
> (through a zero or a null pointer return value or a mandatory call to
> PyErr_Occurred) it should call a function that knows how to translate
> this error to a C++ exception.  Currently this translation is done by
> throw_error_already_set.

And why should a function that would only check by calling
PyErr_Occurred have to check for itself?

An extra call to PyErr_Occurred when you're about to throw a C++
exception anyway (e.g. because you detected a -1) isn't going to hurt
anyone.

> Yes, error_already_set should go.  An intermediate solution in the
> deprecation phase could be a runtime switch so new and old code can
> coexist.  In the transitional phase the base exception class could
> provide a restore method that puts the error indicator back.  This way
> code that's handling the Python exceptions itself can be changed from
> 
> catch (error_already_set &) {
>   ...
> }
> 
> to
> 
> catch (exceptions::python_exception &e) {
>   e.restore();
>   ...
> }
> 
> Maybe it's possible to do some wizardry to do that automatically but I
> don't know about that.  I also would like to think about deprecation
> warnings and compile time switches later when there is any code and it
> is in a working shape.

Suit yourself, but I won't want to proceed until the transition issue
has been carefully thought through and discussed.

> Suppose you want to define a C++ exception class that wraps python's
> LookupError.  This could look like the following:
> 
> <code>
> // Define the wrapper class.
> class LookupError {
> public:
>   // Every wrapper class must have this member.  It is a handle for the
>   // Python type object whose instances this class will wrap.
>   static object type;
> 
>   // This is the real type of the exception.  It may be LookupError but
>   // also KeyError or IndexError or something else.
>   object real_type;
>   // This is the exception instance.
>   object value;
>   // This is the traceback or None.
>   object traceback;
> 
>   LookupError(PyObject *t, PyObject *v, PyObject *tb);
> };
> 
> LookupError::type = handle<>(borrowed(PyExc_LookupError));
> </code>
> 
> To register the exception you would call something like this somewhere
> in your code:
>
> </code>
>   exceptions::register<LookupError>();
> </code>
> 

Hmm.  My intention had always been to implement the feature discussed in
http://article.gmane.org/gmane.comp.python.c%2B%2B/3812,
Is this compatible with those ideas?

> After that calls to the function that will replace
> throw_error_already_set will throw this LookupError class if
> PyErr_ExceptionMatches(LookupError::type.ptr()).
> 
> The registry would ensure that subclasses come before their base
> classes.
> 
>> I think I'd prefer a single analysis and proposal of what should be
>> done.  We can talk about how to get there after I understand where we're
>> going.
> 
> Ok, I think most of the pieces are already there but I'll try to come up
> with something more formal.

I don't need anything more formal; I'd just like you to leave out
intermediate steps: propose where you think things should end up, not a
plan for getting there over several releases of the code (or several SVN
checkins), just to keep things clearer for me.

Thanks,

-- 
Dave Abrahams
BoostPro Computing
http://www.boostpro.com