[C++-sig] Proposal to improve Python exception handling in Boost.Python

[Frank Benkstein]

Hi,

David Abrahams wrote:
> Frank Benkstein wrote:
>> David Abrahams wrote:
> I don't know if you're aware of this, but Boost.Python is currently
> incompatible with the use of Py_Finalize

We currently call Py_Initialize/Py_Finalize in our applications multiple
times while using Boost.Python and didn't notice any problems other than
ensuring that the error indicator is not set, i.e. calling
PyErr_Clear/PyErr_Print.

Is there a document somewhere summarizing what's expected to fail and
what must be done to make it fully work?

[This probably belongs into another thread.]

> All I was saying was that that single function probably ought to include
> the PyErr_Occurred check

If don't understand how that should work.  Do want to call this function
after every call to the C API?  The way I see it every function that
calls the C API has to check for errors itself.  If it detects one
(through a zero or a null pointer return value or a mandatory call to
PyErr_Occurred) it should call a function that knows how to translate
this error to a C++ exception.  Currently this translation is done by
throw_error_already_set.

>> To reiterate:
>>
>>  - If you catch error_already_set the error indicator is set, it is not
>>    safe to call Py_Finalize().
>>  - If you catch exceptions::Exception the error indicator is not set, it
>>    is safe to call Py_Finalize().
> 
> Okay... well, I really dislike the idea of having both things in the
> library at once, and I was never really thrilled with error_already_set.
>  We should think about how to replace it with your thing.  My guess is
> that if the library starts throwing something not derived from
> error_already_set, regardless of whether it leaves the Python error
> there, it's going to break more code than if we changed the meaning of
> error_already_set or "lied" by deriving your thing from it.  There will
> be transition issues, so we should discuss deprecation, compile-time
> checks, runtime checks, etc.

Yes, error_already_set should go.  An intermediate solution in the
deprecation phase could be a runtime switch so new and old code can
coexist.  In the transitional phase the base exception class could
provide a restore method that puts the error indicator back.  This way
code that's handling the Python exceptions itself can be changed from

catch (error_already_set &) {
  ...
}

to

catch (exceptions::python_exception &e) {
  e.restore();
  ...
}

Maybe it's possible to do some wizardry to do that automatically but I
don't know about that.  I also would like to think about deprecation
warnings and compile time switches later when there is any code and it
is in a working shape.

>>>>    And instance members "type", "value" and "traceback" so it can be
>>>>    set or restored as the Python error indicator using PyErr_Restore
>>>>    or PyErr_SetObject if the traceback is None.
>>> Do you really need to store the type in each instance?  Oh, maybe you
>>> do; it could end up being a python exception derived from one of the
>>> standard exceptions.  Then what's "mapped_type" (a.k.a. "python_type")
>>> for?
> 
> Inquiring minds wanna know.

Suppose you want to define a C++ exception class that wraps python's
LookupError.  This could look like the following:

<code>
// Define the wrapper class.
class LookupError {
public:
  // Every wrapper class must have this member.  It is a handle for the
  // Python type object whose instances this class will wrap.
  static object type;

  // This is the real type of the exception.  It may be LookupError but
  // also KeyError or IndexError or something else.
  object real_type;
  // This is the exception instance.
  object value;
  // This is the traceback or None.
  object traceback;

  LookupError(PyObject *t, PyObject *v, PyObject *tb);
};

LookupError::type = handle<>(borrowed(PyExc_LookupError));
</code>

To register the exception you would call something like this somewhere
in your code:

</code>
  exceptions::register<LookupError>();
</code>

After that calls to the function that will replace
throw_error_already_set will throw this LookupError class if
PyErr_ExceptionMatches(LookupError::type.ptr()).

The registry would ensure that subclasses come before their base
classes.

> I think I'd prefer a single analysis and proposal of what should be
> done.  We can talk about how to get there after I understand where we're
> going.

Ok, I think most of the pieces are already there but I'll try to come up
with something more formal.

Best regards,
Frank Benkstein.

-- 
Frank Benkstein

Software Development

Tel.: +49(0) 351 430093-27
Fax : +49(0) 351 430093-22

mailto:benkstein at langer-emv.de

Langer EMV-Technik GmbH
D-01728 Bannewitz
Nöthnitzer Hang 31
Germany

Registergericht: Amtsgericht Dresden, HRB 15402
Geschäftsführer: Gunter Langer

www.langer-emv.de


-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 252 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/cplusplus-sig/attachments/20080624/44f8f95e/attachment.pgp>