[Cython] [cython-users] C++: how to handle failures of 'new'?

Robert Bradshaw robertwb at gmail.com
Fri Jun 29 11:08:21 CEST 2012


On Thu, Jun 28, 2012 at 10:45 PM, Stefan Behnel <stefan_ml at behnel.de> wrote:
> [moving this to cython-devel as it's getting technical]
>
> Robert Bradshaw, 28.06.2012 21:46:
>> On Thu, Jun 28, 2012 at 11:38 AM, Stefan Behnel wrote:
>>> currently, when I write "new CppClass()" in Cython, it generates a straight
>>> call to the "new" operator. It doesn't do any error handling. And the
>>> current documentation doesn't even mention this case.
>>>
>>> Is there a "standard" way to handle this? It seems that C++ has different
>>> ways to deal with failures here but raises an exception by default. Would
>>> you declare the constructor(s) with an "except +MemoryError"? Is there a
>>> reason Cython shouldn't be doing this automatically (if nothing else was
>>> declared) ?
>>
>> I think it certainly makes sense to declare the default constructor as
>> "except +" (and std::bad_alloc should become MemoryError),
>
> Right. The code in the constructor can raise other exceptions that must
> also be handled properly. An explicit "except +" will handle that.
>
>
>> but whether
>> to implicitly annotate declared constructors is less clear, especially
>> as there's no way to un-annotate them.
>
> I agree, but sadly, it's the default behaviour that is wrong. I'm sure we
> made lots of users run into this trap already. I fixed the documentation
> for now, but the bottom line is that we require users to take care of
> proper declarations themselves. Otherwise, the code that we generate is
> incorrect, although it's 100% certain that an allocation error can occur,
> even if the constructor code doesn't raise any exceptions itself.

This is always the case.

> Apparently, changing the behaviour of the "new" operator requires a special
> annotation "std::nothrow", which then returns NULL on allocation failures.
> You can pass that from Cython by hacking up a cname, e.g.
>
>    Rectangle "(std::nothrow) Rectangle" (int w, int h)
>
> I'm sure there are users out there who figured this out (I mean, I did...)
> and use it in their code, so I agree that this isn't easy to handle because
> Cython simply wouldn't know what the actual error behaviour is for a given
> constructor and how to correctly detect an error.
>
> This problem applies only to heap allocation in that form. However, stack
> allocation and the new exttype field allocation suffer from similar
> problems when the default constructor raises an exception. Exttype fields
> are a particularly nasty case because the user has no control over the
> allocation. A C++ exception in the C++ class constructor would terminate
> the exttype constructor unexpectedly and thus leak resources (in the best
> case - no idea how CPython reacts if you throw a C++ exception through its
> type instantiation code).

If the default constructor raises an exception then it should be
declared (to not do so is an error on the users part). New raising
bad_alloc is a bit of a special case, but doesn't appl to the stack or
exttype allocations.

> Similarly, a C++ exception in the constructor of a stack allocated object
> would then originate from the function entry code and potentially hit the
> Python function wrapper etc. Again, potentially leaking resources or worse.
>
> To me, this sounds like we should do something about it. At least for the
> implicit calls to the default constructor, we should generate "except +"
> code automatically because there is no other way to handle them safely.

If no constructor is declared, it should be "except +" just to be
safe, but otherwise I don't see how this is any different than
forgetting to declare exceptions on any other function. Unfortunately
catching exceptions  (with custom per-object handling) on a set of
stack allocated objects seems difficult if not impossible (without
resorting to ugly hacks like using placement new everywhere).

- Robert


More information about the cython-devel mailing list