[Cython] [cython-users] C++: how to handle failures of 'new'?

[Dag Sverre Seljebotn]

mark florisson <markflorisson88 at gmail.com> wrote:

>On 3 July 2012 20:15, Robert Bradshaw <robertwb at gmail.com> wrote:
>> On Tue, Jul 3, 2012 at 11:43 AM, Dag Sverre Seljebotn
>> <d.s.seljebotn at astro.uio.no> wrote:
>>> On 07/03/2012 08:23 PM, Robert Bradshaw wrote:
>>>>
>>>> On Tue, Jul 3, 2012 at 11:11 AM, Stefan Behnel<stefan_ml at behnel.de>
>>>> wrote:
>>>>>
>>>>> Robert Bradshaw, 03.07.2012 19:58:
>>>>>>
>>>>>> On Tue, Jul 3, 2012 at 9:38 AM, Stefan Behnel wrote:
>>>>>>>
>>>>>>> Dag Sverre Seljebotn, 03.07.2012 18:11:
>>>>>>>>
>>>>>>>> On 07/03/2012 09:14 AM, Stefan Behnel wrote:
>>>>>>>>>
>>>>>>>>> I don't know what happens if a C++ exception is not being
>caught, but
>>>>>>>>> I
>>>>>>>>> guess it would simply crash the application. That's a bit more
>>>>>>>>> visible than
>>>>>>>>
>>>>>>>>
>>>>>>>> Yep.
>>>>>>>>
>>>>>>>>> just printing a warning when a Python exception is being
>ignored due
>>>>>>>>> to a
>>>>>>>>> missing declaration. It's really unfortunate that our
>documentation
>>>>>>>>> didn't
>>>>>>>>> even mention the need for this, because it's not immediately
>obvious
>>>>>>>>> that
>>>>>>>>> Cython won't handle errors in "new", and testing for memory
>errors
>>>>>>>>> isn't
>>>>>>>>> quite what people commonly do in their test suites.
>>>>>>>>>
>>>>>>>>> Apart from that, I agree, users have to take care to properly
>declare
>>>>>>>>> the
>>>>>>>>> API they are using.
>>>>>>>>
>>>>>>>>
>>>>>>>> Is there any time you do NOT want a "catch (...) {}" block? I
>can't
>>>>>>>> see a
>>>>>>>> C++ exception propagating to Python-land doing anything useful
>ever.
>>>>>>>
>>>>>>>
>>>>>>> That would have been my intuition, too.
>>>>>>
>>>>>>
>>>>>> If it's actually embedded, with the main driver in C++, one might
>want
>>>>>> it to propagate up.
>>>>>
>>>>>
>>>>> But what kind of a propagation would that be? On the way out, it
>could
>>>>> induce anything, from side effects to resource leaks to crashes,
>>>>> depending
>>>>> on what the state of the surrounding code is. It would leave the
>whole
>>>>> system in an unpredictable state. I cannot imagine anyone really
>wanting
>>>>> this.
>>>>>
>>>>>
>>>>>>>> So shouldn't we just make --cplus turn *all* external functions
>and
>>>>>>>> methods
>>>>>>>> (whether C-like or C++-like) into "except +"? (Or keep except+
>for
>>>>>>>> manual
>>>>>>>> translation, but always have a catch(...)".
>>>>>>>>
>>>>>>>> Performance overhead is the only reason I can think of to not
>do this,
>>>>>>>> although IIRC C++ catch blocks are only dealt with during stack
>>>>>>>> unwinds and
>>>>>>>> doesn't cost anything/much (?) when they're not triggered.
>>>>>>>>
>>>>>>>> "except -1" should then actually mean both; "except + except
>-1". So
>>>>>>>> it's
>>>>>>>> more a question of just adding catch(...) *everywhere*, than
>making
>>>>>>>> "except
>>>>>>>> +" the default.
>>>>>>>
>>>>>>>
>>>>>>> I have no idea if there is a performance impact, but if there
>isn't,
>>>>>>> always
>>>>>>> catching all exceptions sounds like a reasonable thing to do.
>After
>>>>>>> all, we
>>>>>>> have no support for catching C++ exceptions on user side.
>>>>>>
>>>>>>
>>>>>> This is a bit like following every C call with "except *" (though
>the
>>>>>> performance ratios are unclear). It just seems a lot to wrap
>every
>>>>>> single line of a non-trivial C++ using function with try..catch
>>>>>> blocks.
>>>
>>>
>>> It seems "a lot" of just what exactly? Generated code? Binary size?
>Time
>>> spent in GCC parser?
>>
>> All of the above. And we should take a look at the runtime overhead
>> (which is hopefully nil, but who knows.)
>>
>>> Though I guess one might want to try to pull out the try-catch to at
>least
>>> only one per code line rather than one per SimpleCallNode.
>>
>> Or even higher, if possible. It's still a lot.
>
>Why would you have to do that? Can't you just insert a try/catch per
>try/except or try/finally block, or if absent, the function body. That
>will still work with the way temporaries are cleaned up. (It should
>also be implemented for parallel/prange sections).

One disadvantage is that you don't get source code line for the .pyx file in the stack trace. Which is often exactly the information you are looking for (even worse, since C++ stack isn't in the stack trace, the lineno for what seems like the ' ultimate cause' is not there). Having to surround statements with try/except just to pinpoint which one is raising the exception would be incredibly irritating.

Dag

>
>>> "except *" only has a point when calling functions using the CPython
>API,
>>> but most external C functions are pure C, not
>CPython-API-using-functions.
>>> OTOH, all external C++ functions are C++ :-)
>>
>> Fair point.
>>
>>> (Also, if we wrote Cython from scratch now I'm pretty sure the
>"except *"
>>> defaults would be a tad different.)
>>
>> For sure.
>>
>>>>> But if users are correct about their declarations, we'd end up
>with the
>>>>> same thing. I think it's worth a try.
>>>>
>>>>
>>>> Most C++ code (that I've ever run into) doesn't use exceptions,
>>>> because exception handling is so broken in C++ anyways.
>>>
>>>
>>> Except for the fact that any code touching "new" could be raising
>>> exceptions? That propagates.
>>
>> I would guess most of the time people don't bother catching these and
>> let the program die, as there's often no sane recovery (the same as
>> MemoryErrors in Python, though I guess C++ is less often used from an
>> event loop).
>>
>>> There is a lot of C++ code out there using exceptions. I'd guess
>that both
>>> mathematical code and Google-written code is unlike most C++ code
>out there
>>> :-) Many C++ programmers go on and on about RAII and auto_ptrs and
>so on,
>>> and that doesn't have much point unless you throw an exception now
>and then
>>> (OK, there's the occasional return statement where it matters well).
>>
>> True, I've seen a small subset of the C++ code that's out there.
>Maybe
>> numerical computations use it a lot?
>>
>> +1 to making catch-everywhere a directive at least. Lets see what the
>> impact is before we decide to make it the default.
>>
>> - Robert
>> _______________________________________________
>> cython-devel mailing list
>> cython-devel at python.org
>> http://mail.python.org/mailman/listinfo/cython-devel
>_______________________________________________
>cython-devel mailing list
>cython-devel at python.org
>http://mail.python.org/mailman/listinfo/cython-devel

-- 
Sent from my Android phone with K-9 Mail. Please excuse my brevity.