[Cython] [cython-users] C++: how to handle failures of 'new'?

mark florisson markflorisson88 at gmail.com
Fri Jul 6 11:33:00 CEST 2012 

On 6 July 2012 03:34, Dag Sverre Seljebotn <d.s.seljebotn at astro.uio.no> wrote:
>
>
> mark florisson <markflorisson88 at gmail.com> wrote:
>
>>On 5 July 2012 21:46, Dag Sverre Seljebotn <d.s.seljebotn at astro.uio.no>
>>wrote:
>>>
>>>
>>> mark florisson <markflorisson88 at gmail.com> wrote:
>>>
>>>>On 3 July 2012 20:15, Robert Bradshaw <robertwb at gmail.com> wrote:
>>>>> On Tue, Jul 3, 2012 at 11:43 AM, Dag Sverre Seljebotn
>>>>> <d.s.seljebotn at astro.uio.no> wrote:
>>>>>> On 07/03/2012 08:23 PM, Robert Bradshaw wrote:
>>>>>>>
>>>>>>> On Tue, Jul 3, 2012 at 11:11 AM, Stefan
>>Behnel<stefan_ml at behnel.de>
>>>>>>> wrote:
>>>>>>>>
>>>>>>>> Robert Bradshaw, 03.07.2012 19:58:
>>>>>>>>>
>>>>>>>>> On Tue, Jul 3, 2012 at 9:38 AM, Stefan Behnel wrote:
>>>>>>>>>>
>>>>>>>>>> Dag Sverre Seljebotn, 03.07.2012 18:11:
>>>>>>>>>>>
>>>>>>>>>>> On 07/03/2012 09:14 AM, Stefan Behnel wrote:
>>>>>>>>>>>>
>>>>>>>>>>>> I don't know what happens if a C++ exception is not being
>>>>caught, but
>>>>>>>>>>>> I
>>>>>>>>>>>> guess it would simply crash the application. That's a bit
>>more
>>>>>>>>>>>> visible than
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Yep.
>>>>>>>>>>>
>>>>>>>>>>>> just printing a warning when a Python exception is being
>>>>ignored due
>>>>>>>>>>>> to a
>>>>>>>>>>>> missing declaration. It's really unfortunate that our
>>>>documentation
>>>>>>>>>>>> didn't
>>>>>>>>>>>> even mention the need for this, because it's not immediately
>>>>obvious
>>>>>>>>>>>> that
>>>>>>>>>>>> Cython won't handle errors in "new", and testing for memory
>>>>errors
>>>>>>>>>>>> isn't
>>>>>>>>>>>> quite what people commonly do in their test suites.
>>>>>>>>>>>>
>>>>>>>>>>>> Apart from that, I agree, users have to take care to
>>properly
>>>>declare
>>>>>>>>>>>> the
>>>>>>>>>>>> API they are using.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Is there any time you do NOT want a "catch (...) {}" block? I
>>>>can't
>>>>>>>>>>> see a
>>>>>>>>>>> C++ exception propagating to Python-land doing anything
>>useful
>>>>ever.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> That would have been my intuition, too.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> If it's actually embedded, with the main driver in C++, one
>>might
>>>>want
>>>>>>>>> it to propagate up.
>>>>>>>>
>>>>>>>>
>>>>>>>> But what kind of a propagation would that be? On the way out, it
>>>>could
>>>>>>>> induce anything, from side effects to resource leaks to crashes,
>>>>>>>> depending
>>>>>>>> on what the state of the surrounding code is. It would leave the
>>>>whole
>>>>>>>> system in an unpredictable state. I cannot imagine anyone really
>>>>wanting
>>>>>>>> this.
>>>>>>>>
>>>>>>>>
>>>>>>>>>>> So shouldn't we just make --cplus turn *all* external
>>functions
>>>>and
>>>>>>>>>>> methods
>>>>>>>>>>> (whether C-like or C++-like) into "except +"? (Or keep
>>except+
>>>>for
>>>>>>>>>>> manual
>>>>>>>>>>> translation, but always have a catch(...)".
>>>>>>>>>>>
>>>>>>>>>>> Performance overhead is the only reason I can think of to not
>>>>do this,
>>>>>>>>>>> although IIRC C++ catch blocks are only dealt with during
>>stack
>>>>>>>>>>> unwinds and
>>>>>>>>>>> doesn't cost anything/much (?) when they're not triggered.
>>>>>>>>>>>
>>>>>>>>>>> "except -1" should then actually mean both; "except + except
>>>>-1". So
>>>>>>>>>>> it's
>>>>>>>>>>> more a question of just adding catch(...) *everywhere*, than
>>>>making
>>>>>>>>>>> "except
>>>>>>>>>>> +" the default.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> I have no idea if there is a performance impact, but if there
>>>>isn't,
>>>>>>>>>> always
>>>>>>>>>> catching all exceptions sounds like a reasonable thing to do.
>>>>After
>>>>>>>>>> all, we
>>>>>>>>>> have no support for catching C++ exceptions on user side.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> This is a bit like following every C call with "except *"
>>(though
>>>>the
>>>>>>>>> performance ratios are unclear). It just seems a lot to wrap
>>>>every
>>>>>>>>> single line of a non-trivial C++ using function with try..catch
>>>>>>>>> blocks.
>>>>>>
>>>>>>
>>>>>> It seems "a lot" of just what exactly? Generated code? Binary
>>size?
>>>>Time
>>>>>> spent in GCC parser?
>>>>>
>>>>> All of the above. And we should take a look at the runtime overhead
>>>>> (which is hopefully nil, but who knows.)
>>>>>
>>>>>> Though I guess one might want to try to pull out the try-catch to
>>at
>>>>least
>>>>>> only one per code line rather than one per SimpleCallNode.
>>>>>
>>>>> Or even higher, if possible. It's still a lot.
>>>>
>>>>Why would you have to do that? Can't you just insert a try/catch per
>>>>try/except or try/finally block, or if absent, the function body.
>>That
>>>>will still work with the way temporaries are cleaned up. (It should
>>>>also be implemented for parallel/prange sections).
>>>
>>> One disadvantage is that you don't get source code line for the .pyx
>>file in the stack trace. Which is often exactly the information you are
>>looking for (even worse, since C++ stack isn't in the stack trace, the
>>lineno for what seems like the ' ultimate cause' is not there). Having
>>to surround statements with try/except just to pinpoint which one is
>>raising the exception would be incredibly irritating.
>>>
>>> Dag
>>
>>Oh yeah, good point. Maybe we could use these zero-cost exceptions for
>>cdef functions in Cython though, instead of error checks (if it
>>appears to make any significant difference). Basically instead of the
>>'error' argument in CEP 526. It'd need version that ABI as well...
>>
>
> Wait, one can just do
>
> lineno = 25;
> a()
> linono = 27;
> b()
>
> And wrap the block with a try catch.

Heh, I just had the same idea before going to bed yesterday :) But it
means the common case (no exception) will get some overhead. That can
probably be afforded since normal error checking branches anyway.
Maybe it's worth investigating how many instructions try/catch blocks
generate and seeing whether a) it's (significantly) better than
branching and b) whether a try/catch block can be afforded per
subexpression/lineno.

> I'm not sure if I like C++ exceptions internally in Cython myself. It would mean that C-compiled Cython modules would not be able to call C++-compiled Cython modules, which I think would create a lot of headache. And I think we want to keep the ability to compile with a C compiler if you don't call C++ code...

I don't know how often people mix the two. It's probably not worth
making a directive either... Are there any platforms where installing
a C++ compiler is more of a hassle than a C compiler? E.g. Theano
depends on a C++ compiler (and a lot more as well).

The cool thing about it would be that as you said earlier in the
thread, you could at the very least do one try/catch per line number,
so you wouldn't need N branches for N subexpressions that can fail.

> Dag
>
>>>>
>>>>>> "except *" only has a point when calling functions using the
>>CPython
>>>>API,
>>>>>> but most external C functions are pure C, not
>>>>CPython-API-using-functions.
>>>>>> OTOH, all external C++ functions are C++ :-)
>>>>>
>>>>> Fair point.
>>>>>
>>>>>> (Also, if we wrote Cython from scratch now I'm pretty sure the
>>>>"except *"
>>>>>> defaults would be a tad different.)
>>>>>
>>>>> For sure.
>>>>>
>>>>>>>> But if users are correct about their declarations, we'd end up
>>>>with the
>>>>>>>> same thing. I think it's worth a try.
>>>>>>>
>>>>>>>
>>>>>>> Most C++ code (that I've ever run into) doesn't use exceptions,
>>>>>>> because exception handling is so broken in C++ anyways.
>>>>>>
>>>>>>
>>>>>> Except for the fact that any code touching "new" could be raising
>>>>>> exceptions? That propagates.
>>>>>
>>>>> I would guess most of the time people don't bother catching these
>>and
>>>>> let the program die, as there's often no sane recovery (the same as
>>>>> MemoryErrors in Python, though I guess C++ is less often used from
>>an
>>>>> event loop).
>>>>>
>>>>>> There is a lot of C++ code out there using exceptions. I'd guess
>>>>that both
>>>>>> mathematical code and Google-written code is unlike most C++ code
>>>>out there
>>>>>> :-) Many C++ programmers go on and on about RAII and auto_ptrs and
>>>>so on,
>>>>>> and that doesn't have much point unless you throw an exception now
>>>>and then
>>>>>> (OK, there's the occasional return statement where it matters
>>well).
>>>>>
>>>>> True, I've seen a small subset of the C++ code that's out there.
>>>>Maybe
>>>>> numerical computations use it a lot?
>>>>>
>>>>> +1 to making catch-everywhere a directive at least. Lets see what
>>the
>>>>> impact is before we decide to make it the default.
>>>>>
>>>>> - Robert
>>>>> _______________________________________________
>>>>> cython-devel mailing list
>>>>> cython-devel at python.org
>>>>> http://mail.python.org/mailman/listinfo/cython-devel
>>>>_______________________________________________
>>>>cython-devel mailing list
>>>>cython-devel at python.org
>>>>http://mail.python.org/mailman/listinfo/cython-devel
>>>
>>> --
>>> Sent from my Android phone with K-9 Mail. Please excuse my brevity.
>>> _______________________________________________
>>> cython-devel mailing list
>>> cython-devel at python.org
>>> http://mail.python.org/mailman/listinfo/cython-devel
>>_______________________________________________
>>cython-devel mailing list
>>cython-devel at python.org
>>http://mail.python.org/mailman/listinfo/cython-devel
>
> --
> Sent from my Android phone with K-9 Mail. Please excuse my brevity.
> _______________________________________________
> cython-devel mailing list
> cython-devel at python.org
> http://mail.python.org/mailman/listinfo/cython-devel

More information about the cython-devel mailing list