[Cython] [cython-users] Re: How to find out where an AttributeError is ignored

[mark florisson]

On 27 January 2012 20:03, Dag Sverre Seljebotn
<d.s.seljebotn at astro.uio.no> wrote:
> On 01/27/2012 05:58 PM, Stefan Behnel wrote:
>>
>> mark florisson, 27.01.2012 17:30:
>>>
>>> On 27 January 2012 16:22, mark florisson<markflorisson88 at gmail.com>
>>>  wrote:
>>>>
>>>> On 27 January 2012 15:47, Simon King<simon.king at uni-jena.de>  wrote:
>>>>>
>>>>> Hi all,
>>>>>
>>>>> I am still *very* frustrated about the fact that Cython does not tell
>>>>> where the error occurs. Since about one week, I am adding lots and
>>>>> lots of lines into Sage that write a log into some file, so that I get
>>>>> at least some idea where the error occurs. But still: Even these
>>>>> extensive logs do not provide a hint on what exactly is happening.
>>>>>
>>>>> How can I patch Cython such that some more information on the location
>>>>> of the error is printed? I unpacked Sage's Cython spkg, and did "grep -
>>>>> R ignored .", but the code lines containing the word "ignored" did not
>>>>> seem to be the lines that are responsible for printing the warning
>>>>> message
>>>>>   Exception AttributeError: 'PolynomialRing_field_with_category'
>>>>> object has no attribute '_modulus' in  ignored
>>>>>
>>>>> Can you point me to the file in Sage's Cython spkg which is
>>>>> responsible for printing the warning?
>>>>>
>>>>> Best regards,
>>>>> Simon
>>>>
>>>>
>>>> These messages are written by PyErr_WriteUnraisable, which is a
>>>> CPython C API function that writes unraisable exceptions. There are
>>>> typically two reasons for unraisable exceptions:
>>>>
>>>>    1) as Robert mentioned, a function that does not allow propagation
>>>> of exceptions, e.g.
>>>>
>>>>        cdef int func():
>>>>            raise Exception
>>>>
>>>>        Here there is no way to propagate the raised exception, so
>>>> instead one should write something like
>>>>
>>>>            cdef int func() except -1: ...
>>>>
>>>>        Alternatively one may use 'except *' in case there is no error
>>>> indicator and Cython should always check, or "except ? -1" which means
>>>> "-1 may or may not indicate an error".
>>>>
>>>>    2) in deallocators or finalizers (e.g. __dealloc__ or __del__)
>>>>
>>>> For functions the right thing is to add an except clause, for
>>>> finalizers and destructors one could use the traceback module, e.g.
>>>>
>>>>    try:
>>>>        ...
>>>>    except:
>>>>        traceback.print_exc()
>>>>
>>>> If this all still doesn't help, try setting a (deferred) breakpoint on
>>>> __Pyx_WriteUnraisable or PyErr_WriteUnraisable.
>>>
>>>
>>> Actually, I don't see why the default is to write unraisable
>>> exceptions. Instead Cython could detect that exceptions may propagate
>>> and have callers do the check (i.e. make it implicitly "except *").
>
>
> As for speed, there's optimizations on this, e.g., "except? 32434623" if the
> return type is int, "except? 0xfffff..." if the return type is a pointer.
>
> And for floating point, we could make our own NaN -- that's obscure enough
> that it could probably be made "except cython.cython_exception_nan" by
> default, not "except? cython.cython_exception_nan".
>
>
>>> Was this not implemented because Cython only knows whether functions
>>> may propagate exceptions at code generation time by looking at the
>>> presence of an error label?
>>> Maybe it could keep code insertion points around for every call to
>>> such a potential function and if the function uses the error label
>>> have the caller perform the check? Although I do forsee problems for
>>> external such functions... maybe Cython could have it's own
>>> threadstate regardless of the GIL which would indicate whether an
>>> error has occurred? e.g. CyErr_Occurred()?
>>
>>
>> Yep, those are the kind of reasons why writing unraisable exceptions is
>> the
>> default.
>
>
> Still, the need to explicitly declare "except *" keeps coming up again and
> again, and is really a blemish on the usability of Cython. When teaching
> people Cython, then it's really irritating to have to follow "all you need
> to do is add some 'cdef' and some types" with "and then you need to remember
> to say "except *", or you're in deep trouble". Cython sort of looks very
> elegant until that point...

I totally agree. The syntax is kind of bothersome, 'with gil' and
'nogil' are bad enough as it is.

> Long-term we should change CPython to make sure that PyErr_Occurred doesn't
> need the GIL :-) (there's really no reason it should need to go beyond
> checking a thread-local variable).

I think the problem is that thread states are not actually
thread-local data. They are saved on the stack when you release the
GIL and restored when you re-acquire it. As such I think it just keeps
one global C variable around. I have no idea how much overhead it
would be to make it a thread-local variable. But if needed we could
always use a cython-specific thread-local error indicator.

> We could also change the Cython ABI/calling convention -- use the return
> code always for reporting error status (unless there's an "except" or it is
> "cdef extern") and report return value in a pointer out-argument. That'd
> generalize to support multiple typed return values as well. Of course,
> there's a lot of downsides to changing ABI...

I think you can't change the ABI for a public/api function, as it
would need to be callable from C. Also, if the address of the function
is takes you can't do it. But those could be special cases, which
would be promoted to the "except badval" case or the "except *" case.
Or perhaps public and api functions without an except clause should
always use the write unraisable approach, as their callers might be
ignoring the exception entirely (although that could be viewed as a
bug as well).

In general I think a cython thread-local error indicator (in addition
to checking against bad values) would be robust. It wouldn't need to
add a lot of overhead as you can often use the bad value approach
anyway.

> Dag Sverre
>
> _______________________________________________
> cython-devel mailing list
> cython-devel at python.org
> http://mail.python.org/mailman/listinfo/cython-devel