[Numpy-discussion] failure to register ufunc loops for user defined types

[mark florisson]

On 5 December 2011 17:25, Mark Wiebe <mwwiebe at gmail.com> wrote:
> On Sun, Dec 4, 2011 at 11:37 PM, Geoffrey Irving <irving at naml.us> wrote:
>>
>> <snip>
>>
>>
>> Back to the bugs: here's a branch with all the changes I needed to get
>> rational arithmetic to work:
>>
>>    https://github.com/girving/numpy
>>
>> I discovered two more after the last email.  One is another simple 0
>> vs. 1 bug, and another is somewhat optional:
>>
>> commit 730b05a892371d6f18d9317e5ae6dc306c0211b0
>> Author: Geoffrey Irving <irving at naml.us>
>> Date:   Sun Dec 4 20:03:46 2011 -0800
>>
>>    After loops, check for PyErr_Occurred() even if needs_api is 0
>>
>>    For certain types of user defined classes, casting and ufunc loops
>>    normally run without the Python API, but occasionally need to throw
>>    an error.  Currently we assume that !needs_api means no error occur.
>>    However, the fastest way to implement such loops is to run without
>>    the GIL normally and use PyGILState_Ensure/Release if an error occurs.
>>
>>    In order to support this usage pattern, change all post-loop checks
>> from
>>
>>        needs_api && PyErr_Occurred()
>>
>>    to simply
>>
>>        PyErr_Occurred()
>
>
> To support this properly, I think we would need to convert needs_api into an
> enum with this hybrid mode as another case. While it isn't done currently, I
> was imagining using a thread pool to multithread the trivially data-parallel
> operations when needs_api is false, and I suspect the
> PyGILState_Ensure/Release would trigger undefined behavior in a thread
> created entirely outside of the Python system.

PyGILState_Ensure/Release can be safely used by non-python threads
with the only requirement that the GIL has been initialized previously
in the main thread (PyEval_InitThreads).

> For comparison, I created a
> special mechanism for simplified multi-threaded exceptions in the nditer in
> the 'errmsg' parameter:
>
> http://docs.scipy.org/doc/numpy/reference/c-api.iterator.html#NpyIter_GetIterNext
>
> Worth considering is also the fact that the PyGILState API is incompatible
> with multiple embedded interpreters. Maybe that's not something anyone does
> with NumPy, though.
>
> -Mark
>
>>
>>
>> Geoffrey
>> _______________________________________________
>> NumPy-Discussion mailing list
>> NumPy-Discussion at scipy.org
>> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>