[Numpy-discussion] failure to register ufunc loops for user defined types

Mark Wiebe mwwiebe at gmail.com
Mon Dec 5 12:25:19 EST 2011


On Sun, Dec 4, 2011 at 11:37 PM, Geoffrey Irving <irving at naml.us> wrote:

> <snip>
>
> Back to the bugs: here's a branch with all the changes I needed to get
> rational arithmetic to work:
>
>    https://github.com/girving/numpy
>
> I discovered two more after the last email.  One is another simple 0
> vs. 1 bug, and another is somewhat optional:
>
> commit 730b05a892371d6f18d9317e5ae6dc306c0211b0
> Author: Geoffrey Irving <irving at naml.us>
> Date:   Sun Dec 4 20:03:46 2011 -0800
>
>    After loops, check for PyErr_Occurred() even if needs_api is 0
>
>    For certain types of user defined classes, casting and ufunc loops
>    normally run without the Python API, but occasionally need to throw
>    an error.  Currently we assume that !needs_api means no error occur.
>    However, the fastest way to implement such loops is to run without
>    the GIL normally and use PyGILState_Ensure/Release if an error occurs.
>
>    In order to support this usage pattern, change all post-loop checks from
>
>        needs_api && PyErr_Occurred()
>
>    to simply
>
>        PyErr_Occurred()
>

To support this properly, I think we would need to convert needs_api into
an enum with this hybrid mode as another case. While it isn't done
currently, I was imagining using a thread pool to multithread the trivially
data-parallel operations when needs_api is false, and I suspect the
PyGILState_Ensure/Release would trigger undefined behavior in a thread
created entirely outside of the Python system. For comparison, I created a
special mechanism for simplified multi-threaded exceptions in the nditer in
the 'errmsg' parameter:

http://docs.scipy.org/doc/numpy/reference/c-api.iterator.html#NpyIter_GetIterNext

Worth considering is also the fact that the PyGILState API is incompatible
with multiple embedded interpreters. Maybe that's not something anyone does
with NumPy, though.

-Mark


>
> Geoffrey
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20111205/a56ea16f/attachment.html>


More information about the NumPy-Discussion mailing list