[Numpy-discussion] using scalar input on np.PyArray_MultiIterNew2
blkzol001 at myuct.ac.za
Sun Jan 10 13:39:02 EST 2021
Sebastian Berg wrote
> On Sun, 2021-01-10 at 09:59 -0700, zoj613 wrote:
>> Hi all,
>> I am looking for a way to use `np.PyArray_MultiIterNew2` in Cython to
>> broadcast parameters of a function. The requirement is that the two
>> arguments can be scalar and/or sequences. Using the usual
>> function works well but is slow when iterating over the broadcasted
>> input in
>> a tight loop. I want to achieve the same using the C API.
> NB: There is also a different, newer, iterator (`NpyIter_New`).
> Depending on what you do, that might be a lot faster and better
> (although, I admit it tends to be more verbose too).
> That iterator also does broadcasting, and one nice property is that it
> can allocate a result array. Most importantly, it casts for you and
> allows you to take charge of the inner-loop (a one dimensional loop)
> for performance. (This is the core of how ufuncs work.)
>> Currently, if I used `(<double*>np.PyArray_MultiIter_DATA(bcast,
>> i))` to
>> iterate over the input when one of them is a scalar,
>> I get no errors, but I notice the output of the parent function
>> returns an
>> array of zeros, which implies this approach didn't work. After
>> investigating, it seems that np.PyArray_MultiIter_DATA only accepts
> I am honestly confused by this (did you mean a different command?).
> `PyArray_MultiIter_DATA` returns a pointer to the raw data, i.e. in
> cython you would probably do:
> cdef double *value_ptr =
> <double *>
> npc.PyArray_MultiIter_DATA(iter, 0)
> value_ptr = 3.1416
> Do want to reference the original data? You could reference the
> original data for scalars (read-only since scalars are immutable), but
> for lists/tuples there is no original data to begin with (unless you
> have Python objects inside, but it would seem strange if NumPy to
> attempted to transparently expose that).
>> I could write a function to handle all combinations of
>> scalar/array/list/tuple, and create temporary arrays to store the
> The typical pattern is to convert everything to array first, but
> `PyArray_MultiIter()` does that for you. Unless you want to optimize
> that conversion away?
>> data, but that seems daunting and error prone. Is there a way I can
>> this and have scalar arguments passed to np.PyArray_MultiIter_DATA be
>> converted to same-element arrays without writing my own code from
>> Any suggestions are welcome.
>> Sent from: http://numpy-discussion.10968.n7.nabble.com/
>> NumPy-Discussion mailing list
> I think I wasn't clear enough in my original question. Here is what I
> a python function with the signature: def pyfunc(a, b). This function
> calls another cython function internally such that
> def pyfunc(a, b):
> return cfunc(a, b)
> I want the pyfunc to support scalar and array_like input for both
> parameters, so that internally I can have:
> def pyfunc(a, b):
> # some part of the function
> for i, j in some_broadcasted_iterator:
> output[indx] = cfunc(i, j)
> Broadcasting using python level functions allows me to do this but then
> the iteration part is slow because it uses python python code, meaning I
> can't release the gil. What I was thinking of doing is using the C API to
> create the broadcasted iterator using PyArray_MultiIterNew2 then iterating
> over that. I kept getting a zero array as output when I tested it so I
> assumed that the function doesnt work with scalars. It turns out the bug
> was caused by me using integer scalars when the C function expects doubles
> (But I fixed that after creating this post).
> Now I'm left with figuring out how I can efficiently convert even
> lists/tuple inputs into a broadcasted iterator (without requiring the user
> to pass numpy arrays). When I pass a list for parameter a and a scalar for
> parameter b, the program hangs, so Im not sure if the
> PyArray_MultiIterNew2 function does it for me?
> Btw, do you have a link to the more recent NpyIter_New? I couldn't find it
> reading the C-API page in the docs.
> NumPy-Discussion mailing list
> signature.asc (849 bytes)
Sent from: http://numpy-discussion.10968.n7.nabble.com/
More information about the NumPy-Discussion