Re: [Numpy-discussion] Obscure code in concatenate code path?

On Sep 13, 2012, at 11:39 AM, Warren Weckesser wrote:

On Thu, Sep 13, 2012 at 9:01 AM, Travis Oliphant wrote:

On Sep 13, 2012, at 8:40 AM, Nathaniel Smith wrote:

...

On Thu, Sep 13, 2012 at 11:12 AM, Matthew Brett wrote:

...

Hi,

While writing some tests for np.concatenate, I ran foul of this code:

if (axis >= NPY_MAXDIMS) { ret = PyArray_ConcatenateFlattenedArrays(narrays, arrays, NPY_CORDER); } else { ret = PyArray_ConcatenateArrays(narrays, arrays, axis); }

in multiarraymodule.c

How deeply weird

This is expected behavior.

Heh, I guess "expected" is subjective:

"Expected" only in the sense that the current C-API has been intentional for 6 years. The side-effect of the Python-side being confusing can be changed --- it just hasn't been yet --- the documented approach is to use None. A patch to PyArray_AxisConverter might be the answer. -Travis

In [23]: np.__version__ Out[23]: '1.6.1'

In [24]: a = zeros((2,2))

In [25]: b = ones((2,3))

In [26]: concatenate((a, b), axis=0) # Expected error. --------------------------------------------------------------------------- ValueError Traceback (most recent call last) /Users/warren/gitwork/class-material/demo/pytables/<ipython-input-26-7cefb735e507> in <module>() ----> 1 concatenate((a, b), axis=0) # Expected error.

ValueError: array dimensions must agree except for d_0

In [27]: concatenate((a, b), axis=1) # Normal behavior. Out[27]: array([[ 0., 0., 1., 1., 1.], [ 0., 0., 1., 1., 1.]])

In [28]: concatenate((a, b), axis=2) # Cryptic error message. --------------------------------------------------------------------------- ValueError Traceback (most recent call last) /Users/warren/gitwork/class-material/demo/pytables/<ipython-input-28-0bce84c34ef1> in <module>() ----> 1 concatenate((a, b), axis=2) # Cryptic error message.

ValueError: bad axis1 argument to swapaxes

In [29]: concatenate((a, b), axis=32) # What the... ? Out[29]: array([ 0., 0., 0., 0., 1., 1., 1., 1., 1., 1.])

I would expect an error, consistent with the behavior when 1 < axis < 32.

Warren

It's how the concatenate Python function manages to handle axis=None to flatten the arrays before concatenation. This has been in NumPy since 1.0 and should not be changed without deprecation warnings which I am -0 on.

Now, it is true that the C-API could have been written differently (I think this is what Mark was trying to encourage) so that there are two C-API functions and they are dispatched separately from the array_concatenate method depending on whether or not a None is passed in. But, the behavior is documented and has been for a long time.

Reference PyArray_AxisConverter (which turns a "None" Python argument into an axis=MAX_DIMS). This is consistent behavior throughout the C-API.

-Travis

_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion

_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion