[Cython] Upcoming issues with NumPy deprecated APIs and Cython's sizeof checks

mark florisson markflorisson88 at gmail.com
Tue Jan 31 22:58:01 CET 2012


On 31 January 2012 21:11, Dag Sverre Seljebotn
<d.s.seljebotn at astro.uio.no> wrote:
> On 01/31/2012 09:53 PM, mark florisson wrote:
>>
>> On 31 January 2012 15:40, Dag Sverre Seljebotn
>> <d.s.seljebotn at astro.uio.no>  wrote:
>>>
>>> On 01/31/2012 03:29 PM, mark florisson wrote:
>>>>
>>>>
>>>> On 30 January 2012 21:03, Lisandro Dalcin<dalcinl at gmail.com>    wrote:
>>>>>
>>>>>
>>>>> I'm testing my code with numpy-dev. They are trying to discourage use
>>>>> of deprecated APIs, this includes direct access to the ndarray struct.
>>>>> In order to update your code, you have to pass -DNPY_NO_DEPRECATED_API
>>>>> to the C compiler (or #define it before including NumPy headers).
>>>>>
>>>>> However, they have implemented this feature by exposing the ndarray
>>>>> type with just the Python object header:
>>>>>
>>>>>
>>>>> https://github.com/numpy/numpy/blob/master/numpy/core/include/numpy/ndarraytypes.h#L695
>>>>>
>>>>> Obviously, this interact bad with Cython's sizeof check, I'm getting
>>>>> this runtime warning:
>>>>>
>>>>> build/lib.linux-x86_64-2.7/petsc4py/lib/__init__.py:64:
>>>>> RuntimeWarning: numpy.ndarray size changed, may indicate binary
>>>>> incompatibility
>>>>>
>>>>> I think there is nothing Cython can do about this (other than
>>>>> special-casing NumPy to disable this VERY useful warning).
>>>
>>>
>>>
>>> Hmm...but you can still recompile the Cython module and then don't get
>>> the
>>> warning, right?
>>>
>>> We've already been through at least one such round. People tend to ignore
>>> it, or install warning filters...
>>>
>>> If one does want a workaround, we don't have to special case NumPy as
>>> such
>>> -- I think it is marginally cleaner to add new obscure syntax which we
>>> only
>>> use in numpy.pxd:
>>>
>>>    ctypedef class numpy.ndarray [object PyArrayObject nosizecheck]:
>>>
>>> Or, if anybody bothers, a way to register and automatically run the
>>> functions NumPy provides for checking ABI compatability.
>>>
>>> I don't think any changes should be done on the NumPy end.
>>>
>>
>> I really don't care about the warning, more about the possibility of
>> numpy rearranging/removing or adding to its private fields (I guess
>> they won't be doing that anytime soon, but prudence is a virtue). I
>> suppose we would notice any changes soon enough though, as it would
>> break all the code.
>>
>>>>
>>>> Weird, shouldn't you be getting an error? Because the size of the
>>>> PyArrayObject should be less than what Cython expects.
>>>>
>>>>>  I've tried the patch below with success, but I'm not convinced...
>>>>> Does any of you have a suggestion for NumPy folks about how to improve
>>>>> this?
>>>>>
>>>>
>>>> I'm not sure this should be fixed in NumPy. Their entire point is that
>>>> people shouldn't use those attributes directly. I think numpy.pxd
>>>> should be fixed, but the problem is that some attributes might be used
>>>> in user code (especially shape), and we still want that to work in
>>>> nogil mode. As such, I'm not sure what the best way of fixing it is,
>>>> without special casing these attributes in the compiler directly.
>>>> Maybe Dag will have some thoughts about this.
>>>
>>>
>>>
>>> Well, we should definitely deprecate direct access to the PyArrayObject
>>> fields -- you can either use "cdef int[:]", or, if you use "cdef
>>> np.ndarray[int]", you should use "PyArray_SHAPE".
>>
>>
>> Yeah. However, PyArray_SHAPE seems to be new in numpy 1.7. I also see
>> PyArray_BASE and PyArray_DESCR are commented out (because apparently
>> they may be NULL. Should we, for the sake of consistency, rename
>> 'get_array_base' to PyArray_BASE in numpy.pxd?
>>
>> And maybe we could provide our own implementation of PyArray_SHAPE,
>> which would be portable across numpy versions.
>
>
> PyArray_DIMS

Ah, neat :)

> Dag
>
> _______________________________________________
> cython-devel mailing list
> cython-devel at python.org
> http://mail.python.org/mailman/listinfo/cython-devel


More information about the cython-devel mailing list