[Numpy-discussion] Numpy 1.7b1 API change cause big trouble

[Charles R Harris]

On Sun, Sep 9, 2012 at 1:42 PM, Charles R Harris
<charlesr.harris at gmail.com>wrote:

>
>
> On Sun, Sep 9, 2012 at 11:12 AM, Frédéric Bastien <nouiz at nouiz.org> wrote:
>
>> Hi,
>>
>> On Thu, Sep 6, 2012 at 11:32 AM, Charles R Harris
>> <charlesr.harris at gmail.com> wrote:
>> >
>> >
>> > On Thu, Sep 6, 2012 at 10:07 AM, Frédéric Bastien <nouiz at nouiz.org>
>> wrote:
>> >>
>> >> Hi,
>> >>
>> >> I reply with more information probably later today or tomorrow, but I
>> >> think i need to finish everything to give you the exact information.
>> >>
>> >> Part of the problem I had was that by default there is a warning that
>> >> is generated. It tell that to remove this warning we need to set
>> >> NPY_NO_DEPRECATED_API=NPY_1_7_API_VERSION.
>> >
>> >
>> > You don't want to define this macro if you need to directly access the
>> > fields. What warning are you getting if you don't define it? Are you
>> using
>> > Cython?
>>
>> If I don't define it and I remove the -Werror, I got 3 errors. 1 is
>> related to an error message that was changed.
>>
>> The second was that we called numpy.dot() with 2 sparse matrix(from
>> scipy). It worked in the past, but not now. Changing the test is easy.
>> I don't expect people to have done this frequently, but maybe warning
>> about this in the release note would help people to fix it faster. The
>> error message is not helpful, it tell that it can't find a common
>> dtype between float32 and float32 dtype. I changed the np.dot(a,b) to
>> a*b as this is the matrix multiplication function for sparse matrix in
>> scipy. This change remove the possibility to make a function that use
>> matrix product to work with both ndarray and sparse matrix without
>> special case for the object type. Not great, but there is an easy work
>> around. So this stay like this in the release, there should be a
>> warning.
>>
>> The third is releated to change to the casting rules in numpy. Before
>> a scalar complex128 * vector float32 gived a vector of dtype
>> complex128. Now it give a vector of complex64. The reason is that now
>> the scalar of different category only change the category, not the
>> precision. I would consider a must that we warn clearly about this
>> interface change. Most people won't see it, but people that optimize
>> there code heavily could depend on such thing.
>>
>> The other problem I had was related to the fact that I tryed to use
>> only the new API. This took me a few day and it is not finished, as
>> now I have a seg fault that is not easy to trigger. It happen in one
>> tests, but only when other tests a ran before... This is probably an
>> error from my change
>>
>> The sed script that replace some macro helped, but there is few macro
>> change that is not in the file: NPY_ALIGNED to NPY_ARRAY_ALIGNED. idem
>> for NPY_WRITABLE, NPY_UPDATE_ALL, NPY_C_CONTIGUOUS and
>> NPY_F_CONTIGUOUS.
>>
>
> I can add those, they seem to have been present since at least Numpy 1.5.
>
>
>> The sed script change NPY_ENSURECOPY to NPY_ARRAY_ENSURECOPY, but I
>> think that NPY_ARRAY_ENSURECOPY was introduced in numpy 1.7. Maybe
>> warning somewhere in the API transition doc that if people want to
>> stay compatible with older version of numpy, the should use an
>> "#ifndef NPY_ARRAY_ENSURECOPY ..." in there code.
>>
>
> Hmm... Looks like you are right about NPY_ARRAY_ENSURECOPY. An alternative
> would be to not deprecate it, but an #ifndef would be better for the long
> term goal of having everyone use newer macros.
>
>

And the other *_ARRAY_* macros seem to have been defined in 1.5. If 1.7 is
intended to be a long term release, they probably shouldn't be deprecated
until a later release.


>
>> I won't have the time to make a PR with those small change as I have a
>> deadline the 16 september and the 1 october. I hope my comment will be
>> helpful. If you still have questions, don't hesitate.
>>
>>
>
Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20120909/4e8c7b90/attachment.html>