[Numpy-discussion] tighten up ufunc casting rule

[Mark Wiebe]

On Tue, Jun 7, 2011 at 2:08 PM, Ralf Gommers <ralf.gommers at googlemail.com>wrote:

> On Tue, Jun 7, 2011 at 8:59 PM, Mark Wiebe <mwwiebe at gmail.com> wrote:
>
>> On Tue, Jun 7, 2011 at 1:41 PM, Ralf Gommers <ralf.gommers at googlemail.com
>> > wrote:
>>
>>> On Mon, Jun 6, 2011 at 6:56 PM, Mark Wiebe <mwwiebe at gmail.com> wrote:
>>>
>>>> On Mon, Jun 6, 2011 at 10:30 AM, Mark Wiebe <mwwiebe at gmail.com> wrote:
>>>>
>>>>> On Sun, Jun 5, 2011 at 3:43 PM, Ralf Gommers <
>>>>> ralf.gommers at googlemail.com> wrote:
>>>>>
>>>>>> On Thu, Jun 2, 2011 at 10:12 PM, Mark Wiebe <mwwiebe at gmail.com>wrote:
>>>>>>
>>>>>>> On Thu, Jun 2, 2011 at 3:09 PM, Gael Varoquaux <
>>>>>>> gael.varoquaux at normalesup.org> wrote:
>>>>>>>
>>>>>>>> On Thu, Jun 02, 2011 at 03:06:58PM -0500, Mark Wiebe wrote:
>>>>>>>> >    Would anyone object to, at least temporarily, tightening up the
>>>>>>>> default
>>>>>>>> >    ufunc casting rule to 'same_kind' in NumPy master? It's a one
>>>>>>>> line change,
>>>>>>>> >    so would be easy to undo, but such a change is very desirable
>>>>>>>> in my
>>>>>>>> >    opinion.
>>>>>>>> >    This would raise an exception, since it's np.add(a, 1.9,
>>>>>>>> out=a),
>>>>>>>> >    converting a float to an int:
>>>>>>>>
>>>>>>>> >    >>> a = np.arange(3, dtype=np.int32)
>>>>>>>>
>>>>>>>> >    >>> a += 1.9
>>>>>>>>
>>>>>>>> That's probably going to break a huge amount of code which relies on
>>>>>>>> the
>>>>>>>> current behavior.
>>>>>>>>
>>>>>>>> Am I right in believing that this should only be considered for a
>>>>>>>> major
>>>>>>>> release of numpy, say numpy 2.0?
>>>>>>>
>>>>>>>
>>>>>>> Absolutely, and that's why I'm proposing to do it in master now,
>>>>>>> fairly early in a development cycle, so we can evaluate its effects. If the
>>>>>>> next version is 1.7, we probably would roll it back for release (a 1 line
>>>>>>> change), and if the next version is 2.0, we probably would keep it in.
>>>>>>>
>>>>>>> I suspect at least some of the code relying on the current behavior
>>>>>>> may have bugs, and tightening this up is a way to reveal them.
>>>>>>>
>>>>>>>
>>>>>> Here are some results of testing your tighten_casting branch on a few
>>>>>> projects - no need to first put it in master first to do that. Four failures
>>>>>> in numpy, two in scipy, four in scikit-learn (plus two that don't look
>>>>>> related), none in scikits.statsmodels. I didn't check how many of them are
>>>>>> actual bugs.
>>>>>>
>>>>>> I'm not against trying out your change, but it would probably be good
>>>>>> to do some more testing first and fix the issues found before putting it in.
>>>>>> Then at least if people run into issues with the already tested packages,
>>>>>> you can just tell them to update those to latest master.
>>>>>>
>>>>>
>>>>> Cool, thanks for running those. I already took a chunk out of the NumPy
>>>>> failures. The ones_like function shouldn't really be a ufunc, but rather be
>>>>> like zeros_like and empty_like, but that's probably not something to change
>>>>> right now. The datetime-fixes type resolution change provides a mechanism to
>>>>> fix that up pretty easily.
>>>>>
>>>>> For Scipy, what do you think is the best way to resolve it? If NumPy
>>>>> 1.6 is the minimum version for the next scipy, I would add casting='unsafe'
>>>>> to the failing sqrt call.
>>>>>
>>>>
>>>>
>>> There's no reason to set the minimum required numpy to 1.6 AFAIK, and
>>> it's definitely not desirable.
>>>
>>
>> Ok, I think there are two ways to resolve this kind of error. One would be
>> to add a condition testing for a version >= 1.6, and setting
>> casting='unsafe' when that occurs, and the other would be to insert a call
>> to .astype() to force the type. The former is probably preferable, to avoid
>> the unnecessary copy.
>>
>> Does numpy provide the version in tuple form, so a version comparison like
>> this can be done easily? numpy.version doesn't seem to have one in it.
>>
>> No, just as a string. I think it's fine to do:
>
> In [4]: np.version.short_version
> Out[4]: '2.0.0'
> In [5]: np.version.short_version > '1.5.1'
> Out[5]: True
>
> For a very convoluted version that produces a tuple see
> parse_numpy_version() in scipy.pavement.py
>

I've submitted a pull request to scipy which fixes up the ndimage errors, as
well as a netcdf bug. I think it would be a good idea to add something like
np.version.short_version_tuple, at least before we get to numpy 10.0.0 or
1.10.0, where this comparison will break down.

-Mark



> Ralf
>
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20110615/f2e75930/attachment.html>