[Numpy-discussion] tighten up ufunc casting rule

[Ralf Gommers]

On Mon, Jun 6, 2011 at 6:56 PM, Mark Wiebe <mwwiebe at gmail.com> wrote:

> On Mon, Jun 6, 2011 at 10:30 AM, Mark Wiebe <mwwiebe at gmail.com> wrote:
>
>> On Sun, Jun 5, 2011 at 3:43 PM, Ralf Gommers <ralf.gommers at googlemail.com
>> > wrote:
>>
>>> On Thu, Jun 2, 2011 at 10:12 PM, Mark Wiebe <mwwiebe at gmail.com> wrote:
>>>
>>>> On Thu, Jun 2, 2011 at 3:09 PM, Gael Varoquaux <
>>>> gael.varoquaux at normalesup.org> wrote:
>>>>
>>>>> On Thu, Jun 02, 2011 at 03:06:58PM -0500, Mark Wiebe wrote:
>>>>> >    Would anyone object to, at least temporarily, tightening up the
>>>>> default
>>>>> >    ufunc casting rule to 'same_kind' in NumPy master? It's a one line
>>>>> change,
>>>>> >    so would be easy to undo, but such a change is very desirable in
>>>>> my
>>>>> >    opinion.
>>>>> >    This would raise an exception, since it's np.add(a, 1.9, out=a),
>>>>> >    converting a float to an int:
>>>>>
>>>>> >    >>> a = np.arange(3, dtype=np.int32)
>>>>>
>>>>> >    >>> a += 1.9
>>>>>
>>>>> That's probably going to break a huge amount of code which relies on
>>>>> the
>>>>> current behavior.
>>>>>
>>>>> Am I right in believing that this should only be considered for a major
>>>>> release of numpy, say numpy 2.0?
>>>>
>>>>
>>>> Absolutely, and that's why I'm proposing to do it in master now, fairly
>>>> early in a development cycle, so we can evaluate its effects. If the next
>>>> version is 1.7, we probably would roll it back for release (a 1 line
>>>> change), and if the next version is 2.0, we probably would keep it in.
>>>>
>>>> I suspect at least some of the code relying on the current behavior may
>>>> have bugs, and tightening this up is a way to reveal them.
>>>>
>>>>
>>> Here are some results of testing your tighten_casting branch on a few
>>> projects - no need to first put it in master first to do that. Four failures
>>> in numpy, two in scipy, four in scikit-learn (plus two that don't look
>>> related), none in scikits.statsmodels. I didn't check how many of them are
>>> actual bugs.
>>>
>>> I'm not against trying out your change, but it would probably be good to
>>> do some more testing first and fix the issues found before putting it in.
>>> Then at least if people run into issues with the already tested packages,
>>> you can just tell them to update those to latest master.
>>>
>>
>> Cool, thanks for running those. I already took a chunk out of the NumPy
>> failures. The ones_like function shouldn't really be a ufunc, but rather be
>> like zeros_like and empty_like, but that's probably not something to change
>> right now. The datetime-fixes type resolution change provides a mechanism to
>> fix that up pretty easily.
>>
>> For Scipy, what do you think is the best way to resolve it? If NumPy 1.6
>> is the minimum version for the next scipy, I would add casting='unsafe' to
>> the failing sqrt call.
>>
>
>
There's no reason to set the minimum required numpy to 1.6 AFAIK, and it's
definitely not desirable.

Ralf


> I've updated the tighten_casting branch so it now passes all tests. For
> masked arrays, this required changing some tests to not assume float -> int
> casts are fine by default, but otherwise I fixed things by relaxing the
> rules just where necessary. It now depends on the datetime-fixes branch,
> which I would like to merge at its current point.
>
> -Mark
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20110607/96611179/attachment.html>