[Numpy-discussion] tighten up ufunc casting rule

[Mark Wiebe]

On Mon, Jun 6, 2011 at 10:30 AM, Mark Wiebe <mwwiebe at gmail.com> wrote:

> On Sun, Jun 5, 2011 at 3:43 PM, Ralf Gommers <ralf.gommers at googlemail.com>wrote:
>
>> On Thu, Jun 2, 2011 at 10:12 PM, Mark Wiebe <mwwiebe at gmail.com> wrote:
>>
>>> On Thu, Jun 2, 2011 at 3:09 PM, Gael Varoquaux <
>>> gael.varoquaux at normalesup.org> wrote:
>>>
>>>> On Thu, Jun 02, 2011 at 03:06:58PM -0500, Mark Wiebe wrote:
>>>> >    Would anyone object to, at least temporarily, tightening up the
>>>> default
>>>> >    ufunc casting rule to 'same_kind' in NumPy master? It's a one line
>>>> change,
>>>> >    so would be easy to undo, but such a change is very desirable in my
>>>> >    opinion.
>>>> >    This would raise an exception, since it's np.add(a, 1.9, out=a),
>>>> >    converting a float to an int:
>>>>
>>>> >    >>> a = np.arange(3, dtype=np.int32)
>>>>
>>>> >    >>> a += 1.9
>>>>
>>>> That's probably going to break a huge amount of code which relies on the
>>>> current behavior.
>>>>
>>>> Am I right in believing that this should only be considered for a major
>>>> release of numpy, say numpy 2.0?
>>>
>>>
>>> Absolutely, and that's why I'm proposing to do it in master now, fairly
>>> early in a development cycle, so we can evaluate its effects. If the next
>>> version is 1.7, we probably would roll it back for release (a 1 line
>>> change), and if the next version is 2.0, we probably would keep it in.
>>>
>>> I suspect at least some of the code relying on the current behavior may
>>> have bugs, and tightening this up is a way to reveal them.
>>>
>>>
>> Here are some results of testing your tighten_casting branch on a few
>> projects - no need to first put it in master first to do that. Four failures
>> in numpy, two in scipy, four in scikit-learn (plus two that don't look
>> related), none in scikits.statsmodels. I didn't check how many of them are
>> actual bugs.
>>
>> I'm not against trying out your change, but it would probably be good to
>> do some more testing first and fix the issues found before putting it in.
>> Then at least if people run into issues with the already tested packages,
>> you can just tell them to update those to latest master.
>>
>
> Cool, thanks for running those. I already took a chunk out of the NumPy
> failures. The ones_like function shouldn't really be a ufunc, but rather be
> like zeros_like and empty_like, but that's probably not something to change
> right now. The datetime-fixes type resolution change provides a mechanism to
> fix that up pretty easily.
>
> For Scipy, what do you think is the best way to resolve it? If NumPy 1.6 is
> the minimum version for the next scipy, I would add casting='unsafe' to the
> failing sqrt call.
>

I've updated the tighten_casting branch so it now passes all tests. For
masked arrays, this required changing some tests to not assume float -> int
casts are fine by default, but otherwise I fixed things by relaxing the
rules just where necessary. It now depends on the datetime-fixes branch,
which I would like to merge at its current point.

-Mark
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20110606/fe26685d/attachment.html>