[Numpy-discussion] Warnings in numpy.ma.test()
Bruce Southey
bsouthey at gmail.com
Thu Mar 18 12:03:38 EDT 2010
On 03/17/2010 04:20 PM, Pierre GM wrote:
> On Mar 17, 2010, at 11:09 AM, Bruce Southey wrote:
>
>> On 03/17/2010 01:07 AM, Pierre GM wrote:
>>
>>> All,
>>> As you're probably aware, the current test suite for numpy.ma raises some nagging warnings such as "invalid value in ...". These warnings are only issued when a standard numpy ufunc (eg., np.sqrt) is called on a MaskedArray, instead of its numpy.ma (eg., np.ma.sqrt) equivalent. The reason is that the masked versions of the ufuncs temporarily set the numpy error status to 'ignore' before the operation takes place, and reset the status to its original value.
>>>
>>>
>> Perhaps naive question, what is really being tested here?
>>
>> That is, it appears that you are testing both the generation of the
>> invalid values and function. So if the generation fails, then the
>> function will also fail. However, the test for the generation of invalid
>> values should be elsewhere so you have to assume that the generation of
>> values will work correctly.
>>
> That's not really the point here. The issue is that when numpy ufuncs are called on a MaskedArray, a warning or an exception is raised when an invalid is met. With the numpy.ma version of those functions, the error is trapped and processed. Of course, using the numpy.ma version of the ufuncs is the right way to go
>
>
>
>> I think that you should be only testing that the specific function
>> passes the test. Why not just use 'invalid' values like np.inf directly?
>>
>> For example, in numpy/ma/tests/test_core.py
>> We have this test:
>> def test_fix_invalid(self):
>> "Checks fix_invalid."
>> data = masked_array(np.sqrt([-1., 0., 1.]), mask=[0, 0, 1])
>> data_fixed = fix_invalid(data)
>>
>> If that is to test that fix_invalid Why not create the data array as:
>> data = masked_array([np.inf, 0., 1.]), mask=[0, 0, 1])
>>
> Sure, that's nicer. But once again, that's not really the core of the issue.
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
I needed to point out that your statement was not completely correct:
'These warnings are only issued when a standard numpy ufunc (eg.,
np.sqrt) is called on a MaskedArray...'.
There are valid warnings for some of the tests because these are do not
operate on masked arrays. As in the above example, the masked array only
occurs *after* the square root has been taken and hence the warning. So
some of the warnings in the tests should be eliminated just by changing
the test input.
Furthermore, this warning is technically valid for non-masked values:
>>> np.sqrt(np.ma.array([-1], mask=[0]))
Warning: invalid value encountered in sqrt
masked_array(data = [--],
mask = [ True],
fill_value = 1e+20)
In contrast not warning is emitted with ma function:
>>> np.ma.sqrt(np.ma.array([-1], mask=[0]))
masked_array(data = [--],
mask = [ True],
fill_value = 1e+20)
But I fully agree that there is a problem when the value is masked
because it should have ignored the operation:
>>> np.sqrt(np.ma.array([-1], mask=[1]))
Warning: invalid value encountered in sqrt
masked_array(data = [--],
mask = [ True],
fill_value = 1e+20)
Here I understand your view is that if the operation is on a masked
array then all 'invalid' operations like square root then these should
be silently converted to masked values.
Bruce
More information about the NumPy-Discussion
mailing list