The discussion below relates to porting scipy to numarray, starting with some small changes to scipy_test.testing. The discussion began as private e-mail but has crossed over to scipy-dev at Pearu's suggestion. On Tue, 2004-12-07 at 16:16, Pearu Peterson wrote:
Hi Todd,
On 7 Dec 2004, Todd Miller wrote:
I took a closer look at how my scipy_base changes affect scipy.test() and found a couple things in my scipy_base changes that needed fixing. Sorry I didn't think of this sooner. These fixes are committed now on the numarray branch in CVS. I also honed in on what I think should be the starting point for discussion: the changes I made to scipy_test/testing.py.
Note that assert_equal, assert_almost_equal, assert_approx_equal were not meant to be used with array arguments (I didn't implement them but its obvious from reading the code).
Thanks for pointing this out... I noticed the array versions only peripherally and didn't understand the distinction.
For checking the equality of array arguments assert_array_equal or assert_array_almost_equal should be used. If some scipy test suite uses assert_equal, etc with array arguments then I think this is a bug of this particular test suite, not of testing.py. So, using scipy_base.all in assert_equal, etc is not necessary (unless we want to drop assert_array_* functions).
Understood. Are we agreed that it is appropriate to use all() in the assert_array_* functions?
In order to do the numarray scipy_base changes and pass scipy_base.test(), I modified scipy_test/testing.py in ways which I assert are good. Mostly, I used all() in contexts where an array result was being used as a truth value. In those contexts, the array __nonzero__() function is executed. NumArray.__nonzero__() raises an exception causing many tests to fail that should succeed. Using all() reduces the "truth array" to a single scalar value, so NumArray.__nonzero__() is never called.
Where I think early agreement needs to happen is that my testing.py changes are good even though they expose a handful of latent scipy bugs or unit test bugs because Numeric's __nonzero__() has the meaning of any() and not all(). So, for say an equality test, if any of the values were equal, the test would succeed even if most of the arrays' values were *not* equal. Switching to all() as I am advocating exposes hidden problems like these.
At the moment I have no comments on this as I haven't tried the numarray branch of scipy yet.
I think we should square away the scipy_test.testing changes before anyone messes with the numarray branch.
Btw, your discussion about the changes to scipy sounds reasonable. However, why do you say that using alltrue is an error? I thought all(x) is equivalent to alltrue(ravel(x)) and in assert_array_* functions the arguments to alltrue are already ravel'ed.
I looked at my diffs more carefully and see that you're right, the alltrues are not bugs because they're only fed 1D arrays. I have not reexamined all my scipy_base changes for instances of alltrue.
If my testing.py changes are agreed upon and the exposed bugs are either fixed or acknowledged as known, we have a better basis for examining the rest of the numarray changes to scipy_base.
I attached 3 files for you to look at:
1. testing.diffs (changes to -r HEAD of scipy_test/testing.py)
2. results.HEAD (scipy.test() results against the HEAD of CVS using Numeric) I got 4 failures and can't see the nightly scoreboard yet so I don't know if these are expected.
3. results.testing.changes (scipy.test() results with testing.diffs applied) I got 11 failures (including the original 4) most of which I believe are either bugs or unit test bugs.
The failures, where you see almost equal arrays, may occur when using Fortran blas;
My scipy.test() does complain about not finding clapack so maybe I am using a Fortran blas too.
these errors should be fixed by using proper decimal argument in assert_array_almost_equal call of the corresponding tests (I thought I have fixed these errors in scipy HEAD already).
As you said I think, the (incorrect) changes to assert_equal exposed (incorrect) uses of assert_equal in array contexts.
Other failures need more attention.
If any of you would rather not be included in my future numarray mailings, or you have suggestions for a better venue for them, please let me know.
I suggest using scipy-dev.
Thanks, Pearu
I re-attached the attachments in case anyone on scipy-dev wants to look as well; there's nothing new there. Thanks for looking it over, Todd