The discussion below relates to porting scipy to numarray, starting with
some small changes to scipy_test.testing. The discussion began as
private e-mail but has crossed over to scipy-dev at Pearu's suggestion.
On Tue, 2004-12-07 at 16:16, Pearu Peterson wrote:
> Hi Todd,
>
> On 7 Dec 2004, Todd Miller wrote:
>
> > I took a closer look at how my scipy_base changes affect scipy.test()
> > and found a couple things in my scipy_base changes that needed fixing.
> > Sorry I didn't think of this sooner. These fixes are committed now on
> > the numarray branch in CVS. I also honed in on what I think should be
> > the starting point for discussion: the changes I made to
> > scipy_test/testing.py.
>
> Note that assert_equal, assert_almost_equal, assert_approx_equal
> were not meant to be used with array arguments (I didn't implement them
> but its obvious from reading the code).
Thanks for pointing this out... I noticed the array versions only
peripherally and didn't understand the distinction.
> For checking the equality of
> array arguments assert_array_equal or assert_array_almost_equal should be
> used. If some scipy test suite uses assert_equal, etc with array
> arguments then I think this is a bug of this particular test suite,
> not of testing.py. So, using scipy_base.all in assert_equal, etc is not
> necessary (unless we want to drop assert_array_* functions).
Understood. Are we agreed that it is appropriate to use all() in the
assert_array_* functions?
> > In order to do the numarray scipy_base changes and pass
> > scipy_base.test(), I modified scipy_test/testing.py in ways which I
> > assert are good. Mostly, I used all() in contexts where an array result
> > was being used as a truth value. In those contexts, the array
> > __nonzero__() function is executed. NumArray.__nonzero__() raises an
> > exception causing many tests to fail that should succeed. Using all()
> > reduces the "truth array" to a single scalar value, so
> > NumArray.__nonzero__() is never called.
> >
> > Where I think early agreement needs to happen is that my testing.py
> > changes are good even though they expose a handful of latent scipy bugs
> > or unit test bugs because Numeric's __nonzero__() has the meaning of
> > any() and not all(). So, for say an equality test, if any of the
> > values were equal, the test would succeed even if most of the arrays'
> > values were *not* equal. Switching to all() as I am advocating exposes
> > hidden problems like these.
>
> At the moment I have no comments on this as I haven't tried the numarray
> branch of scipy yet.
I think we should square away the scipy_test.testing changes before
anyone messes with the numarray branch.
> Btw, your discussion about the changes to scipy
> sounds reasonable. However, why do you say that using alltrue is an error?
> I thought all(x) is equivalent to alltrue(ravel(x)) and in assert_array_*
> functions the arguments to alltrue are already ravel'ed.
I looked at my diffs more carefully and see that you're right, the
alltrues are not bugs because they're only fed 1D arrays. I have not
reexamined all my scipy_base changes for instances of alltrue.
> > If my testing.py changes are agreed upon and the exposed bugs are either
> > fixed or acknowledged as known, we have a better basis for examining
> > the rest of the numarray changes to scipy_base.
> >
> > I attached 3 files for you to look at:
> >
> > 1. testing.diffs (changes to -r HEAD of scipy_test/testing.py)
> >
> > 2. results.HEAD (scipy.test() results against the HEAD of CVS using
> > Numeric)
> > I got 4 failures and can't see the nightly scoreboard yet so I don't
> > know if these are expected.
> >
> > 3. results.testing.changes (scipy.test() results with testing.diffs
> > applied)
> > I got 11 failures (including the original 4) most of which I believe are
> > either bugs or unit test bugs.
>
> The failures, where you see almost equal arrays, may occur when using
> Fortran blas;
My scipy.test() does complain about not finding clapack so maybe I am
using a Fortran blas too.
> these errors should be fixed by using proper decimal
> argument in assert_array_almost_equal call of the corresponding tests (I
> thought I have fixed these errors in scipy HEAD already).
As you said I think, the (incorrect) changes to assert_equal exposed
(incorrect) uses of assert_equal in array contexts.
> Other failures need more attention.
>
> > If any of you would rather not be included in my future numarray
> > mailings, or you have suggestions for a better venue for them, please
> > let me know.
>
> I suggest using scipy-dev.
>
> Thanks,
> Pearu
I re-attached the attachments in case anyone on scipy-dev wants to look
as well; there's nothing new there.
Thanks for looking it over,
Todd