A week or so ago we were discussing the set of changes to scipy's testing framework needed to add support for numarray. The changes I'm talking about here should be made to the main trunk of scipy CVS. After further review, I now think the array versions of the assert family of functions (e.g. assert_array_equal) are correct with respect to truth value testing. So, the only change I think is needed is the addition of "delegation code" so that calls to assert_equal, etc. defer to assert_array_equal, etc. when passed array parameters. Here's the patch: Index: testing.py =================================================================== RCS file: /home/cvsroot/world/scipy_core/scipy_test/testing.py,v retrieving revision 1.49 retrieving revision 1.49.2.3 diff -c -r1.49 -r1.49.2.3 *** testing.py 1 Dec 2004 07:08:51 -0000 1.49 --- testing.py 17 Dec 2004 18:20:28 -0000 1.49.2.3 *************** *** 634,639 **** --- 633,640 ---- """ Raise an assertion if two items are not equal. I think this should be part of unittest.py """ + if isinstance(actual, ArrayType): + return assert_array_equal(actual, desired, err_msg) msg = '\nItems are not equal:\n' + err_msg try: if ( verbose and len(repr(desired)) < 100 and len(repr(actual)) ): *************** *** 651,656 **** --- 652,659 ---- """ Raise an assertion if two items are not equal. I think this should be part of unittest.py """ + if isinstance(actual, ArrayType): + return assert_array_almost_equal(actual, desired, decimal, err_msg) msg = '\nItems are not equal:\n' + err_msg try: if ( verbose and len(repr(desired)) < 100 and len(repr(actual)) ): These changes are necessary for adding numarray support to scipy because without them there are 12 testing failures (in scipy_base.test()). The failures are all related to numarray's "outlawed" __nonzero__() and testers apparently calling the wrong assertion function. With these changes, 7 of the numarray failures go away and 5 identical failures remain for both numarray and Numeric. I'm arguing that the 5 remaining failures are either real problems or testing bugs which were masked by testers calling the wrong assert functions and by Numeric's sometrue() definition of __nonzero__(). Here are the test failures I see: ====================================================================== FAIL: check_basic (scipy_base.function_base.test_function_base.test_amax) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/jmiller/work/lib/python2.4/site-packages/scipy_base/tests/test_function_base.py", line 72, in check_basic assert_equal(amax(b),[8.0,10.0,9.0]) File "/home/jmiller/work/lib/python2.4/site-packages/scipy_test/testing.py", line 638, in assert_equal return assert_array_equal(actual, desired, err_msg) File "/home/jmiller/work/lib/python2.4/site-packages/scipy_test/testing.py", line 721, in assert_array_equal assert cond,\ AssertionError: Arrays are not equal (mismatch 66.6666666667%): Array 1: [ 9. 10. 8.] Array 2: [ 8. 10. 9.] ====================================================================== FAIL: check_basic (scipy_base.function_base.test_function_base.test_amin) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/jmiller/work/lib/python2.4/site-packages/scipy_base/tests/test_function_base.py", line 82, in check_basic assert_equal(amin(b),[3.0,3.0,2.0]) File "/home/jmiller/work/lib/python2.4/site-packages/scipy_test/testing.py", line 638, in assert_equal return assert_array_equal(actual, desired, err_msg) File "/home/jmiller/work/lib/python2.4/site-packages/scipy_test/testing.py", line 721, in assert_array_equal assert cond,\ AssertionError: Arrays are not equal (mismatch 33.3333333333%): Array 1: [ 3. 4. 2.] Array 2: [ 3. 3. 2.] ====================================================================== FAIL: check_arange (scipy.special.basic.test_basic.test_arange) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/jmiller/work/lib/python2.4/site-packages/scipy/special/tests/test_basic.py", line 495, in check_arange assert_equal(numstring,array([0.,0.1,0.2,0.3, File "/home/jmiller/work/lib/python2.4/site-packages/scipy_test/testing.py", line 638, in assert_equal return assert_array_equal(actual, desired, err_msg) File "/home/jmiller/work/lib/python2.4/site-packages/scipy_test/testing.py", line 721, in assert_array_equal assert cond,\ AssertionError: Arrays are not equal (mismatch 30.4347826087%): Array 1: [ 0. 0.1 0.2 0.3 0.4 0.5 ... Array 2: [ 0. 0.1 0.2 0.3 0.4 0.5 ... ====================================================================== FAIL: check_genlaguerre (scipy.special.basic.test_basic.test_laguerre) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/jmiller/work/lib/python2.4/site-packages/scipy/special/tests/test_basic.py", line 1573, in check_genlaguerre assert_equal(lag2.c,array([1,-2*(k+2),(k+1.)*(k+2.)])/2.0) File "/home/jmiller/work/lib/python2.4/site-packages/scipy_test/testing.py", line 638, in assert_equal return assert_array_equal(actual, desired, err_msg) File "/home/jmiller/work/lib/python2.4/site-packages/scipy_test/testing.py", line 721, in assert_array_equal assert cond,\ AssertionError: Arrays are not equal (mismatch 66.6666666667%): Array 1: [ 0.5 -2.5842644468705798 2.0470791422443622] Array 2: [ 0.5 -2.5842644468705802 2.047079142244363 ] ====================================================================== FAIL: check_legendre (scipy.special.basic.test_basic.test_legendre) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/jmiller/work/lib/python2.4/site-packages/scipy/special/tests/test_basic.py", line 1590, in check_legendre assert_equal(leg3.c,array([5,0,-3,0])/2.0) File "/home/jmiller/work/lib/python2.4/site-packages/scipy_test/testing.py", line 638, in assert_equal return assert_array_equal(actual, desired, err_msg) File "/home/jmiller/work/lib/python2.4/site-packages/scipy_test/testing.py", line 721, in assert_array_equal assert cond,\ AssertionError: Arrays are not equal (mismatch 75.0%): Array 1: [ 2.5000000000000000e+00 -8.3266726846886741e-16 -1.4999999999999991e+00 1.1340467527089692e-17] Array 2: [ 2.5 0. -1.5 0. ] ---------------------------------------------------------------------- Ran 690 tests in 3.949s FAILED (failures=5) In each case you can see that assert_array_equal has been called from the new delegation code in assert_equal. At this point, I guess I have two questions: 1. Is this patch acceptable for the main trunk now? 2. If so, who should fix the test failures? Regards, Todd