[SciPy-dev] scipy testing changes revisited
Todd Miller
jmiller at stsci.edu
Wed Dec 22 17:25:25 EST 2004
A week or so ago we were discussing the set of changes to scipy's
testing framework needed to add support for numarray. The changes I'm
talking about here should be made to the main trunk of scipy CVS.
After further review, I now think the array versions of the assert
family of functions (e.g. assert_array_equal) are correct with respect
to truth value testing. So, the only change I think is needed is the
addition of "delegation code" so that calls to assert_equal, etc. defer
to assert_array_equal, etc. when passed array parameters. Here's the
patch:
Index: testing.py
===================================================================
RCS file: /home/cvsroot/world/scipy_core/scipy_test/testing.py,v
retrieving revision 1.49
retrieving revision 1.49.2.3
diff -c -r1.49 -r1.49.2.3
*** testing.py 1 Dec 2004 07:08:51 -0000 1.49
--- testing.py 17 Dec 2004 18:20:28 -0000 1.49.2.3
***************
*** 634,639 ****
--- 633,640 ----
""" Raise an assertion if two items are not
equal. I think this should be part of unittest.py
"""
+ if isinstance(actual, ArrayType):
+ return assert_array_equal(actual, desired, err_msg)
msg = '\nItems are not equal:\n' + err_msg
try:
if ( verbose and len(repr(desired)) < 100 and len(repr(actual)) ):
***************
*** 651,656 ****
--- 652,659 ----
""" Raise an assertion if two items are not
equal. I think this should be part of unittest.py
"""
+ if isinstance(actual, ArrayType):
+ return assert_array_almost_equal(actual, desired, decimal, err_msg)
msg = '\nItems are not equal:\n' + err_msg
try:
if ( verbose and len(repr(desired)) < 100 and len(repr(actual)) ):
These changes are necessary for adding numarray support to scipy because
without them there are 12 testing failures (in scipy_base.test()). The
failures are all related to numarray's "outlawed" __nonzero__() and
testers apparently calling the wrong assertion function. With these
changes, 7 of the numarray failures go away and 5 identical failures
remain for both numarray and Numeric. I'm arguing that the 5 remaining
failures are either real problems or testing bugs which were masked by
testers calling the wrong assert functions and by Numeric's sometrue()
definition of __nonzero__().
Here are the test failures I see:
======================================================================
FAIL: check_basic
(scipy_base.function_base.test_function_base.test_amax)
----------------------------------------------------------------------
Traceback (most recent call last):
File
"/home/jmiller/work/lib/python2.4/site-packages/scipy_base/tests/test_function_base.py", line 72, in check_basic
assert_equal(amax(b),[8.0,10.0,9.0])
File
"/home/jmiller/work/lib/python2.4/site-packages/scipy_test/testing.py",
line 638, in assert_equal
return assert_array_equal(actual, desired, err_msg)
File
"/home/jmiller/work/lib/python2.4/site-packages/scipy_test/testing.py",
line 721, in assert_array_equal
assert cond,\
AssertionError:
Arrays are not equal (mismatch 66.6666666667%):
Array 1: [ 9. 10. 8.]
Array 2: [ 8. 10. 9.]
======================================================================
FAIL: check_basic
(scipy_base.function_base.test_function_base.test_amin)
----------------------------------------------------------------------
Traceback (most recent call last):
File
"/home/jmiller/work/lib/python2.4/site-packages/scipy_base/tests/test_function_base.py", line 82, in check_basic
assert_equal(amin(b),[3.0,3.0,2.0])
File
"/home/jmiller/work/lib/python2.4/site-packages/scipy_test/testing.py",
line 638, in assert_equal
return assert_array_equal(actual, desired, err_msg)
File
"/home/jmiller/work/lib/python2.4/site-packages/scipy_test/testing.py",
line 721, in assert_array_equal
assert cond,\
AssertionError:
Arrays are not equal (mismatch 33.3333333333%):
Array 1: [ 3. 4. 2.]
Array 2: [ 3. 3. 2.]
======================================================================
FAIL: check_arange (scipy.special.basic.test_basic.test_arange)
----------------------------------------------------------------------
Traceback (most recent call last):
File
"/home/jmiller/work/lib/python2.4/site-packages/scipy/special/tests/test_basic.py", line 495, in check_arange
assert_equal(numstring,array([0.,0.1,0.2,0.3,
File
"/home/jmiller/work/lib/python2.4/site-packages/scipy_test/testing.py",
line 638, in assert_equal
return assert_array_equal(actual, desired, err_msg)
File
"/home/jmiller/work/lib/python2.4/site-packages/scipy_test/testing.py",
line 721, in assert_array_equal
assert cond,\
AssertionError:
Arrays are not equal (mismatch 30.4347826087%):
Array 1: [ 0. 0.1 0.2
0.3 0.4 0.5 ...
Array 2: [ 0. 0.1 0.2
0.3 0.4 0.5 ...
======================================================================
FAIL: check_genlaguerre (scipy.special.basic.test_basic.test_laguerre)
----------------------------------------------------------------------
Traceback (most recent call last):
File
"/home/jmiller/work/lib/python2.4/site-packages/scipy/special/tests/test_basic.py", line 1573, in check_genlaguerre
assert_equal(lag2.c,array([1,-2*(k+2),(k+1.)*(k+2.)])/2.0)
File
"/home/jmiller/work/lib/python2.4/site-packages/scipy_test/testing.py",
line 638, in assert_equal
return assert_array_equal(actual, desired, err_msg)
File
"/home/jmiller/work/lib/python2.4/site-packages/scipy_test/testing.py",
line 721, in assert_array_equal
assert cond,\
AssertionError:
Arrays are not equal (mismatch 66.6666666667%):
Array 1: [ 0.5 -2.5842644468705798 2.0470791422443622]
Array 2: [ 0.5 -2.5842644468705802 2.047079142244363 ]
======================================================================
FAIL: check_legendre (scipy.special.basic.test_basic.test_legendre)
----------------------------------------------------------------------
Traceback (most recent call last):
File
"/home/jmiller/work/lib/python2.4/site-packages/scipy/special/tests/test_basic.py", line 1590, in check_legendre
assert_equal(leg3.c,array([5,0,-3,0])/2.0)
File
"/home/jmiller/work/lib/python2.4/site-packages/scipy_test/testing.py",
line 638, in assert_equal
return assert_array_equal(actual, desired, err_msg)
File
"/home/jmiller/work/lib/python2.4/site-packages/scipy_test/testing.py",
line 721, in assert_array_equal
assert cond,\
AssertionError:
Arrays are not equal (mismatch 75.0%):
Array 1: [ 2.5000000000000000e+00 -8.3266726846886741e-16
-1.4999999999999991e+00
1.1340467527089692e-17]
Array 2: [ 2.5 0. -1.5 0. ]
----------------------------------------------------------------------
Ran 690 tests in 3.949s
FAILED (failures=5)
In each case you can see that assert_array_equal has been called from
the new delegation code in assert_equal.
At this point, I guess I have two questions:
1. Is this patch acceptable for the main trunk now?
2. If so, who should fix the test failures?
Regards,
Todd
-------------- next part --------------
A non-text attachment was scrubbed...
Name: testing.patch
Type: text/x-patch
Size: 1164 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20041222/4357f2c2/attachment.bin>
More information about the SciPy-Dev
mailing list