[Numpy-discussion] nose test return values

[Christopher Hanley]

Hi,

I have received the following message from our system guru here at STScI 
regarding the recent changes to the way testing is done with nose.  The 
automated scripts he was using to monitor the success or failure of 
numpy builds is now broken.  Below is his message:

The new numpy test interface returns None instead of 
unittest._TextTestResult

Even if you don't want to return unittest objects, the test should 
return some indication of which tests failed, which could not be run due 
to errors (a test that failed is different from a test that couldn't 
even run), and whether the test suite as a whole can be considered passing.

For my purposes, the wasSuccessful() method of the returned object is 
absolutely essential.  If wasSuccessful() returns true, I am not going 
to look at the numpy tests; if it returns false, somebody has to find 
which tests failed and decide whether that is going to affect the other 
developers.



OLD:

 >>> n=numpy.test(level=10)
...lots out output...
 >>> n
<unittest._TextTestResult run=693 errors=0 failures=0>
 >>> n.errors
[]
 >>> n.failures
[]
 >>> n.wasSuccessful()
True
 >>>



NEW:

 >>> n = numpy.test(label="full", verbose=0)
 >>> n
 >>> print n
None
 >>> numpy.__version__
'1.2.0.dev5292'



My application:

import sys
import numpy

n = numpy.test(level=10)

print "errors: "
for x in n.errors:
    print " ",x

print "failures: "
for x in n.failures:
    print " ",x

if n.wasSuccessful() :
    sys.exit(0)
else :
    sys.exit(1)



Of course, these same comments will apply to scipy when I next install it.

--

I believe that he makes a very good point.  Is there any way that some 
form of test report object can be returned?

Thanks,
Chris



-- 
Christopher Hanley
Systems Software Engineer
Space Telescope Science Institute
3700 San Martin Drive
Baltimore MD, 21218
(410) 338-4338