[Numpy-discussion] How to run NumPy's tests with coverage?

Chris Ball ceball at gmail.com
Sun May 6 15:08:36 EDT 2012


Hi,

I'm trying to figure out how to run NumPy's tests with coverage enabled (i.e. 
numpy.test(coverage=True) ). I can run the tests successfully like this:

$ git clone git://github.com/numpy/numpy.git
[...]
$ cd numpy/
$ python setup.py build_ext -i
[...]
$ cd ..  # (avoid running from source directory)
$ export PYTHONPATH=numpy/
$ python
Python 2.6.5 (r265:79063, Apr 16 2010, 13:57:41) 
[GCC 4.4.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
python> import numpy
python> numpy.test()
Running unit tests for numpy
NumPy version 1.7.0.dev-259fff8
NumPy is installed in [...]/numpy
Python version 2.6.5 (r265:79063, Apr 16 2010, 13:57:41) [GCC 4.4.3]
nose version 0.11.1
[...]
Ran 3710 tests in 27.654s

OK (KNOWNFAIL=3, SKIP=6)
<nose.result.TextTestResult run=3710 errors=0 failures=0>


However, if I try to run the tests with coverage, I get lots of errors (and 
seven more tests are run than without coverage):

python> numpy.test(coverage=True)
Running unit tests for numpy
NumPy version 1.7.0.dev-259fff8
NumPy is installed in [...]/numpy
Python version 2.6.5 (r265:79063, Apr 16 2010, 13:57:41) [GCC 4.4.3]
nose version 0.11.1
Could not locate executable icc
Could not locate executable ecc
[...]/numpy/numarray/alter_code2.py:12: UserWarning: numpy.numarray.alter_code2 
is not working yet.
  warnings.warn("numpy.numarray.alter_code2 is not working yet.")
[...]/numpy/oldnumeric/alter_code2.py:26: UserWarning: 
numpy.oldnumeric.alter_code2 is not working yet.
  warnings.warn("numpy.oldnumeric.alter_code2 is not working yet.")
[...]
======================================================================
ERROR: Failure: ImportError (No module named waflib.Configure)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/pymodules/python2.6/nose/loader.py", line 379, in 
loadTestsFromName
    addr.filename, addr.module)
  File "/usr/lib/pymodules/python2.6/nose/importer.py", line 39, in 
importFromPath
    return self.importFromDir(dir_path, fqname)
  File "/usr/lib/pymodules/python2.6/nose/importer.py", line 86, in 
importFromDir
    mod = load_module(part_fqname, fh, filename, desc)
  File "[...]/numpy/build_utils/waf.py", line 4, in <module>
    import waflib.Configure
ImportError: No module named waflib.Configure

======================================================================
ERROR: Failure: ImportError (No module named numscons.numdist)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/pymodules/python2.6/nose/loader.py", line 379, in 
loadTestsFromName
    addr.filename, addr.module)
  File "/usr/lib/pymodules/python2.6/nose/importer.py", line 39, in 
importFromPath
    return self.importFromDir(dir_path, fqname)
  File "/usr/lib/pymodules/python2.6/nose/importer.py", line 86, in 
importFromDir
    mod = load_module(part_fqname, fh, filename, desc)
  File "[...]/numpy/core/scons_support.py", line 21, in <module>
    from numscons.numdist import process_c_str as process_str
ImportError: No module named numscons.numdist

======================================================================
ERROR: Failure: ImportError (No module named numscons)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/pymodules/python2.6/nose/loader.py", line 379, in 
loadTestsFromName
    addr.filename, addr.module)
  File "/usr/lib/pymodules/python2.6/nose/importer.py", line 39, in 
importFromPath
    return self.importFromDir(dir_path, fqname)
  File "/usr/lib/pymodules/python2.6/nose/importer.py", line 86, in 
importFromDir
    mod = load_module(part_fqname, fh, filename, desc)
  File "[...]/numpy/core/setupscons.py", line 8, in <module>
    from numscons import get_scons_build_dir
ImportError: No module named numscons

======================================================================
ERROR: test_multiarray.TestNewBufferProtocol.test_roundtrip
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/pymodules/python2.6/nose/case.py", line 183, in runTest
    self.test(*self.arg)
  File "[...]/numpy/core/tests/test_multiarray.py", line 2233, in test_roundtrip
    assert_raises(ValueError, self._check_roundtrip, x)
  File "[...]/numpy/testing/utils.py", line 1053, in assert_raises
    return nose.tools.assert_raises(*args,**kwargs)
  File "/usr/lib/python2.6/unittest.py", line 336, in failUnlessRaises
    callableObj(*args, **kwargs)
  File "[...]/numpy/core/tests/test_multiarray.py", line 2167, in 
_check_roundtrip
    y = np.asarray(x)
  File "[...]/numpy/core/tests/test_multiarray.py", line 2167, in 
_check_roundtrip
    y = np.asarray(x)
  File "/usr/lib/python2.6/dist-packages/coverage.py", line 322, in t
    self.c[(f.f_code.co_filename, f.f_lineno)] = 1
RuntimeWarning: tp_compare didn't return -1 or -2 for exception

======================================================================
ERROR: Failure: ImportError (No module named np.core.fromnumeric)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/pymodules/python2.6/nose/loader.py", line 379, in 
loadTestsFromName
    addr.filename, addr.module)
  File "/usr/lib/pymodules/python2.6/nose/importer.py", line 39, in 
importFromPath
    return self.importFromDir(dir_path, fqname)
  File "/usr/lib/pymodules/python2.6/nose/importer.py", line 86, in 
importFromDir
    mod = load_module(part_fqname, fh, filename, desc)
  File "[...]/numpy/ma/timer_comparison.py", line 6, in <module>
    import np.core.fromnumeric as fromnumeric
ImportError: No module named np.core.fromnumeric

======================================================================
ERROR: Failure: AttributeError ('module' object has no attribute '__revision__')
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/pymodules/python2.6/nose/loader.py", line 379, in 
loadTestsFromName
    addr.filename, addr.module)
  File "/usr/lib/pymodules/python2.6/nose/importer.py", line 39, in 
importFromPath
    return self.importFromDir(dir_path, fqname)
  File "/usr/lib/pymodules/python2.6/nose/importer.py", line 86, in 
importFromDir
    mod = load_module(part_fqname, fh, filename, desc)
  File "[...]/numpy/ma/version.py", line 9, in <module>
    revision = [core.__revision__.split(':')[-1][:-1].strip(),
AttributeError: 'module' object has no attribute '__revision__'

======================================================================
ERROR: Failure: ImportError (The convolve package is not installed.

It can be downloaded by checking out the latest source from
http://svn.scipy.org/svn/scipy/trunk/Lib/stsci or by downloading and
installing all of SciPy from http://www.scipy.org.
)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/pymodules/python2.6/nose/loader.py", line 379, in 
loadTestsFromName
    addr.filename, addr.module)
  File "/usr/lib/pymodules/python2.6/nose/importer.py", line 39, in 
importFromPath
    return self.importFromDir(dir_path, fqname)
  File "/usr/lib/pymodules/python2.6/nose/importer.py", line 86, in 
importFromDir
    mod = load_module(part_fqname, fh, filename, desc)
  File "[...]/numpy/numarray/convolve.py", line 14, in <module>
    raise ImportError(msg)
ImportError: The convolve package is not installed.

It can be downloaded by checking out the latest source from
http://svn.scipy.org/svn/scipy/trunk/Lib/stsci or by downloading and
installing all of SciPy from http://www.scipy.org.


======================================================================
ERROR: Failure: ImportError (The image package is not installed

It can be downloaded by checking out the latest source from
http://svn.scipy.org/svn/scipy/trunk/Lib/stsci or by downloading and
installing all of SciPy from http://www.scipy.org.
)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/pymodules/python2.6/nose/loader.py", line 379, in 
loadTestsFromName
    addr.filename, addr.module)
  File "/usr/lib/pymodules/python2.6/nose/importer.py", line 39, in 
importFromPath
    return self.importFromDir(dir_path, fqname)
  File "/usr/lib/pymodules/python2.6/nose/importer.py", line 86, in 
importFromDir
    mod = load_module(part_fqname, fh, filename, desc)
  File "[...]/numpy/numarray/image.py", line 14, in <module>
    raise ImportError(msg)
ImportError: The image package is not installed

It can be downloaded by checking out the latest source from
http://svn.scipy.org/svn/scipy/trunk/Lib/stsci or by downloading and
installing all of SciPy from http://www.scipy.org.


======================================================================
FAIL: test_blasdot.test_dot_3args
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/pymodules/python2.6/nose/case.py", line 183, in runTest
    self.test(*self.arg)
  File "[...]/numpy/core/tests/test_blasdot.py", line 52, in test_dot_3args
    assert_equal(sys.getrefcount(r), 2)
  File "[...]/numpy/testing/utils.py", line 313, in assert_equal
    raise AssertionError(msg)
AssertionError: 
Items are not equal:
 ACTUAL: 3
 DESIRED: 2

======================================================================
FAIL: test_dot_3args (test_multiarray.TestDot)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "[...]/numpy/core/tests/test_multiarray.py", line 1698, in test_dot_3args
    assert_equal(sys.getrefcount(r), 2)
  File "[...]/numpy/testing/utils.py", line 313, in assert_equal
    raise AssertionError(msg)
AssertionError: 
Items are not equal:
 ACTUAL: 3
 DESIRED: 2

[...]

Ran 3717 tests in 31.447s

FAILED (KNOWNFAIL=3, SKIP=6, errors=8, failures=2)
<nose.result.TextTestResult run=3717 errors=8 failures=2>


Anyone know what I'm doing wrong? 

I'm using Ubuntu 10.04 LTS in case that matters.

Thanks,
Chris




More information about the NumPy-Discussion mailing list