problem with NumPy test harness in 1.1.0?
Hi, Our code depends on NumPy and we have been taking advantage of the NumPy test harness to create test scripts for new features as we develop them. As recently as version 1.1.0.dev5064 we had no problem running our tests from the directory in which they live, e.g.: python test_script.py .....<much output> . ---------------------------------------------------------------------- Ran 10 tests in 0.397s OK However, with the released version NumPy 1.1.0 none of our tests work any longer and instead we get this error message: Traceback (most recent call last): File "test_mfio.py", line 426, in <module> NumpyTest().run() File "/usr/local/lib/python2.5/site-packages/numpy/testing/ numpytest.py", line 655, in run testcase_pattern=options.testcase_pattern) File "/usr/local/lib/python2.5/site-packages/numpy/testing/ numpytest.py", line 575, in test level, verbosity) File "/usr/local/lib/python2.5/site-packages/numpy/testing/ numpytest.py", line 453, in _test_suite_from_all_tests importall(this_package) File "/usr/local/lib/python2.5/site-packages/numpy/testing/ numpytest.py", line 681, in importall for subpackage_name in os.listdir(package_dir): OSError: [Errno 2] No such file or directory: '' Looking through the archives I have found a reference to the same error at http://www.nabble.com/NumpyTest-problem-td17603890.html. However, this thread is focused on the use of NumpyTest for test scripts within the NumPy source tree and the suggestions do not seem to apply to our situation. My question is whether using NumpyTest for our own tests is considered to be a proper application of the test module. If not why not? If it is, is there some modification we can make to our tests to make them work with NumpyTest as it is now configured? Or is this possibly a bug in NumPyTest? Dave Brown PyNGL/PyNIO development team
On Tue, Jul 1, 2008 at 17:53, David Brown <dbrown@ucar.edu> wrote:
Hi, Our code depends on NumPy and we have been taking advantage of the NumPy test harness to create test scripts for new features as we develop them. As recently as version 1.1.0.dev5064 we had no problem running our tests from the directory in which they live, e.g.:
python test_script.py
.....<much output>
. ---------------------------------------------------------------------- Ran 10 tests in 0.397s
OK
However, with the released version NumPy 1.1.0 none of our tests work any longer and instead we get this error message:
Traceback (most recent call last): File "test_mfio.py", line 426, in <module> NumpyTest().run() File "/usr/local/lib/python2.5/site-packages/numpy/testing/ numpytest.py", line 655, in run testcase_pattern=options.testcase_pattern) File "/usr/local/lib/python2.5/site-packages/numpy/testing/ numpytest.py", line 575, in test level, verbosity) File "/usr/local/lib/python2.5/site-packages/numpy/testing/ numpytest.py", line 453, in _test_suite_from_all_tests importall(this_package) File "/usr/local/lib/python2.5/site-packages/numpy/testing/ numpytest.py", line 681, in importall for subpackage_name in os.listdir(package_dir): OSError: [Errno 2] No such file or directory: ''
Looking through the archives I have found a reference to the same error at http://www.nabble.com/NumpyTest-problem-td17603890.html.
However, this thread is focused on the use of NumpyTest for test scripts within the NumPy source tree and the suggestions do not seem to apply to our situation.
My question is whether using NumpyTest for our own tests is considered to be a proper application of the test module. If not why not?
You should migrate your tests to some other test runner. NumpyTest and NumpyTestCase will be deprecated in numpy 1.2 and removed as early as numpy 1.3. They were never particularly intended to be used outside of numpy and scipy. We are migrating to using nose as our test runner so we do not have to maintain our own; it's not our core competency. As you can see.
If it is, is there some modification we can make to our tests to make them work with NumpyTest as it is now configured?
Try NumpyTest().test(level=11, all=False)
Or is this possibly a bug in NumPyTest?
Possibly a bug. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco
OK, thanks for the info Robert. Any recommendations for another test harness? -dave On Jul 1, 2008, at 5:24 PM, Robert Kern wrote:
On Tue, Jul 1, 2008 at 17:53, David Brown <dbrown@ucar.edu> wrote:
Hi, Our code depends on NumPy and we have been taking advantage of the NumPy test harness to create test scripts for new features as we develop them. As recently as version 1.1.0.dev5064 we had no problem running our tests from the directory in which they live, e.g.:
python test_script.py
.....<much output>
. --------------------------------------------------------------------- - Ran 10 tests in 0.397s
OK
However, with the released version NumPy 1.1.0 none of our tests work any longer and instead we get this error message:
Traceback (most recent call last): File "test_mfio.py", line 426, in <module> NumpyTest().run() File "/usr/local/lib/python2.5/site-packages/numpy/testing/ numpytest.py", line 655, in run testcase_pattern=options.testcase_pattern) File "/usr/local/lib/python2.5/site-packages/numpy/testing/ numpytest.py", line 575, in test level, verbosity) File "/usr/local/lib/python2.5/site-packages/numpy/testing/ numpytest.py", line 453, in _test_suite_from_all_tests importall(this_package) File "/usr/local/lib/python2.5/site-packages/numpy/testing/ numpytest.py", line 681, in importall for subpackage_name in os.listdir(package_dir): OSError: [Errno 2] No such file or directory: ''
Looking through the archives I have found a reference to the same error at http://www.nabble.com/NumpyTest-problem-td17603890.html.
However, this thread is focused on the use of NumpyTest for test scripts within the NumPy source tree and the suggestions do not seem to apply to our situation.
My question is whether using NumpyTest for our own tests is considered to be a proper application of the test module. If not why not?
You should migrate your tests to some other test runner. NumpyTest and NumpyTestCase will be deprecated in numpy 1.2 and removed as early as numpy 1.3. They were never particularly intended to be used outside of numpy and scipy. We are migrating to using nose as our test runner so we do not have to maintain our own; it's not our core competency. As you can see.
If it is, is there some modification we can make to our tests to make them work with NumpyTest as it is now configured?
Try NumpyTest().test(level=11, all=False)
Or is this possibly a bug in NumPyTest?
Possibly a bug.
-- Robert Kern
"I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco _______________________________________________ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
On Tue, Jul 1, 2008 at 18:39, David Brown <dbrown@ucar.edu> wrote:
OK, thanks for the info Robert. Any recommendations for another test harness?
numpy and scipy are moving to nose as of numpy 1.2 and scipy 0.7. http://www.somethingaboutorange.com/mrl/projects/nose/ It has been a personal favorite of mine for some time, too. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco
On Tue, Jul 1, 2008 at 4:43 PM, Robert Kern <robert.kern@gmail.com> wrote:
On Tue, Jul 1, 2008 at 18:39, David Brown <dbrown@ucar.edu> wrote:
OK, thanks for the info Robert. Any recommendations for another test harness?
numpy and scipy are moving to nose as of numpy 1.2 and scipy 0.7.
http://www.somethingaboutorange.com/mrl/projects/nose/
It has been a personal favorite of mine for some time, too.
So has ipython recently, for what it's worth. It does make easy a lot of things that should be easy but aren't under unittest, which is a big plus in my view. We also adopted it after advice from Robert, others at Enthought and Titus Brown during a BOF session at last year's SciPy. The fact that nose absorbs/recognizes all regular unittest tests makes the transition fairly easy. Cheers, f
Thanks for the input Fernando. -dave On Jul 1, 2008, at 5:48 PM, Fernando Perez wrote:
On Tue, Jul 1, 2008 at 4:43 PM, Robert Kern <robert.kern@gmail.com> wrote:
On Tue, Jul 1, 2008 at 18:39, David Brown <dbrown@ucar.edu> wrote:
OK, thanks for the info Robert. Any recommendations for another test harness?
numpy and scipy are moving to nose as of numpy 1.2 and scipy 0.7.
http://www.somethingaboutorange.com/mrl/projects/nose/
It has been a personal favorite of mine for some time, too.
So has ipython recently, for what it's worth. It does make easy a lot of things that should be easy but aren't under unittest, which is a big plus in my view. We also adopted it after advice from Robert, others at Enthought and Titus Brown during a BOF session at last year's SciPy.
The fact that nose absorbs/recognizes all regular unittest tests makes the transition fairly easy.
Cheers,
f _______________________________________________ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
participants (3)
-
David Brown -
Fernando Perez -
Robert Kern