test errors on deprecation/runtime warnings
Hi, Last week we merged https://github.com/numpy/numpy/pull/201, which causes DeprecationWarning's and RuntimeWarning's to be converted to errors if they occur when running the test suite. The purpose of that is to make sure that code that still uses other deprecated code (or code that for some reason generates warnings) to be cleaned up. In principle this is a good idea IMHO, but after merging we quickly found some problems with failing tests in scipy. Because this potentially affects any other projects or users that use the numpy NoseTester test runner, here's a proposal to deal with this issue: - make this behavior configurable by a keyword in `NoseTester.__init__()` - default to raising an error in numpy master - when making a branch for release, immediately set the default to not raise. Do this not only for the 1.7 release, but for any future release (at least until the oldest numpy version still in use has the keyword). The reason for this is that otherwise a new numpy release will trigger test failures in older releases of scipy or other packages. The pros: - We find issues that otherwise get ignored (see the issues Christoph just found when compiling with MSVC). - We're forced to clean up code when submitting PRs, instead of letting the warnings accumulate and having to deal with them just before release time. The con: - You may see test errors if you run a released version of scipy with a development version of numpy. Opinions? Concerns? Ralf
Hi Ralf On Thu, Feb 16, 2012 at 11:05 AM, Ralf Gommers <ralf.gommers@googlemail.com> wrote:
Last week we merged https://github.com/numpy/numpy/pull/201, which causes DeprecationWarning's and RuntimeWarning's to be converted to errors if they occur when running the test suite.
It looks like this change affects other packages, too, which may legitimately raise RuntimeWarnings while running their test suites (unless I read the patch wrong). Would it be an option to rather add a flag (False by default) to enable this behaviour, and enable it inside of numpy.test() ? Regards Stéfan
2012/2/17 Stéfan van der Walt <stefan@sun.ac.za>
Hi Ralf
Last week we merged https://github.com/numpy/numpy/pull/201, which causes DeprecationWarning's and RuntimeWarning's to be converted to errors if
On Thu, Feb 16, 2012 at 11:05 AM, Ralf Gommers <ralf.gommers@googlemail.com> wrote: they
occur when running the test suite.
It looks like this change affects other packages, too,
It does, which is why I wanted to bring it up here.
which may legitimately raise RuntimeWarnings while running their test suites (unless I read the patch wrong). Would it be an option to rather add a flag (False by default) to enable this behaviour, and enable it inside of numpy.test() ?
Well, the idea is that this behavior is the correct one for all packages. It calls attention to those RuntimeWarnings, which may only occur on certain platforms. If they're legitimate, you silence them in the test suite of that package. If not, you fix them. Would you agree with that? Or would you prefer to just ignore DeprecationWarnings and/or RuntimeWarnings in skimage for example? Note that the changed behavior would only be visible for people running numpy master. Ralf
On Sat, Feb 18, 2012 at 10:13 AM, Ralf Gommers <ralf.gommers@googlemail.com>wrote:
2012/2/17 Stéfan van der Walt <stefan@sun.ac.za>
Hi Ralf
Last week we merged https://github.com/numpy/numpy/pull/201, which causes DeprecationWarning's and RuntimeWarning's to be converted to errors if
On Thu, Feb 16, 2012 at 11:05 AM, Ralf Gommers <ralf.gommers@googlemail.com> wrote: they
occur when running the test suite.
It looks like this change affects other packages, too,
It does, which is why I wanted to bring it up here.
which may legitimately raise RuntimeWarnings while running their test suites (unless I read the patch wrong). Would it be an option to rather add a flag (False by default) to enable this behaviour, and enable it inside of numpy.test() ?
Well, the idea is that this behavior is the correct one for all packages. It calls attention to those RuntimeWarnings, which may only occur on certain platforms. If they're legitimate, you silence them in the test suite of that package. If not, you fix them. Would you agree with that? Or would you prefer to just ignore DeprecationWarnings and/or RuntimeWarnings in skimage for example?
Note that the changed behavior would only be visible for people running numpy master.
This behavior has been made configurable, and I added instructions in HOWTO_RELEASE to turn it off in maintenance branches, in https://github.com/rgommers/numpy/compare/pull-219-warnings. There is some discussion at https://github.com/numpy/numpy/pull/219. The intention is to merge this soon, so now is the time to comment. Ralf
participants (2)
-
Ralf Gommers -
Stéfan van der Walt