Testing: Failed examples don't raise errors on buildbot.
Alan, Stefan Not raising errors seems ok for examples, but some of the unit tests are also implemented as doctests and the failures are hidden in the logs. I'm not sure what to do about this, but thought it worth pointing out. Also, it would be nice if skipped tests didn't generate large bits of printout, it makes it hard to find relevant failures. Chuck
On Sun, Jul 20, 2008 at 7:10 PM, Charles R Harris <charlesr.harris@gmail.com> wrote:
Not raising errors seems ok for examples, but some of the unit tests are also implemented as doctests and the failures are hidden in the logs. I'm not sure what to do about this, but thought it worth pointing out. Also, it would be nice if skipped tests didn't generate large bits of printout, it makes it hard to find relevant failures.
Yeah I was just looking at that; right off the top of my head I don't know why that doctest failure wouldn't bubble all the way up to become a unit test failure. Personally, I'm not a big fan of having a test like that in a docstring and then trying to run it as a unit test, but I'll see if I can fix it. :) The skipped test verbosity is annoying; I'll see if there's a way to make that a bit cleaner-looking for some low verbosity level.
On Sun, Jul 20, 2008 at 9:17 PM, Alan McIntyre <alan.mcintyre@gmail.com> wrote:
The skipped test verbosity is annoying; I'll see if there's a way to make that a bit cleaner-looking for some low verbosity level.
The latest release version of nose from easy_install (0.10.3) doesn't generate that verbose output for skipped tests. Should we move up to requiring 0.10.3 for tests?
On Sun, Jul 20, 2008 at 21:47, Alan McIntyre <alan.mcintyre@gmail.com> wrote:
On Sun, Jul 20, 2008 at 9:17 PM, Alan McIntyre <alan.mcintyre@gmail.com> wrote:
The skipped test verbosity is annoying; I'll see if there's a way to make that a bit cleaner-looking for some low verbosity level.
The latest release version of nose from easy_install (0.10.3) doesn't generate that verbose output for skipped tests. Should we move up to requiring 0.10.3 for tests?
I don't think aesthetics are worth requiring a particular version. numpy doesn't need it; the users can decide whether they want it or not. We should try to have it installed on the buildbots, though, since we *are* the users in that case. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco
On Sun, Jul 20, 2008 at 10:56 PM, Robert Kern <robert.kern@gmail.com> wrote:
I don't think aesthetics are worth requiring a particular version. numpy doesn't need it; the users can decide whether they want it or not. We should try to have it installed on the buildbots, though, since we *are* the users in that case.
Actually I was considering asking to move the minimum nose version up to 0.10.3 just because it's the current version before this aesthetic issue came up. There's about 30 bug fixes between 0.10.0 and 0.10.3, including one that fixed some situations in which exceptions were being hidden and one that makes the coverage reporting more accurate. It's not a big deal, though.
On Sun, Jul 20, 2008 at 11:09:04PM -0400, Alan McIntyre wrote:
Actually I was considering asking to move the minimum nose version up to 0.10.3 just because it's the current version before this aesthetic issue came up. There's about 30 bug fixes between 0.10.0 and 0.10.3, including one that fixed some situations in which exceptions were being hidden and one that makes the coverage reporting more accurate. It's not a big deal, though.
There might be a case to move to 10.3, considering the large amount of bug fixes, but in general I think it is a bad idea to require leading edge packages. The reason being that you would like people to be able to rely on packaged version of the different tools to build an test a package. By packaged versions, I mean versions in the repositories of the main linux distributions, and macport and fink. Each time we require something outside a repository, we loose testers. Gaël
On Sun, Jul 20, 2008 at 11:17 PM, Gael Varoquaux <gael.varoquaux@normalesup.org> wrote:
There might be a case to move to 10.3, considering the large amount of bug fixes, but in general I think it is a bad idea to require leading edge packages. The reason being that you would like people to be able to rely on packaged version of the different tools to build an test a package. By packaged versions, I mean versions in the repositories of the main linux distributions, and macport and fink. Each time we require something outside a repository, we loose testers.
Fair enough; does anybody have any idea which version of nose is generally available from distributions like the ones you mentioned?
On Sun, Jul 20, 2008 at 11:19:57PM -0400, Alan McIntyre wrote:
On Sun, Jul 20, 2008 at 11:17 PM, Gael Varoquaux <gael.varoquaux@normalesup.org> wrote:
There might be a case to move to 10.3, considering the large amount of bug fixes, but in general I think it is a bad idea to require leading edge packages. The reason being that you would like people to be able to rely on packaged version of the different tools to build an test a package. By packaged versions, I mean versions in the repositories of the main linux distributions, and macport and fink. Each time we require something outside a repository, we loose testers.
Fair enough; does anybody have any idea which version of nose is generally available from distributions like the ones you mentioned?
Ubuntu hardy (current): 10.0 (http://packages.ubuntu.com) Ubuntu intrepid (next): 10.3 (http://packages.ubuntu.com) Debian unstable: 10.3 (http://packages.dbian.com) Fedora 8: 10.0 (https://admin.fedoraproject.org/pkgdb/) For the rest I can't figure out how to get the information. I suspect we can standardise on things around six month old. Debian unstable tracks closely upstream, Ubuntu and Fedora have a release cycle of 6 months, I don't know about SUSE, but I think it is similar, and macports, fink, or Gentoo trac closely upstream. Gaël
On Sun, Jul 20, 2008 at 11:34 PM, Gael Varoquaux <gael.varoquaux@normalesup.org> wrote:
For the rest I can't figure out how to get the information. I suspect we can standardise on things around six month old. Debian unstable tracks closely upstream, Ubuntu and Fedora have a release cycle of 6 months, I don't know about SUSE, but I think it is similar, and macports, fink, or Gentoo trac closely upstream.
It looks like Macports is at 0.10.1: http://py-nose.darwinports.com/ So it looks like 0.10.0 should still be a safe bet for being generally available.
participants (4)
-
Alan McIntyre -
Charles R Harris -
Gael Varoquaux -
Robert Kern