[Python-Dev] Aborting unit tests on first failure

[Nikolaus Rath]

Hello,

I've just run the testsuite of hg tip with

./python -m test -u network,urlfetch -j 8 -G -v

and it finished with

,----
| [...]
| test_extract_dir (test.test_zipfile.TestWithDirectory) ... ok
| test_store_dir (test.test_zipfile.TestWithDirectory) ... ok
| test_different_file (test.test_zipfile.TestsWithMultipleOpens) ... ok
| test_interleaved (test.test_zipfile.TestsWithMultipleOpens) ... ok
| test_same_file (test.test_zipfile.TestsWithMultipleOpens) ... ok
| 
| ----------------------------------------------------------------------
| Ran 163 tests in 14.522s
| 
| OK (skipped=25)
| 368 tests OK.
| 2 tests failed:
|     test_decimal test_itertools
| 1 test altered the execution environment:
|     test___all__
| 17 tests skipped:
|     test_bz2 test_curses test_dbm_gnu test_devpoll test_idle
|     test_kqueue test_msilib test_ossaudiodev test_readline
|     test_startfile test_tcl test_tk test_ttk_guionly test_ttk_textonly
|     test_winreg test_winsound test_zipfile64
`----

I thought the -G option is would cause the test to stop as soon as an
error occured:

|  -G, --failfast        fail as soon as a test fails (only with -v or -W)

But it my case it seems that it actually continued to run all the other
test modules. Did I misunderstand what -G is supposed to do, or is this
a bug in the test runner?

It seems to work fine within a single test module, i.e. if I run

| ./python -m test -u network,urlfetch -j 8 -G -v test_decimal

..then execution stops right after the failed test.


Best,
-Nikolaus

--
GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F
Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F

             »Time flies like an arrow, fruit flies like a Banana.«