Changing unittest verbose output.
When running unittest with the -v flag, if a test has errors, and has a docstring, the test name is shown on one line, and the docstring is shown on the next line with the `ERROR` word. For example: ./python.exe -m unittest -v Lib.test_verbose_error.randomTest test_one (Lib.test_verbose_error.randomTest) this is a doc string with helpful info ... ERROR ====================================================================== ERROR: test_one (Lib.test_verbose_error.randomTest) this is a doc string with helpful info ---------------------------------------------------------------------- Traceback (most recent call last): File "/workspaces/cpython/Lib/test_verbose_error.py", line 11, in test_one assert l[1] ~^^^ IndexError: list index out of range ---------------------------------------------------------------------- Ran 1 test in 0.004s This cause for some confusion and we would like to fix it, the possible options are: 1. print the name and docstring, and ERROR, all on one line. 2. provide a command-line switch to select that behavior. The first option has a minor backwards compatibility breakage, since the default output would change and those lines would get longer. Any thoughts on the mentioned solutions? https://bugs.python.org/issue47133
El sáb, 26 mar 2022 a las 17:51, Itay Yeshaya (<itay.yeshaya@gmail.com>) escribió:
Any thoughts on the mentioned solutions?
This is a bit more out there, but can anyone explain why we show the docstring at all? As far as I can tell it causes a bunch of confusion and helps little. If there's a failing test, I want the test name so I can quickly find it in the code; I don't want the docstring which may not even be unique.
On 3/27/2022 10:51 AM, Jelle Zijlstra wrote:
El sáb, 26 mar 2022 a las 17:51, Itay Yeshaya (<itay.yeshaya@gmail.com>) escribió:
Any thoughts on the mentioned solutions?
This is a bit more out there, but can anyone explain why we show the docstring at all? As far as I can tell it causes a bunch of confusion and helps little. If there's a failing test, I want the test name so I can quickly find it in the code; I don't want the docstring which may not even be unique.
I think this is a good idea. The target audience for a docstring isn't the people trying to track down failing tests. Eric
I recall actually really liking the docstring. I think there are situations where the test name just can't be sufficiently explanatory. IIRC in the past we didn't print the docstring at all and it felt right when it was added. ATM I'm not using unittest for anything so I don't have a strong opinion. On Sun, Mar 27, 2022 at 8:21 AM Eric V. Smith <eric@trueblade.com> wrote:
On 3/27/2022 10:51 AM, Jelle Zijlstra wrote:
El sáb, 26 mar 2022 a las 17:51, Itay Yeshaya (<itay.yeshaya@gmail.com>) escribió:
Any thoughts on the mentioned solutions?
This is a bit more out there, but can anyone explain why we show the docstring at all? As far as I can tell it causes a bunch of confusion and helps little. If there's a failing test, I want the test name so I can quickly find it in the code; I don't want the docstring which may not even be unique.
I think this is a good idea. The target audience for a docstring isn't the people trying to track down failing tests.
Eric _______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/BTF4LZWV... Code of Conduct: http://python.org/psf/codeofconduct/
-- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-c...>
On Mar 27, 2022, at 11:20 AM, Eric V. Smith <eric@trueblade.com> wrote:
On 3/27/2022 10:51 AM, Jelle Zijlstra wrote:
El sáb, 26 mar 2022 a las 17:51, Itay Yeshaya (<itay.yeshaya@gmail.com>) escribió:
Any thoughts on the mentioned solutions?
This is a bit more out there, but can anyone explain why we show the docstring at all? As far as I can tell it causes a bunch of confusion and helps little. If there's a failing test, I want the test name so I can quickly find it in the code; I don't want the docstring which may not even be unique.
I think this is a good idea. The target audience for a docstring isn't the people trying to track down failing tests.
I’m wrong about the docstrings having a different audience. So I’m going to retract my support here. Eric
On Mar 26, 2022, at 17:48, Itay Yeshaya <itay.yeshaya@gmail.com> wrote:
When running unittest with the -v flag, if a test has errors, and has a docstring, the test name is shown on one line, and the docstring is shown on the next line with the `ERROR` word.
This has been a long-standing nuisance, but I’m like Guido. I pretty much use pytest for everything these days, except for maybe unittest.mock.patch. -Barry
For many of us, this isn't a nuisance. It is a desirable feature. test_xxx functions and methods typically don't have docstrings at all as the majority of tests tend to be concise and obvious with the function name itself describes what its purpose is. When we added printing the docstring, it was useful, as you can expand upon the purpose of the test more than you can in a reasonable name within the docstring. We do this all the many times in our own test suite. When running the tests, you see the docstring and are given more context as to what the test is about. This can be useful when triaging a failure before you've even loaded the source. I don't doubt that someone writes thesis defenses and stores them in their TestCase.test_method docstrings. I'm just saying that is not the norm. I'd accept a PR adding another command line switch for unittest to disable docstring printing in verbose mode. -gps On Sun, Mar 27, 2022 at 12:59 PM Barry Warsaw <barry@python.org> wrote:
On Mar 26, 2022, at 17:48, Itay Yeshaya <itay.yeshaya@gmail.com> wrote:
When running unittest with the -v flag, if a test has errors, and has a
docstring, the test name is shown on one line, and the docstring is shown on the next line with the `ERROR` word.
This has been a long-standing nuisance, but I’m like Guido. I pretty much use pytest for everything these days, except for maybe unittest.mock.patch.
-Barry
_______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/ZIMXSRQM... Code of Conduct: http://python.org/psf/codeofconduct/
Hopefully it only prints the first line of the docstring? On Sun, Mar 27, 2022 at 2:42 PM Gregory P. Smith <greg@krypto.org> wrote:
For many of us, this isn't a nuisance. It is a desirable feature.
test_xxx functions and methods typically don't have docstrings at all as the majority of tests tend to be concise and obvious with the function name itself describes what its purpose is. When we added printing the docstring, it was useful, as you can expand upon the purpose of the test more than you can in a reasonable name within the docstring. We do this all the many times in our own test suite. When running the tests, you see the docstring and are given more context as to what the test is about. This can be useful when triaging a failure before you've even loaded the source.
I don't doubt that someone writes thesis defenses and stores them in their TestCase.test_method docstrings. I'm just saying that is not the norm.
I'd accept a PR adding another command line switch for unittest to disable docstring printing in verbose mode.
-gps
On Sun, Mar 27, 2022 at 12:59 PM Barry Warsaw <barry@python.org> wrote:
On Mar 26, 2022, at 17:48, Itay Yeshaya <itay.yeshaya@gmail.com> wrote:
When running unittest with the -v flag, if a test has errors, and has a
docstring, the test name is shown on one line, and the docstring is shown on the next line with the `ERROR` word.
This has been a long-standing nuisance, but I’m like Guido. I pretty much use pytest for everything these days, except for maybe unittest.mock.patch.
-Barry
_______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/ZIMXSRQM... Code of Conduct: http://python.org/psf/codeofconduct/
_______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/RJOWQWMM... Code of Conduct: http://python.org/psf/codeofconduct/
-- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-c...>
On Sun, Mar 27, 2022 at 2:50 PM Guido van Rossum <guido@python.org> wrote:
Hopefully it only prints the first line of the docstring?
Indeed, it's just the first line (traditionally a one line summary per style). https://github.com/python/cpython/blob/f6b3a07b7df60dc04d0260169ffef6e9796a2...
On Sun, Mar 27, 2022 at 2:42 PM Gregory P. Smith <greg@krypto.org> wrote:
For many of us, this isn't a nuisance. It is a desirable feature.
test_xxx functions and methods typically don't have docstrings at all as the majority of tests tend to be concise and obvious with the function name itself describes what its purpose is. When we added printing the docstring, it was useful, as you can expand upon the purpose of the test more than you can in a reasonable name within the docstring. We do this all the many times in our own test suite. When running the tests, you see the docstring and are given more context as to what the test is about. This can be useful when triaging a failure before you've even loaded the source.
I don't doubt that someone writes thesis defenses and stores them in their TestCase.test_method docstrings. I'm just saying that is not the norm.
I'd accept a PR adding another command line switch for unittest to disable docstring printing in verbose mode.
-gps
On Sun, Mar 27, 2022 at 12:59 PM Barry Warsaw <barry@python.org> wrote:
On Mar 26, 2022, at 17:48, Itay Yeshaya <itay.yeshaya@gmail.com> wrote:
When running unittest with the -v flag, if a test has errors, and has
a docstring, the test name is shown on one line, and the docstring is shown on the next line with the `ERROR` word.
This has been a long-standing nuisance, but I’m like Guido. I pretty much use pytest for everything these days, except for maybe unittest.mock.patch.
-Barry
_______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/ZIMXSRQM... Code of Conduct: http://python.org/psf/codeofconduct/
_______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/RJOWQWMM... Code of Conduct: http://python.org/psf/codeofconduct/
-- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-c...>
On Sun, 27 Mar 2022 00:48:04 -0000 "Itay Yeshaya" <itay.yeshaya@gmail.com> wrote:
When running unittest with the -v flag, if a test has errors, and has a docstring, the test name is shown on one line, and the docstring is shown on the next line with the `ERROR` word. For example:
./python.exe -m unittest -v Lib.test_verbose_error.randomTest test_one (Lib.test_verbose_error.randomTest) this is a doc string with helpful info ... ERROR
====================================================================== ERROR: test_one (Lib.test_verbose_error.randomTest) this is a doc string with helpful info ---------------------------------------------------------------------- Traceback (most recent call last): File "/workspaces/cpython/Lib/test_verbose_error.py", line 11, in test_one assert l[1] ~^^^ IndexError: list index out of range
---------------------------------------------------------------------- Ran 1 test in 0.004s
This cause for some confusion and we would like to fix it, the possible options are:
Which confusion is this about? Unit tests are not APIs, so the only reason to write a docstring is if you want it to appear in test output. If you don't want it to appear in test output, write a comment instead. Regards Antoine.
participants (7)
-
Antoine Pitrou -
Barry Warsaw -
Eric V. Smith -
Gregory P. Smith -
Guido van Rossum -
Itay Yeshaya -
Jelle Zijlstra