Mailman 3 September 2008 - pytest-dev

[py-dev] py.test --looponfailing (SVN r58313) AssertionError
by Neil Shepperd Sept. 23, 2008

Sept. 23, 2008

Hi all, Py.test dies with an assertion error when I change the name of a failing test, or move the file or directory it's in. It looks like it's just trying to get the result of the test without checking whether it exists first. This is with the HEAD revision. Neil ---------------- Example: [test_blah.py] def test_blah(): assert False $ py.test --looponfailing [edit test_blah.py] -def test_blah(): +def test_x(): Traceback (most recent call last): ... AssertionError: Collector <Module 'test_blah.py'> does not have name 'test_blah' existing names are: ['test_blah.py']

2 1

[py-dev] Information for unittest refugees
by Orestis Markou Sept. 21, 2008

Sept. 21, 2008

After attending Holger's talk at PyCon UK about py.test, I decided to try it out. I'm using a recent svn checkout. I had a bunch of tests (around 50 of them), all inheriting from unittest. I expected that they would work out of the box with py.test, and was surprised when they weren't. I tried adding the py_unittest conftest.py file to my project, but I got this traceback: Traceback (most recent call last): File "/opt/local/Library/Frameworks/Python.framework/Versions/ Current/bin/py.test", line 8, in <module> load_entry_point('py==1.0.0a1', 'console_scripts', 'py.test')() File "/Users/orestis/Developer/pylib/py/cmdline/pytest.py", line 5, in main py.test.cmdline.main() File "/Users/orestis/Developer/pylib/py/test/cmdline.py", line 12, in main config.parse(args) File "/Users/orestis/Developer/pylib/py/test/config.py", line 46, in parse self._conftest.setinitial(args) File "/Users/orestis/Developer/pylib/py/test/conftesthandle.py", line 30, in setinitial self._path2confmods[None] = self.getconftestmodules(anchor) File "/Users/orestis/Developer/pylib/py/test/conftesthandle.py", line 44, in getconftestmodules clist.append(importconfig(conftestpath)) File "/Users/orestis/Developer/pylib/py/test/conftesthandle.py", line 74, in importconfig mod = configpath.pyimport() File "/Users/orestis/Developer/pylib/py/path/local/local.py", line 413, in pyimport mod = __import__(modname, None, None, ['__doc__']) File "/Users/orestis/Developer/pysmell/conftest.py", line 52, in <module> class UnitTestFunction(py.__.test.item.Function): AttributeError: 'module' object has no attribute 'item' I expected that the naming conventions would be enough, but I was surprised again by the fact that py.test requires "test_" methods rather than just "test" methods. Fixing that, I was again surprised that my tests still weren't running, and after some guesswork realized that the class they were in didn't start with "Test". I suggest that you add this crucial information in a more clear way (it is there, but not very discoverable) or even better, integrate the py_unittest collection into py.test - this way people can try out their old tests with py.test, rather than having to convert everything. Best regards, Orestis Markou -- orestis(a)orestis.gr http://orestis.gr/

2 2

[py-dev] py.test problem
by Simon Burton Sept. 21, 2008

Sept. 21, 2008

I have hit this problem several times, when an assertion fails using py.test i get AttributeError: << 'frame' object has no attribute 'eval' with no stack trace of where in py.test this occured. (only the stack trace of my test code). Any ideas ? It looks like it has something to do with the magic that prints out the values of expressions in the assertion. I just did svn up on my py.test, it did not help. Simon.

2 1

[py-dev] showing tracebacks / failure handling (was: Re: [arigo@codespeak.net: [py-svn] r57790 - in py/trunk/py: code test test/report test/report/testing]
by holger krekel Sept. 4, 2008

Sept. 4, 2008

Hi Armin, (CC to py-dev) On Wed, Sep 03, 2008 at 12:10 +0200, Armin Rigo wrote: > Hi Holger, > > On Wed, Sep 03, 2008 at 10:30:53AM +0200, holger krekel wrote: > > i think the/a test should go to > > > > py/code/testing/test_excinfo.py:430 > > True, I can add a test there. But I had to change a few small things in > py/test/ too, so the existing test should not be removed. ok let's have both. > > also, i wonder if "tb=crash" would be useful so that only the > > crashing function is shown. > > Not for me. I don't use tb=short very often, but when I use it it's for > cases where PyPy tests spew tracebacks with a hundred entries. I use > tb=no a lot though. do you mean a few hundred failures? If you mean one failure with a few hundred entries than tb=crash would just show one entry. maybe we better sort this out on IRC :) > One thing I'm really missing from the trunk py.test output is the name > of the failing tests. I know it's often the name of the first function > in the traceback but this is not always reliable (e.g. with generated > code or broken .pyc files). My use case here is again PyPy: often, you > have 5 failures that each spew a lot of stdout and a long traceback. > Figuring out which 5 tests failed takes quite a while. In my > rxvt-terminal-only conftest, I can double-click on the final line that > says "5 failures" and see a list of the names of the failing tests. In > my opinion it would even be useful to always list these names in the > final line, e.g. looking like: > > ============= 2 failures: test_foo, test_bar =============== names are usually much longer, particularly if you include class and module name. > with something reasonable if there are too many failures, e.g.: > > ====== 17 failures: test_foo, test_bar, test_baz, ... ====== > > the point being that it's easy to re-run "py.test -k test_foo" to > reproduce just the first failure, which is often the most relevant one, > without having to use "py.test -x" which can take a long time to run > before it hits the failure. i see the point. hum, we could show a summary (by default or through option) after the tracebacks similar to what you you see with "--verbose", e.g. with curdir=py/doc/ and py.test --verbose pytest/failure_demo.py you currently get pytest/failure_demo.py:15: TestFailing.test_simple FAIL pytest/failure_demo.py:23: TestFailing.test_simple_multiline FAIL pytest/failure_demo.py:28: TestFailing.test_not FAIL pytest/failure_demo.py:33: TestFailing.test_complex_error FAIL pytest/failure_demo.py:40: TestFailing.test_z1_unpack_error FAIL pytest/failure_demo.py:44: TestFailing.test_z2_type_error FAIL ... But a more fundamental solution is to have py.test store info about the last test run (by option and/or some ENV variable) and have options/ways to show and run the last failing tests. The latter is also what we want for the new htmlconftest so i suggest we maybe first concentrate on this. in any case i'd be grateful if you put/copied your hacks around py.test to directories in contrib (maybe with at least one functional test) so that it'seasier to work toegether and ensure that things work after refactorings. best & thanks, holger

2 2