Mailman 3 November 2004 - pytest-dev

[py-dev] more doctest...
by Ian Bicking Nov. 28, 2004

Nov. 28, 2004

Following up on my other doctest email, here's a slightly better technique I'm using: from test_fixture import DoctestCollector import some_module collect_doctest = DoctestCollector(some_module) Then I added something to py.test.collect.Module: def collect_collectors(self, extpy): if extpy.check(basestarts='collect_'): iter = extpy.resolve()(self.extpy) for item in iter: yield item And this is the new test_fixture.py, with just a small addition to DoctestCollector (__init__ and __call__), though extpy starts to feel pretty extraneous: class DoctestCollector(PyCollector): def __init__(self, extpy_or_module): if isinstance(extpy_or_module, types.ModuleType): self.module = extpy_or_module self.extpy = None else: self.extpy = extpy_or_module self.module = self.extpy.getpymodule() def __call__(self, extpy): # we throw it away, because this has been set up to explicitly # check another module; maybe this isn't clean if self.extpy is None: self.extpy = extpy return self def __iter__(self): finder = doctest.DocTestFinder() tests = finder.find(self.module) for t in tests: yield DoctestItem(self.extpy, t) class DoctestItem(DoctestCollector.Item): def __init__(self, extpy, doctestitem, *args): self.extpy = extpy self.doctestitem = doctestitem self.name = extpy.basename self.args = args def execute(self, driver): runner = doctest.DocTestRunner() driver.setup_path(self.extpy) target, teardown = driver.setup_method(self.extpy) try: (failed, tried), run_output = capture_stdout(runner.run, self.doctestitem) if failed: raise self.Failed(msg=run_output, tbindex=-2) finally: if teardown: teardown(target) def capture_stdout(func, *args, **kw): newstdout = StringIO() oldstdout = sys.stdout sys.stdout = newstdout try: result = func(*args, **kw) finally: sys.stdout = oldstdout return result, newstdout.getvalue() -- Ian Bicking / ianb(a)colorstudy.com / http://blog.ianbicking.org

1 1

[py-dev] failing unit tests
by Martijn Faassen Nov. 27, 2004

Nov. 27, 2004

Hi there, I get a number of failing tests when I check out Py and run py.test on it. If I read the output right, the failing tests are: /home/faassen/working/dist-py2/py/misc/test_initpkg.test_importing_all_implementations, line 21 /home/faassen/working/dist-py2/py/test/test/import_test/package/test_import.test_import, line 18 Note that the test.py output is rather intimidating, especially if you do not know the codebase you're testing. This is a realistic scenario for any newbie to the codebase like myself in this case, or possibly for deployment scenarios where someone is asked to run the test suite just to answer the question "which tests are failing?". Perhaps a 'non-verbose' mode would be nice, or a 'sysadmin mode' where only a bit of output is seen directly by the sysadmin, and the rest is dumped into a file that can be emailed to the developers. Anyay, I also seem to have a lot of recorded stdout like this: - - - - - - - - - - - - - - - recorded stdout - - - - - - - - - - - - - - - These modules should be identical: shared_lib: <module '/home/faassen/working/dist-py2/py/test/test/import_test/package/shared_lib/py' from '/home/faassen/working/dist-py2/py/test/test/import_test/package/shared_lib.py'> shared_lib2: <module 'package.shared_lib' from '/home/faassen/working/dist-py2/py/test/test/import_test/package/shared_lib.pyc'> This is all rather mysterious to me as well. Regards, Martijn

3 5

[py-dev] test.py bug report -- string exceptions
by David Ripton Nov. 26, 2004

Nov. 26, 2004

[This is a resend. Apologies if the original gets unstuck and you see this twice.] I just started playing with test.py, and so far it looks very nice. But I found one little bug so far. (Is there a bug tracker?) Using test.py to test a method that raises a string exception (deprecated, I know) results in a fatal exception that ends the test. Reproduction recipe: ---blah.py-------------- def blah(): raise "TODO" def test_blah(): blah() def test_blah_more(): not_reached() ------------------------ Then $ py.test blah.py The exception is: File "/home/dripton/dist-py/py/test/report/text/summary.py", line 111, in repr_failure_result if issubclass(cls, Item.ExceptionFailure): TypeError: issubclass() arg 1 must be a class -- David Ripton dripton(a)ripton.net

2 1

[py-dev] py.test: doctests
by Ian Bicking Nov. 22, 2004

Nov. 22, 2004

I played around with doctests today, here's the collector I have so far (requires the 2.4 version of doctest): class DoctestCollector(PyCollector): def __iter__(self): finder = doctest.DocTestFinder() tests = finder.find(self.extpy.getpymodule()) for t in tests: yield DoctestItem(self.extpy, t) class DoctestItem(DoctestCollector.Item): def __init__(self, extpy, doctestitem, *args): self.extpy = extpy self.doctestitem = doctestitem self.name = extpy.basename self.args = args def execute(self, driver): runner = doctest.DocTestRunner() driver.setup_path(self.extpy) target, teardown = driver.setup_method(self.extpy) try: (tried, failed), output = capture_stdout( runner.run, self.doctestitem) if failed: raise self.Failed(msg=output, tbindex=-2) finally: if teardown: teardown(target) def capture_stdout(func, *args, **kw): newstdout = StringIO() oldstdout = sys.stdout sys.stdout = newstdout try: result = func(*args, **kw) finally: sys.stdout = oldstdout return result, newstdout.getvalue() Some problems: * Still the problem where overriding one collector (in this case, using DoctestCollector) cancels the normal collection, instead of adding to it. I guess it's possible that we could do collect_doctests, then use the finder on each object (since the finder can work on individual functions and methods, not just module objects). * Still the naming problem. DocTest instances (doctestitem) have a name attribute, I suppose that should be involved. * The exception reports in py.test aren't really meaningful. tbindex=-2 got rid of some of them, but I wasn't sure how to get rid of all of them. * Generally the results of running the doctest should be folded into the py.test output better. I find item.Item.Outcome a little confusing, as there's some possible attributes, but they aren't documented in that class (due to self.__dict__.update(kw)). I'd rather those attributes be class attributes, to indicate defaults and provide a place for documentation. Though generally I don't understand Outcome. Also, it seems to be raised, but it's not a subclass of Exception? Anyway, it's probably not too hard, if I understood the reporter better; maybe subclassing DocTestRunner. -- Ian Bicking / ianb(a)colorstudy.com / http://blog.ianbicking.org

1 0

[py-dev] py.test: customizing collector
by Ian Bicking Nov. 21, 2004

Nov. 21, 2004

I got a custom collector working okay; this checks for a function attribute "params", which is a list of tuples, each of which is an argument. I did it like this: from py.test.collect import PyCollector class ParamCollector(PyCollector): def collect_function(self, extpy): if not extpy.check(func=1, basestarts='test_'): return func = extpy.resolve() if hasattr(func, 'params'): params = func.params for param in params: yield self.Item(extpy, *param) else: yield self.Item(extpy) The problem is that all the tests have the same name, at least as displayed. I notice that item.extpy is used as the displayed name for the test; at least some other function should be used (py.test.report.text.summary:192) so that it's easier to override. But I guess it should also fit into --session, and I don't know what that uses for its test ID...? It might be nice if this parameterization was built into the standard collector in some way, as it's a very common case. I'm not sure if it's best as an attribute, a similarly-named object (e.g., test_func and test_func_params), or if there's other ideas. -- Ian Bicking / ianb(a)colorstudy.com / http://blog.ianbicking.org

2 4

[py-dev] Import problems under py.test
by Ian Bicking Nov. 17, 2004

Nov. 17, 2004

I'm having a problem with imports when using py.test. The file db.py gets imported twice. It actually comes from two different file locations, but it's the same file, and there should only be one way to get to it. In this case, importing the module twice immediately throws an exception. /home/ianb/repository/Landscaper/external is in sys.path, but /home is a symlink to /usr/home. This is the class that is initially imported: <class '/usr/home/ianb/repository/Landscaper/external/lib/db/py.SiteSQLObject'> This is the normal class, which is created second: <class 'lib.db.SiteSQLObject'> I guess the problem is that my test script imports db (relatively), then imports a module that imports db (also relatively). I can probably turn this into a simpler test case, but I thought I'd see if the problem was obvious. -- Ian Bicking / ianb(a)colorstudy.com / http://blog.ianbicking.org

3 4

[py-dev] py.path import hook for getpymodule()
by Armin Rigo Nov. 10, 2004

Nov. 10, 2004

Hi all, I tried to implement the idea Holger and me discussed some time ago on IRC: the method getpymodule() of py.path objects, formerly known as getmodule(), should be smarter about importing modules that contain relative imports, i.e. that import other modules from the same directory. This requires a custom import hook, because we might be talking about a py.path which is not a local path. For example, in an Subversion repository, I have two files: x.py: def x1(): return 5 test_x.py: import x def test_x1(): assert x.x1() == 5 The goal is to make this work: mod = py.path.svnurl('svn+ssh://.../test_x.py').getpymodule() mod.test_x1() This is done by putting in the __file__ attribute of the 'test_x' module a subclass of str, as suggested by Holger, which remembers the py.path. A custom import hook detects the "import" statements coming from modules with such a __file__, and tries to resolve the import locally, relative to the py.path attached to the __file__. If it succeeds, it calls getpymodule() again to import the new module. Otherwise, it falls back to the standard import hook. The patch is attached for review (not checked in right now; depends if we want this kind of hack or not). Note that it doesn't work if the modules being imported hack around sys.path themselves. It works, though, if the hack is being performed by py.magic.autopath(): if called from a module with a special __file__, autopath() will not actually hack sys.path but just record the information it deduced on the path object. Then the custom import hook notices this information... Not too clean. It does allow this kind of stuff to work, though: stuff/__init__.py: # empty stuff/x.py: def x1(): return 5 stuff/test_x.py: import py; py.magic.autopath() import stuff.x # or "from stuff import x", which is harder def test_x1(): assert stuff.x.x1() == 5 Finally, note that (I'm not sure but I believe that) this custom import hook would not be needed with the new hooks of Python 2.3, which would allow a similar result in a cleaner way. Of course that would be 2.3-specific. A bientôt, Armin

3 7