Mailman 3 January 2010 - pytest-dev

[py-dev] py.test : setup / teardown at the directory level
by Philippe Fremy April 16, 2010

April 16, 2010

Hi, I was wondering if there is a possibility to have or emulate setup/teardown method for an entire directory of tests ? I have an expensive setup/teardown method, which I share through about 10 modules, each with about 20 tests. I would save time if I could do the setup/teardown just for that directory. cheers, Philippe

3 9

[py-dev] 1.2.1 planned next week
by holger krekel Jan. 31, 2010

Jan. 31, 2010

Hi all, if you have anything small regarding py.test that you'd like to see addressed please speak up. I plan to release a 1.2.1 end next week. cheers, holger

4 9

[py-dev] [patch] greenlet on ARM/linux
by W-Mark Kubacki Jan. 31, 2010

Jan. 31, 2010

Hi, last year Renaud Blanch has provided a patch for ARM/Apple. [1] Based on his works I have ported 'greenlet' to ARM (EABI, non-thumb; all tests passed). Please find my patch attached. -- Grüße, Mark http://mark.ossdl.de/ (I am no subscriber to this list; regdarding this topic please CC me.) [1] http://codespeak.net/pipermail/py-dev/2009q1/001066.html

2 3

Re: [py-dev] [TIP] always call teardownX even if setupX fails?
by holger krekel Jan. 28, 2010

Jan. 28, 2010

On Thu, Jan 28, 2010 at 23:47 +1100, Robert Collins wrote: > On Thu, 2010-01-28 at 13:10 +0100, holger krekel wrote: > > > > > > not calling teardown seemed slightly easier to implement ... i can > > work > > a bit harder though :) > > > > However, I haven't see your example pattern with py.test usages yet. > > Is this a an example from actual code? > > Broadly yes. Usually seen more like this in fact: > > class A > setUp(): does stuff > > class B(A): > setUp(): check if we should skip If you can you'd probably first try to see if to skip and then call the base class' setUp(), though. Anyway, ... > > And if so do you have have and use it in setup_module and setup_class > > - alike > > methods? > > I don't use such methods, so no (see prior conversation about decoupling > resources etc). ... these cases were what seemed complicated to me. However, it turned out, my previous implementation was buggy anyway! So now there is no special casing for Skip anymore and teardown gets called in all cases - and i also refined the "setup_module/class" calling to only happen exactly once (before they'd be happen multiple times for each test if they raised an error). Also added two functional and one unit-test and hope things are settled now :) many thanks for your feedback and insistance on clarity here! cheers, holger

1 0

Re: [py-dev] [TIP] always call teardownX even if setupX fails?
by holger krekel Jan. 28, 2010

Jan. 28, 2010

On Thu, Jan 28, 2010 at 08:19 +1100, Robert Collins wrote: > On Wed, 2010-01-27 at 18:07 +0100, holger krekel wrote: > > > > > Any test with moderately complex external fixtures would benefit > > from > > > this behavior. We use a subclass of unittest.TestCase that always > > calls > > > tearDown() for all of our tests at Racemi. > > > > thanks for your feedback. py.test will do that as well with 1.2.1 - > > with > > the addition that triggering "Skipped" exceptions in a setup function > > will > > not call teardown. > > Why not? > > Consider: > > setUp: > self.foo = get_foo() > if not isinstance(self.foo, TestableThing): > raise TestSkipped() > > Why should this /not/ call tearDown ? not calling teardown seemed slightly easier to implement ... i can work a bit harder though :) However, I haven't see your example pattern with py.test usages yet. Is this a an example from actual code? And if so do you have have and use it in setup_module and setup_class - alike methods? cheers, holger

1 0

Re: [py-dev] [TIP] always call teardownX even if setupX fails?
by holger krekel Jan. 27, 2010

Jan. 27, 2010

Hi Rob, hi Doug, On Sat, Jan 23, 2010 at 10:47 -0500, Doug Hellmann wrote: > > On Jan 23, 2010, at 5:51 AM, Robert Collins wrote: > >> On Sat, 2010-01-23 at 00:43 +0100, holger krekel wrote: >>> >>> Doesn't look like a blocker to me at the moment. What do you think? >>> Any objections or recommendations? Do you regard consistency a >>> blocker? >> >> Trial and testtools run tearDown unconditionally even though its >> inconsistent with unittest. I'm a big fan of doing that, and encourage >> you to do it in py.test. > > +1 > > Any test with moderately complex external fixtures would benefit from > this behavior. We use a subclass of unittest.TestCase that always calls > tearDown() for all of our tests at Racemi. thanks for your feedback. py.test will do that as well with 1.2.1 - with the addition that triggering "Skipped" exceptions in a setup function will not call teardown. cheers, holger

1 0

[py-dev] Local import
by Joan Miller Jan. 23, 2010

Jan. 23, 2010

I'm using the next function to access to the source from tests without install it (python setup.py develop). It's very usefull during development so it could be added to py. ------------------------ import os import sys def import_local(parent='test', child='lib'): """Inserts local library on 'sys.path'. Goes up until the root of the project (upper of 'parent' directory) and inserts the 'child' directory. """ parent_path = os.path.abspath(os.path.dirname(__file__)) while parent in os.path.basename(parent_path): parent_path = os.path.dirname(parent_path) sys.path.insert(0, os.path.join(parent_path, child)) ------------------------

2 6

[py-dev] always call teardownX even if setupX fails?
by holger krekel Jan. 22, 2010

Jan. 22, 2010

Hi all, (cross-post, beware) i'd appreciate some feedback regarding a decision related to this issue: http://bitbucket.org/hpk42/py-trunk/issue/78/ where Jeff Stearns suggests to call teardown_module in any case. To summarize here, please consider this: def setup_module(mod): ... def teardown_module(mod): ... Currently teardown_module will not be called if setup_module raises an error. This is consistent with how nosetests and unittest.py behave (although unitttest doesn't yet have setup_module/class, but it has setUp/tearDown). For py.test we could easily change it such that teardown_module is called independently if the call to setup_module succeeded. The same would be true for setup_class/teardown_class and setup_method/teardown_method pairs. The rationale behind this is that setup_XYZ may set up several resources and fail somewhere in between. Not calling teardown_XYZ leaves resources unfinalized, potentially causing test setup failures in other test modules (this is what happend to Jeff). Do you think it would actually hurt to call teardown in any case? That might cause another error of course - with py.test you would both: http://paste.pocoo.org/show/168612 Doesn't look like a blocker to me at the moment. What do you think? Any objections or recommendations? Do you regard consistency a blocker? Note that the whole issue is not a problem with funcargs and recent unittest: you usually set up exactly one funcarg in a factory and if that fails there usually is no reason to call teardown. And if a function gets several funcargs, each successful funcarg setup will have a guaranteed call to the finalizer even if different funcarg factory calls fail. Moreover, you can always use "addfinalizer()" in a factory (similar to the recent unittest addition addCleanup()) which will trigger an unconditionally teardown. cheers, (feel free to leave out one of the cross-posted mailing lists), holger --

1 0

[py-dev] py.test-1.2.0: junitxml, standalone test scripts, pluginization
by holger krekel Jan. 19, 2010

Jan. 19, 2010

Hi all, i just released some bits related to testing: py-1.2.0: py.test core which grew junitxml, standalone-script generation pytest-xdist-1.0: separately installable dist-testing & looponfailing plugin pytest-figleaf-1.0: separately installable figleaf-coverage testing plugin See below or at this URL for the announcement: http://pylib.org/announce/release-1.2.0.html If you didn't experience much speed-up or had previously problems with distributed testing i recommend you try to install "pytest-xdist" now and see if it works better. For me it speeds up some tests runs by 500% on a 4 CPU machine due to its better internal model and several fixes. (It's five times because several tests depend on IO and don't block CPU meanwhile). Another tip: if you use "pip" (best with a virtualenv) you can do e.g.: pip install pytest-xdist pip uninstall pytest-xdist to conveniently activate/deactivate plugins for py.test. easy_install works ok as well but has no uninstall, yet remains the only option for installing with Python3 at the moment, though. You need to use the 'distribute' project's easy_install for that. cheers & have fun, holger py.test/pylib 1.2.0: junitxml, standalone test scripts, pluginization -------------------------------------------------------------------------------- py.test is an advanced automated testing tool working with Python2, Python3 and Jython versions on all major operating systems. It has a simple plugin architecture and can run many existing common Python test suites without modification. It offers some unique features not found in other testing tools. See http://pytest.org for more info. py.test 1.2.0 brings many bug fixes and interesting new abilities: * --junitxml=path will create an XML file for use with CI processing * --genscript=path creates a standalone py.test-equivalent test-script * --ignore=path prevents collection of anything below that path * --confcutdir=path only lookup conftest.py test configs below that path * a 'pytest_report_header' hook to add info to the terminal report header * a 'pytestconfig' function argument gives direct access to option values * 'pytest_generate_tests' can now be put into a class as well * on CPython py.test additionally installs as "py.test-VERSION", on Jython as py.test-jython and on PyPy as py.test-pypy-XYZ Apart from many bug fixes 1.2.0 also has better pluginization: Distributed testing and looponfailing testing now live in the separately installable 'pytest-xdist' plugin. The same is true for 'pytest-figleaf' for doing coverage reporting. Those two plugins can serve well now as blue prints for doing your own. thanks to all who helped and gave feedback, have fun, holger krekel, January 2010 Changes between 1.2.0 and 1.1.1 ===================================== - moved dist/looponfailing from py.test core into a new separately released pytest-xdist plugin. - new junitxml plugin: --junitxml=path will generate a junit style xml file which is processable e.g. by the Hudson CI system. - new option: --genscript=path will generate a standalone py.test script which will not need any libraries installed. thanks to Ralf Schmitt. - new option: --ignore will prevent specified path from collection. Can be specified multiple times. - new option: --confcutdir=dir will make py.test only consider conftest files that are relative to the specified dir. - new funcarg: "pytestconfig" is the pytest config object for access to command line args and can now be easily used in a test. - install 'py.test' and `py.which` with a ``-$VERSION`` suffix to disambiguate between Python3, python2.X, Jython and PyPy installed versions. - new "pytestconfig" funcarg allows access to test config object - new "pytest_report_header" hook can return additional lines to be displayed at the header of a test run. - (experimental) allow "py.test path::name1::name2::..." for pointing to a test within a test collection directly. This might eventually evolve as a full substitute to "-k" specifications. - streamlined plugin loading: order is now as documented in customize.html: setuptools, ENV, commandline, conftest. also setuptools entry point names are turned to canonical namees ("pytest_*") - automatically skip tests that need 'capfd' but have no os.dup - allow pytest_generate_tests to be defined in classes as well - deprecate usage of 'disabled' attribute in favour of pytestmark - deprecate definition of Directory, Module, Class and Function nodes in conftest.py files. Use pytest collect hooks instead. - collection/item node specific runtest/collect hooks are only called exactly on matching conftest.py files, i.e. ones which are exactly below the filesystem path of an item - change: the first pytest_collect_directory hook to return something will now prevent further hooks to be called. - change: figleaf plugin now requires --figleaf to run. Also change its long command line options to be a bit shorter (see py.test -h). - change: pytest doctest plugin is now enabled by default and has a new option --doctest-glob to set a pattern for file matches. - change: remove internal py._* helper vars, only keep py._pydir - robustify capturing to survive if custom pytest_runtest_setup code failed and prevented the capturing setup code from running. - make py.test.* helpers provided by default plugins visible early - works transparently both for pydoc and for interactive sessions which will regularly see e.g. py.test.mark and py.test.importorskip. - simplify internal plugin manager machinery - simplify internal collection tree by introducing a RootCollector node - fix assert reinterpreation that sees a call containing "keyword=..." - fix issue66: invoke pytest_sessionstart and pytest_sessionfinish hooks on slaves during dist-testing, report module/session teardown hooks correctly. - fix issue65: properly handle dist-testing if no execnet/py lib installed remotely. - skip some install-tests if no execnet is available - fix docs, fix internal bin/ script generation --

1 1

[py-dev] another traceback cut weirdness
by Ronny Pfannschmidt Jan. 18, 2010

Jan. 18, 2010

i just noticed that traceback.cut will return a traceback with all other filtering undone again since it passes a raw traceback entry into the Traceback ctor. is there any deeper reason for that or did it just work, cause cutting always was first if its not intentional i could add a .filter(lambda entry: entry in self) to get it solved -- Ronny

2 1