Mailman 3 May 2011 - pytest-dev

Re: [py-dev] Decorators and funcargs in py.test
by Ronny Pfannschmidt June 6, 2011

June 6, 2011

On Mon, 2011-05-30 at 16:23 -0400, Vyacheslav Rafalskiy wrote: > No problem. Here is my (real life) example. > > My functional test functions may or may not return for different > reasons (like a faulty web application or middleware). I want to > declare a fail if it takes more than so many seconds to complete. So I > write a decorator > run_with_timeout(), which will start the function in a new thread and > abandon it after timeout. This can easily be solved by combining something like pytest.mark('timeout') and a override to pytest_pyfunc_call using it i suppose this could also take a look at extending the "--boxed" mode (which forks for each test and uses subprocesses but doesn’t handle timeouts atm. > > A I stated in OP, it is not that I cannot do it. I can and I do. My > point is that it makes sense to allow decorators and should not be > very difficult (see the example in OP). personally i am opposed to creating new data conventions for problems that can be solved with plain marks + a hook usually there are 2 reasons people use to decorate tests my opinions for implementing those are a) add arguments + their cleanups -> funcargs please, they are made for that b) use more sophisticated call's -> hooks please, maybe add a bug ticket for empowering one to return a exception-info, so stuff like thread-wrappers can pass that more nicely for failures *i* really think it is wrong to decorate test functions that way and expect stuff to work there are already plenty of mechanisms to change the behavior of pytest test function calling in the desired way, none of those require hacks to pass around argspecs -- Ronny > > Thanks, > Vyacheslav > > On Mon, May 30, 2011 at 3:55 PM, Ronny Pfannschmidt > <Ronny.Pfannschmidt(a)gmx.de> wrote: > > Hi, > > > > can you try to explain the usecase those decorators are fulfilling, > > > > there may be a better integrated way using pytest.mark + setup/teardown > > hooks > > > > -- Ronny > > > > On Mon, 2011-05-30 at 15:17 -0400, Vyacheslav Rafalskiy wrote: > >> Hi Holger, > >> > >> I am trying to make decorators work with test functions, which depend > >> on funcargs. As it stands, they don't. > >> Decorated functions lose funcargs. A workaround would be to decorate > >> an internal function like this: > >> > >> def test_it(funcarg_it): > >> @decorate_it > >> def _test_it(): > >> # test it > >> > >> _test_it() > >> > >> This works, but it is not nice. I'd rather wrote a decorator like > >> > >> def decorate_it(f): > >> def _wrap_it(*args, **kwargs): > >> # wrap f() here > >> > >> _wrap_it._varnames = _pytest.core.varnames(f) > >> return _wrap_it > >> > >> and apply it straight to the test function. > >> > >> After examining the source code I even expected it to just work > >> (magically of course) but it didn't. > >> Do you think it is worthwhile? If so I can enter a feature request. > >> > >> Thanks, > >> Vyacheslav > >> _______________________________________________ > >> py-dev mailing list > >> py-dev(a)codespeak.net > >> http://codespeak.net/mailman/listinfo/py-dev > > > >

3 8

[py-dev] migration from 1.3 to 2.0
by Vyacheslav Rafalskiy June 1, 2011

June 1, 2011

Hi Holger and list members, I have been using py.test for some time after switching from nose and quite happy with it. However, when migrating from 1.3.1 to 2.0.2 I hit a couple of snags, which makes me hesitate. Perhaps you can help me. 1. Just to clear the point. I understand from another post that option_xxxx variables in conftest.py are gone and recommended replacement is to use pytest_cmdline_preparse(). Is that so? 2. The new behavior is that even if you run py.test --help, the pytest_configure() is still run. In my case pytest_configure() does quite a bit of heavy lifting and raises exceptions at wrong parameters. The result is that if the parameter is wrong I cannot use help to find out what is the right parameter. Of course, the workaround could be: def pytest_configure(config): if config.getvalue('help'): return ... but it seems ugly. Am I in a wrong hook? 3. In my pytest_configure() there are numerous conditions when it can fail. For this conditions I have exceptions with specially crafted messages, intended for different people. Now they are gone, replaced by a long and scary listing with every line prepended with INTERNALERROR. What is internal about it? Can I continue managing my configuration errors? Thanks, Vyacheslav

2 13

[py-dev] Decorators and funcargs in py.test
by Vyacheslav Rafalskiy May 30, 2011

May 30, 2011

Hi Holger, I am trying to make decorators work with test functions, which depend on funcargs. As it stands, they don't. Decorated functions lose funcargs. A workaround would be to decorate an internal function like this: def test_it(funcarg_it): @decorate_it def _test_it(): # test it _test_it() This works, but it is not nice. I'd rather wrote a decorator like def decorate_it(f): def _wrap_it(*args, **kwargs): # wrap f() here _wrap_it._varnames = _pytest.core.varnames(f) return _wrap_it and apply it straight to the test function. After examining the source code I even expected it to just work (magically of course) but it didn't. Do you think it is worthwhile? If so I can enter a feature request. Thanks, Vyacheslav

3 2

[py-dev] tox 1.0 - rapid multi-python test automation
by holger krekel May 28, 2011

May 28, 2011

tox 1.0: the rapid multi-python test automation =========================================================================== I am happy to announce tox 1.0, a stabilization and maintenance release with some small improvements. tox automates tedious test activities driven from a simple ``tox.ini`` file, including: * creation and management of different virtualenv environments with different Python interpreters * packaging and installing your package into each of them * running your test tool of choice, be it nose, py.test or unittest2 or other tools such as "sphinx" doc checks * testing dev packages against each other without needing to upload to PyPI Docs and examples are now hosted at: http://tox.readthedocs.org Installation or upgrade with: pip install -U tox Note that code hosting and issue tracking has moved from Google to Bitbucket: http://bitbucket.org/hpk42/tox The 1.0 release includes contributions and is based on feedback and work from Chris Rose, Ronny Pfannschmidt, Jannis Leidel, Jakob Kaplan-Moss, Sridhar Ratnakumar, Carl Meyer and others. Many thanks! best, Holger Krekel CHANGES --------------------- - fix issue24: introduce a way to set environment variables for for test commands (thanks Chris Rose) - fix issue22: require virtualenv-1.6.1, obsoleting virtualenv5 (thanks Jannis Leidel) and making things work with pypy-1.5 and python3 more seemlessly - toxbootstrap.py (used by jenkins build slaves) now follows the latest release of virtualenv - fix issue20: document format of URLs for specifying dependencies - fix issue19: substitute Hudson for Jenkins everywhere following the renaming of the project. NOTE: if you used the special [tox:hudson] section it will now need to be named [tox:jenkins]. - fix issue 23 / apply some ReST fixes - change the positional argument specifier to use {posargs:} syntax and fix issues #15 and #10 by refining the argument parsing method (Chris Rose) - remove use of inipkg lazy importing logic - the namespace/imports are anyway very small with tox. - fix a fspath related assertion to work with debian installs which uses symlinks - show path of the underlying virtualenv invocation and bootstrap virtualenv.py into a working subdir - added a CONTRIBUTORS file

1 0

[py-dev] assertion code moving out of pylib
by Benjamin Peterson May 26, 2011

May 26, 2011

Hello everyone, I've been working on refactoring the assertion debugging code for py.test. Part of this has involved moving the assertion code from pylib to py.test proper. I don't think many people are using the assertion related code from pylib, but I wanted to give a heads up I plan to remove it soon from pylib. If you do use pylib assertion code and removing it would be a problem for you, I'd like to hear from you. -- Regards, Benjamin

1 0

[py-dev] What is the recommended way to run test with a coverage report?
by Baptiste Lepilleur May 22, 2011

May 22, 2011

The pytest documentation page indicates that it is supported, but provides no pointer on how do to this... Doing a search seem to reveal multiple plug-ins to do that. What is the recommended one? I'm working on Windows XP / Python 2.6 & 3.2. By the way, is there a centralized list of useful plug-ins for pytest ? Baptiste.

3 5

[py-dev] [ANN] plugin: pytest-incremental
by Eduardo Schettino May 6, 2011

May 6, 2011

Hi all, I have just pushed a new pytest plugin to pypi (http://pypi.python.org/pypi/pytest-incremental). The idea is to execute your tests faster by executing not all of them but only the "required" ones. This is very new (alpha), feedback is welcome :) Regards, Eduardo +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ pytest-incremental ==================== an incremental test runner (pytest plugin) What is an "incremental test runner" ? ======================================= When talking about build-tools it is common to refer to the terms: * initial (full) build - all files are compiled * incremental build (or partial rebuild) - just changed files are compiled * no-op build - no files are compiled (none changed since last execution) So an "incremental test runner" will only re-execute tests that were affected by changes in the source code since last test execution. How it works ? ================ `pytest-incremental` is a `pytest <http://pytest.org/>`_ plugin. So if you can run your test suite with pytest you can use `pytest-incremental`. The plugin will analyse your python source files and through its imports define the dependencies of the modules. `doit <http://python-doit.sourceforge.net>`_ is used to keep track of the dependencies and save results. The plugin will modify how pytest collect your tests. pytest do the rest of the job of actually running the tests and reporting the results. Install ========= pytest-incremental is tested on python 2.6, 2.7. ``pip install pytest-incremental``` ``python setup.py install`` local installation -------------------- You can also just grab the plugin `module <https://bitbucket.org/schettino72/pytest-incremental/src/tip/pytest_increme…>`_ file and put in your project path. Then enable it (check `pytest docs <http://pytest.org/plugins.html#requiring-loading-plugins-in-a-test-module-o…>`_). Usage ====== Just pass the parameter ``--incremental`` when calling from the command line:: $ py.test --incremental You can also enable it by default adding the following line to your ``pytest.ini``:: [pytest] addopts = --incremental watched packages ------------------ By default all modules collected by pytest will used as dependencies if imported. In order to limit or extend the watched folders you must use the parameter ``--watch-pkg`` Limitations ============== ``pytest-incremental`` looks for imports recursively to find dependencies (using AST). But given the very dynamic nature of python there are still some cases that a module can be affected by a module that are not detected. * `from package import *` modules imported from __all__ in a package are not counted as a dependency * modules imported not using the *import* statement * modules not explictitly imported but used at runtime (i.e. conftest.py when running your tests with pytest) * monkey-patching. (i.e. A imports X. B monkey-patches X. In this case A might depend on B) * others ? Project Details =============== - Project code + issue track on `bitbucket <https://bitbucket.org/schettino72/pytest-incremental>`_ - `Discussion group <http://groups.google.co.in/group/python-doit>`_

4 7

[py-dev] Help understanding pytest_generate_tests related error message
by Baptiste Lepilleur May 1, 2011

May 1, 2011

Hi, I just upgraded from py.test 1.3.4 to 2.0.3, and I'm running into a new failure related to pytest_generate_tests that I can not make sense of: --- py.test src\nitroz\test_dbo2.py ================================================= test session starts ================================================= platform win32 -- Python 3.2.0 -- pytest-2.0.3 collected 0 items / 1 errors ======================================================= ERRORS ======================================================== ______________________________________ ERROR collecting src/nitroz/test_dbo2.py _______________________________________ src\nitroz\test_dbo2.py:151: in pytest_generate_tests > good_value=good_value) ) C:\Python32\lib\site-packages\pytest-2.0.3-py3.2.egg\_pytest\python.py:550: in addcall > pytest.fail("funcarg %r not used in this function." % name) E Failed: funcarg 'property' not used in this function. =============================================== 1 error in 0.12 seconds =============================================== --- What does this error message means? What 'this function' refer to? Source of the test module can be found there: http://pastebin.com/1D2VArGt Baptiste.

2 6

[py-dev] How to run a given test by name?
by Baptiste Lepilleur May 1, 2011

May 1, 2011

Hi, typically when troubleshooting multiple test failures, I want to be able to run a single parametrized test case. How can you tell py.test to run only tests matching a specific name. For example, I'd like to be able to run: py.test src --filter-by-name "test_mandatory_property[ZText]" This would run all tests with a name matching "test_mandatory_property[ZText]". Is there a way to do that? Baptiste.

2 2

[py-dev] conftest global scope
by meme dough May 1, 2011

May 1, 2011

Hi, pytest-cov has an issue ticket that conftest global scope has no coverage reported. I don't think it will be possible to cover though. My understanding is that conftest file is local plugin which pytest will load quite early. It then can have hooks in conftest like to add args or change args. Coverage starts at session start hook which I think most appropriate. The hooks in conftest (like configure) could change things to turn coverage on / off or set coverage paths. So starting coverage any earlier than session start may not be good. Is this correct with conftest? It is not reloaded is it? It gets loaded early and hooks in it called to modify pytest operations. So no way we could get coverage at conftest global scope anyway? :)

1 0