Mailman 3 March 2011 - pytest-dev

[py-dev] latest pytest questions for pytest-cov
by meme dough April 6, 2011

April 6, 2011

Hi, Currently looking at fixing pytest-cov to work with latest pytest. I see two issues so far. 1. How to determine if non distributed, or distributed master, or distributed slave. This used to look at the session class Session = Central (non distributed), DSession = distributed master, and SlaveSession = distributed slave. So now I'm thinking look at session.config.pluginmanager.hasplugin('dsession') which means distributed master. For distributed slave check if config.slaveinput is there which means distributed slave. If none of those then must be non distributed. Is there a better way to determine at pytest_sessionstart if the process is central (non distributed), distributed master or distributed slave? 2. Need to know topdir. This used to use config.topdir to know what the root dir is. This is needed since distributed testing may be co-located or not. If not then have to rewrite all source paths back to what the master is set to so that all coverage files combine correctly even if have many separate tx envs on diff machines on diff paths. Has this moved somewhere? I see config.fspath, is that it? I look more I just got to this bit. :)

2 3

Re: [py-dev] Testing web services
by James Mills March 15, 2011

March 15, 2011

On Mon, Mar 14, 2011 at 5:40 PM, Frederik Dohr <fdg001(a)gmx.net> wrote: > FWIW, TiddlyWeb's HTTP API is being tested using httplib2, with tests > described concisely in YAML: > https://github.com/tiddlyweb/tiddlyweb/blob/master/test/httptest.yaml > https://github.com/tiddlyweb/tiddlyweb/blob/master/test/test_web_http_api.py > https://github.com/tiddlyweb/tiddlyweb/blob/master/test/http_runner.py I wonder how hard it would be to make this work with pytest and the example of running non-python test that Holger presented here: http://pytest.org/example/nonpython.html I'll have a go at it today... cheers James -- -- James Mills -- -- "Problems are solved by method"

1 2

Re: [py-dev] pytest_cov: start session before test improts
by James Mills March 15, 2011

March 15, 2011

On Mon, Mar 14, 2011 at 10:07 PM, meme dough <memedough(a)gmail.com> wrote: > I just tried with one of my projects and it's got the def lines and > all the imports as covered. > > Do you have a small example so I can see? The command you're using > and the the versions would be good too. Maybe drop at > http://paste.pocoo.org/ After some testing I reproduced the problem. The behavior I'm seeing only occurs with Python 3.2 Would this be a problem with pytest, puytest_cov or coverage wrt Python 3 ? Bed time :) cheers James -- -- James Mills -- -- "Problems are solved by method"

1 1

[py-dev] Testing web services
by James Mills March 14, 2011

March 14, 2011

Hello all, I'm thinking about how to go about testing web services in a new services-oriented application framework we're working on. Are there any assisting libraries, tools, hooks or plugins to make this job a litt easier with pytest ? Right now, I'm taking the naive approach which is to simply start the application in a ``setup_module(...)``, perform the necessary tests, then shut the application down in a ``teardown_module(...)``. I was simply going to be using ``urllib`` to perform requests against it and assert it's results. Maybe there's a more sophisticated way ? In the end, we will likely setup and use selenium to do more rigorous system testing, but right now I'm looking for more service-level testing at the data/logic layers of our application. cheers James -- -- James Mills -- -- "Problems are solved by method"

3 4

Re: [py-dev] Testing web services
by James Mills March 14, 2011

March 14, 2011

On Mon, Mar 14, 2011 at 5:40 PM, Frederik Dohr <fdg001(a)gmx.net> wrote: > FWIW, TiddlyWeb's HTTP API is being tested using httplib2, with tests > described concisely in YAML: > https://github.com/tiddlyweb/tiddlyweb/blob/master/test/httptest.yaml > https://github.com/tiddlyweb/tiddlyweb/blob/master/test/test_web_http_api.py > https://github.com/tiddlyweb/tiddlyweb/blob/master/test/http_runner.py Thank you! Judging from the YAML declarations this looks really good. cheers James -- -- James Mills -- -- "Problems are solved by method"

1 0

[py-dev] py.test doctest and hooks...
by Christian Klinger March 14, 2011

March 14, 2011

Hello, are there any hooks in pytest which are executed before doctests are executed? I try to setup some stuff, which i want to hide in the doctest. I already tried some "normal" pytest hooks ala: - pytest_runtest_call(item):- pytest_runtest_setup(item): But none of them seems to be executed on doctest execution. Any ideas? Christian

2 3

[py-dev] pytest_cov: start session before test improts
by James Mills March 14, 2011

March 14, 2011

Helo all, >From the FAQ of the coverage module (1): """ Q: Why do the bodies of functions (or classes) show as executed, but the def lines do not? This happens because coverage is started after the functions are defined. The definition lines are executed without coverage measurement, then coverage is started, then the function is called. This means the body is measured, but the definition of the function itself is not. To fix this, start coverage earlier. If you use the command line to run your program with coverage, then your entire program will be monitored. If you are using the API, you need to call coverage.start() before importing the modules that define your functions. """ I'm not exactly sure (after some playing with pytest_cov plugin) how to do this... I'll take another look tonight if I can, but appreciate someone more experienced with pytest internals to take a look :) cheers James 1. http://nedbatchelder.com/code/coverage/faq.html#faq -- -- James Mills -- -- "Problems are solved by method"

1 0

[py-dev] pytest-2.0.2: bug fixes, improved xfail/skip expressions, speedups
by holger krekel March 9, 2011

March 9, 2011

Welcome to pytest-2.0.2, a maintenance and bug fix release. py.test is a mature testing tool for Python, supporting CPython 2.4-3.2, Jython and latest PyPy interpreters. See the extensive docs with tested examples here: http://pytest.org/ If you want to install or upgrade pytest, just type one of:: pip install -U pytest # or easy_install -U pytest Many thanks to all issue reporters and people asking questions or complaining, particularly Jurko for his insistence, Laura, Victor and Brianna for helping with improving and Ronny for his general advise. best, holger krekel Changes between 2.0.1 and 2.0.2 ---------------------------------------------- - tackle issue32 - speed up test runs of very quick test functions by reducing the relative overhead - fix issue30 - extended xfail/skipif handling and improved reporting. If you have a syntax error in your skip/xfail expressions you now get nice error reports. Also you can now access module globals from xfail/skipif expressions so that this for example works now:: import pytest import mymodule @pytest.mark.skipif("mymodule.__version__[0] == "1") def test_function(): pass This will not run the test function if the module's version string does not start with a "1". Note that specifying a string instead of a boolean expressions allows py.test to report meaningful information when summarizing a test run as to what conditions lead to skipping (or xfail-ing) tests. - fix issue28 - setup_method and pytest_generate_tests work together The setup_method fixture method now gets called also for test function invocations generated from the pytest_generate_tests hook. - fix issue27 - collectonly and keyword-selection (-k) now work together Also, if you do "py.test --collectonly -q" you now get a flat list of test ids that you can use to paste to the py.test commandline in order to execute a particular test. - fix issue25 avoid reported problems with --pdb and python3.2/encodings output - fix issue23 - tmpdir argument now works on Python3.2 and WindowsXP Starting with Python3.2 os.symlink may be supported. By requiring a newer py lib version the py.path.local() implementation acknowledges this. - fixed typos in the docs (thanks Victor Garcia, Brianna Laugher) and particular thanks to Laura Creighton who also revieved parts of the documentation. - fix slighly wrong output of verbose progress reporting for classes (thanks Amaury) - more precise (avoiding of) deprecation warnings for node.Class|Function accesses - avoid std unittest assertion helper code in tracebacks (thanks Ronny)

1 0

Re: [py-dev] New issue 1 in pytest-cov: Allow branch coverage
by meme dough March 6, 2011

March 6, 2011

Hi, Thanks Ned. That's pretty much what I've been thinking. As much as I'd like to have branch coverage available easily it does raise all the pass through issues. The command line options I think are very minimal and nice. --cov set's a source tree root, can have any number. --cov-report produces a report, can have any number. --cov-config set's the config file for full control over all coverage features. :) On 4 March 2011 00:21, Ned Batchelder <ned(a)nedbatchelder.com> wrote: > One of the reasons I implemented the config file support was specifically to > solve the "pass-through UI" problem with test runners. Adding a feature to > coverage.py was useless unless the test runner or plugin also supported the > switch. I'm all in favor of getting off that treadmill by encouraging > people to use the config file to control coverage.py. I'd use a message of, > "the command line is for simple use, once you are using coverage.py in a > more sophisticated way, create a config file." > > --Ned. > > On 2/26/2011 2:42 AM, meme dough wrote: >> >> Hi, >> >> I'm cross posting to pytest and coverage to discuss. >> >> At the start pytest-cov had command line options to control all of >> coverage. >> >> Talk on pytest mailing list there was concern about so many options >> and that it should have minimal options. That way py.test --help >> output is shorter and more useful as a single plugin doesn't output >> huge amounts of text hiding the other plugin help text. Also it could >> be fully controlled through coverage config file. >> >> We could add a small number of options in but I'd like to get others >> opinions first. Plus how do we decide which ones should be in or not. >> If we add branch then maybe someone else wants timid or cover pylib. >> >> Another thing to consider is should coverage have env vars for all of >> it's options. That way it could be controlled from the command line >> but independent of the test runner (pytest, nose, unittest2) and so it >> would work with all of them. Something like: >> >> {{{ >> COVERAGE_BRANCH=1 py.test --cov-report term-missing --cov myproj tests/ >> }}} >> >> Or the following as coverage already reads COVERAGE_OPTIONS to get timid >> option: >> >> {{{ >> COVERAGE_OPTIONS="--branch" py.test --cov-report term-missing --cov >> myproj tests/ >> }}} >> >> These days I quite like the minimal options with full control through >> config file. But I do see that it is nice to control through the >> command line which is why I had full control of every option at the >> very beginning (in fact I had command line option, env var, then >> config file for each option). >> >> Any ideas on leaving it as it is? Adding small number of options to >> pytest-cov? Adding env vars to coverage to control independently so >> works will all test runners? >> >> :) >> >> On 26 February 2011 08:36, Bitbucket<issues-noreply(a)bitbucket.org> wrote: >>> >>> --- you can reply above this line --- >>> >>> New issue 1: Allow branch coverage >>> https://bitbucket.org/memedough/pytest-cov/issue/1/allow-branch-coverage >>> >>> jimslack on Fri, 25 Feb 2011 22:36:33 +0100: >>> >>> Description: >>> I know it is possible to activate branch coverage using a config file, >>> but I'd like to use branch coverage with a command-line option. Perhaps >>> something like: >>> >>> {{{ >>> py.test --cov-report term-missing --cov --branch myproj tests/ >>> }}} >>> >>> or >>> >>> {{{ >>> py.test --cov-report term-missing --covbranch myproj tests/ >>> }}} >>> >>> Another possibility would be to make branch coverage the default. >>> >>> Thanks. >>> >>> >>> >>> >>> -- >>> >>> This is an issue notification from bitbucket.org. You are receiving >>> this either because you are the owner of the issue, or you are >>> following the issue. >>> >

1 0

[py-dev] pytest for python 3 ?
by James Mills March 1, 2011

March 1, 2011

Has this happened yet ? We want to use Python 3.2 at work (on our product) and also pytest (it's the best testing framework!) cheers James -- -- James Mills -- -- "Problems are solved by method"

2 1