Mailman 3 December 2012 - pytest-dev

Test parameterization via Fixtures
by Daniel Neuhäuser Dec. 29, 2012

Dec. 29, 2012

Hello, I have recently re-discovered py.test searching for testing solution for a new project and have started using it in said project. However I have also noticed that fixtures are rather heavily constrained, most importantly in that a fixture cannot have multiple values. To give an example one of the things that are part of my new project is password hashing. What I would like to be able to do with fixtures is have a fixture `password` that creates several random passwords and I would like a test that uses the fixture to be called with each of these passwords. Furthermore I would like to be able to create other fixtures such as a `pw_hash` fixture that depends on the `password` fixture and creates hashes for each password. I would also like to be able to write tests that take multiple fixtures and while being parametrized "naturally". An example would be a function `test_pw_hash_verify(pw_hash, password)` that takes both two fixtures `pw_hash` and `password`. In this case I would like `test_pw_hash_verify` to be called with password and the corresponding `pw_hash`. In more abstract terms I would like to have a fixture system for which the following rules hold: - A fixture `fixture()` represents set of values. - A fixture `fixture(a)` is a n:n (n:m?) mapping of the values represented by `a` to a new set of values. - A fixture `fixture(*fixtures)` is a n:m mapping of the cartesian product of the values represented by each in `fixtures` if they are all independent from one another. If any fixture `a` in `fixtures` is dependent upon one or more other fixtures `b_` in `fixtures` `fixture(*fixtures)` shall only be called with the values of `a` and `b_` so that the values `a` was called with are equal to the ones `fixtures(*fixtures)` is called with. I have created a very simple implementation of this algorithm to play around with as an example, which I hope will better illustrate the idea: https://gist.github.com/4408054 What I would like to know is whether there is any convenient way to do that now without abusing py.test significantly, whether anyone is working on this or something equivalent already or if there is some huge flaw in this idea, that I am missing at the moment.

2 1

[py-dev] this list moving to pytest-dev@python.org
by holger krekel Dec. 28, 2012

Dec. 28, 2012

Hi folks, this list is going to move to move python.org, probably even today. The new list address is: pytest-dev(a)python.org the commit list will be: pytest-commit(a)python.org The old addresses (py-dev@codespeak.net,py-svn@codespeak.net) will continue to function so the move shouldn't be disruptive. I am going to send another mail once the move is complete. best, holger

1 1

[py-dev] yield-tests and fixtures: should they have a future?
by holger krekel Dec. 21, 2012

Dec. 21, 2012

Hi testing folks, hi Jason, i am looking at some recent pytest issues and would like to simplify pytest's internal fixture handling. One obstacle/complication are yield-tests, i.e. the style of producing tests with a generator:: def test_gen(self): for x in range(10): yield check, x This currently produces 10 test items, calling the "check" function with the respective parameter. This by itself is not a big deal to support. However, some people expect fixtures/setup_function/method functions to execute before the generator does and this mixes the collection with the runtest phase. Unfortunately, nose also supports this notion although i am wondering how nose2 is going to deal with it as Jason also plans to separate collection from running. So i am thinking about dropping fixture/setup support for yield-tests in pytest but OTOH i'd like to keep backward and nose compatibility. As far as pytest is concerned, it has many others means of parametrization independently from yield, see e. g. http://pytest.org/latest/fixture.html#fixture-parametrize and http://pytest.org/latest/parametrize.html and more and more people are starting to use those (pytest does not document or recommend yield for 1-2 years now). If anyone has any input/thoughts on this, please shoot. best, holger

2 1

[py-dev] reversing fixture/xunit setup call order?
by holger krekel Dec. 20, 2012

Dec. 20, 2012

Hi all, Currently, if you define e.g. an autouse fixture function it is going to be called _after_ the xUnit setup functions. This is especially surprising when you do a session-scoped autouse fixture. I am wondering if we could reverse the order, i.e. call fixture functions (including autouse-fixtures of course) ahead of xUnit setup methods. any thoughts? holger

3 4

[py-dev] Making py.test ignore an __init__.py
by lahwran Dec. 17, 2012

Dec. 17, 2012

Hi, I've got a bit of a problem related to how pytest determines the fully qualified name for a module. I have a django 1.3 layout project which has an __init__.py at its root, due to oddities in how django functions. so I have something like this layout: ~/project_venv/ # containing the virtualenv for the project ~/project_venv/project/ # the directory I cd to when I work ~/project_venv/project/.git ~/project_venv/project/__init__.py # because django's stuff doesn't work without it ~/project_venv/project/applications/ ~/project_venv/project/applications/__init__.py ~/project_venv/project/applications/projectapp/ ~/project_venv/project/applications/projectapp/__init__.py ~/project_venv/project/applications/projectapp/some_file.py ~/project_venv/project/applications/projectapp/some_other_file.py ~/project_venv/project/applications/projectapp/tests/ ~/project_venv/project/applications/projectapp/tests/__init__.py ~/project_venv/project/applications/projectapp/tests/test_some_file.py ~/project_venv/project/applications/projectapp/tests/test_some_other_file.py in test_some_file.py, I have something like: from applications.projectapp.some_file import Herp, Derp, doop and in test_some_other_file.py, I have something similar: from applications.projectapp.some_other_file import Herk, Derk, foo from applications.projectapp.tests.test_some_file import SomeTestUtilityThingy however, because project/ has an __init__.py, when pytest does its collection, rather than importing applications.projectapp.tests.test_some_file, it imports, project.applications.projectapp.tests.test_some_file! worse, when test_some_other_file.py imports test_some_file, it creates a *duplicate* - the import creates an applications.projectapp.tests.test_some_file when project.applications.projectapp.tests.test_some_file already existed. so, my question is: how can I tell pytest to please just pretend that __init__.py doesn't exist? right now I'm using conftest.py to shove dirname(__file__) onto sys.path, but that was written before I thought to check if there was an __init__.py in the root of the project.

3 3

[py-dev] Tricky parametrization problem
by Tomi Pieviläinen Dec. 17, 2012

Dec. 17, 2012

I have a a simulation that has different kinds of forces to simulate, each a different function. I have implemented those functions in several modules: baseline python (Numpy), C (accessed via ctypes) and PyCUDA. I need to make sure I get the same results from the optimized versions compared to the slow but correct python version. Later I will be adding modules/implementations for cython, numba etc. too. Not all modules are always available. For example cuda depends on the current system hardware, so I need to skip some tests. For each opimized module I want to take the func1, func2 and func3, and compare them to the baseline func1, func2 and func3. So I was hoping I could do test functions with @parametrize(('basefunc', 'fastfunc'), [(func1, mod.func1), (func2, mod.func2), (func3, mod.func3)]) @parametrize(('x, 'y'), [(numpy.randn(8), numpy.randn(8)), (numpy.randn(16), numpy.randn(16)), ... ]) def test_random_arrays(x, y, basefunc, fastfunc) assert_arrays_almost_equal(basefunc(x, y), fastfunc(x, y)) and have the mod parametrized. But I can't use @parametrize for that, since the modules can't be always imported. So I was hoping then to have a fixture for the module or the fastfunc, that would call pytest.skip() if the import fails, but I would need the fixture return a bunch of parameters for just one call which I've understood is not possible. I'm not really sure if what I'm aiming at makes any sense, but hopefully someone has an idea on how to do this in a clean way. -- Tomi Pieviläinen, +358 400 487 504 A: Because it disrupts the natural way of thinking. Q: Why is top posting frowned upon?

2 1

Re: [py-dev] reversing fixture/xunit setup call order?
by holger krekel Dec. 16, 2012

Dec. 16, 2012

On Sun, Dec 16, 2012 at 13:21 +0100, Ronny Pfannschmidt wrote: > sounds like the correct curse of action else legacy tests cant be > integrated with fixtures propperly "curse of action" ... like that one :) > i wonder if we should go as far as allowing fixtures to be arguments > to pytest xunit test functions that'd be tricky at least for setup_module and setup_method/function which support a positional argument. Note that you can easily turn your setup function into one that accepts fixtures: @pytest.fixture def setup_method(self, request, tmpdir, ...): ... In this case there is no positional argument but you can get the current function under test via ``request.function``. I think it's clearer to add that extra line. holger > On 12/16/2012 12:23 PM, holger krekel wrote: > >Hi all, > > > >Currently, if you define e.g. an autouse fixture function it is going to > >be called _after_ the xUnit setup functions. This is especially > >surprising when you do a session-scoped autouse fixture. I am wondering > >if we could reverse the order, i.e. call fixture functions (including > >autouse-fixtures of course) ahead of xUnit setup methods. > > > >any thoughts? > > > >holger > >_______________________________________________ > >py-dev mailing list > >py-dev(a)codespeak.net > >http://codespeak.net/mailman/listinfo/py-dev >

2 2

[py-dev] ANN: pytest-django 2.0.0 released - a better Django test runner
by Andreas Pelme Dec. 4, 2012

Dec. 4, 2012

Hi, We are pleased to announce the availability of pytest-django 2.0.0. This release is rewritten to make use of fixture API introduced in py.test 2.3, which fixes a lot of bugs and edge cases in previous releases. Get it directly from PyPI or install with pip: pip install -U pytest-django pytest is an innovative testing tool for Python programs. pytest-django is a plugin that provides Django integration and helpers. Improvements over Django's standard test runner includes test database re-use, easier selection of running a subset of test cases. Existing test suites based on Django's own test runner should be compatible out of the box. Documentation & how to get started: http://pytest-django.readthedocs.org/en/latest/ Full changelog: http://pytest-django.readthedocs.org/en/latest/changelog.html Report bugs here: https://github.com/pelme/pytest_django/issues py.test documentation: http://pytest.org/latest/ We aim to make testing Django applications as easy as possible: Please submit suggestions and ideas in the issue tracker! Best regards Andreas Pelme

1 0

[py-dev] steps to include a new plugin
by Adam Goucher Dec. 3, 2012

Dec. 3, 2012

If I wanted to try and add https://github.com/adamgoucher/pytest-marks to the main pytest distribution. Is there a process for consideration, code style rules, etc.? The idea of this plugin is to allow script creators to not have to do @pytest.mark.red @pytest.mark.green @pytest.mark.blue @pytest.mark.black @pytest.mark.orange @pytest.mark.pink def some_test_method(self): # some check-y stuff but rather @pytest.marks('red', 'green', 'blue', 'black', 'orange', 'pink') def some_test_method(self): # some check-y stuff -adam

4 5