Test parameterization via Fixtures
Hello, I have recently re-discovered py.test searching for testing solution for a new project and have started using it in said project. However I have also noticed that fixtures are rather heavily constrained, most importantly in that a fixture cannot have multiple values. To give an example one of the things that are part of my new project is password hashing. What I would like to be able to do with fixtures is have a fixture `password` that creates several random passwords and I would like a test that uses the fixture to be called with each of these passwords. Furthermore I would like to be able to create other fixtures such as a `pw_hash` fixture that depends on the `password` fixture and creates hashes for each password. I would also like to be able to write tests that take multiple fixtures and while being parametrized "naturally". An example would be a function `test_pw_hash_verify(pw_hash, password)` that takes both two fixtures `pw_hash` and `password`. In this case I would like `test_pw_hash_verify` to be called with password and the corresponding `pw_hash`. In more abstract terms I would like to have a fixture system for which the following rules hold: - A fixture `fixture()` represents set of values. - A fixture `fixture(a)` is a n:n (n:m?) mapping of the values represented by `a` to a new set of values. - A fixture `fixture(*fixtures)` is a n:m mapping of the cartesian product of the values represented by each in `fixtures` if they are all independent from one another. If any fixture `a` in `fixtures` is dependent upon one or more other fixtures `b_` in `fixtures` `fixture(*fixtures)` shall only be called with the values of `a` and `b_` so that the values `a` was called with are equal to the ones `fixtures(*fixtures)` is called with. I have created a very simple implementation of this algorithm to play around with as an example, which I hope will better illustrate the idea: https://gist.github.com/4408054 What I would like to know is whether there is any convenient way to do that now without abusing py.test significantly, whether anyone is working on this or something equivalent already or if there is some huge flaw in this idea, that I am missing at the moment.
Hi Daniel, On Sat, Dec 29, 2012 at 18:11 +0100, Daniel Neuhäuser wrote:
Hello,
I have recently re-discovered py.test searching for testing solution for a new project and have started using it in said project. However I have also noticed that fixtures are rather heavily constrained, most importantly in that a fixture cannot have multiple values.
To give an example one of the things that are part of my new project is password hashing. What I would like to be able to do with fixtures is have a fixture `password` that creates several random passwords and I would like a test that uses the fixture to be called with each of these passwords.
Did you check out http://pytest.org/latest/fixture.html#parametrizing-a-fixture yet ?
Furthermore I would like to be able to create other fixtures such as a `pw_hash` fixture that depends on the `password` fixture and creates hashes for each password.
That is directly doable like so: @pytest.fixture(params=["pass1", "pass2"] def password(request): return request.param @pytest.fixture def pw_hash(password): return hash(password) This would make all tests that use "pw_hash" run twice.
I would also like to be able to write tests that take multiple fixtures and while being parametrized "naturally". An example would be a function `test_pw_hash_verify(pw_hash, password)` that takes both two fixtures `pw_hash` and `password`. In this case I would like `test_pw_hash_verify` to be called with password and the corresponding `pw_hash`.
This so far sounds like the current scheme.
In more abstract terms I would like to have a fixture system for which the following rules hold:
- A fixture `fixture()` represents set of values. - A fixture `fixture(a)` is a n:n (n:m?) mapping of the values represented by `a` to a new set of values. - A fixture `fixture(*fixtures)` is a n:m mapping of the cartesian product of the values represented by each in `fixtures` if they are all independent from one another. If any fixture `a` in `fixtures` is dependent upon one or more other fixtures `b_` in `fixtures` `fixture(*fixtures)` shall only be called with the values of `a` and `b_` so that the values `a` was called with are equal to the ones `fixtures(*fixtures)` is called with.
I have created a very simple implementation of this algorithm to play around with as an example, which I hope will better illustrate the idea: https://gist.github.com/4408054
This particular API is not supported, i.e. yielding from fixtures. pytest follows a design decision that collection of tests should be independent from running or setting up tests. At collection time pytest has a view on the complete fixture tree neccessary for running the tests and re-orders tests to be grouped by fixtures/scopes. Yielding fixtures would not allow either.
What I would like to know is whether there is any convenient way to do that now without abusing py.test significantly, whether anyone is working on this or something equivalent already or if there is some huge flaw in this idea, that I am missing at the moment.
Let us know if the above methods work for you or what is missing! cheers, holger
_______________________________________________ Pytest-dev mailing list Pytest-dev@python.org http://mail.python.org/mailman/listinfo/pytest-dev
participants (2)
-
Daniel Neuhäuser -
holger krekel