From anto.cuni at gmail.com Mon Dec 3 20:58:14 2012 From: anto.cuni at gmail.com (Antonio Cuni) Date: Mon, 03 Dec 2012 20:58:14 +0100 Subject: [py-dev] steps to include a new plugin In-Reply-To: <20121127222753.GA1720@merlinux.eu> References: <20121127222753.GA1720@merlinux.eu> Message-ID: <50BD0456.4080109@gmail.com> Hi Holger, On 11/27/2012 11:27 PM, holger krekel wrote: >> @pytest.marks('red', 'green', 'blue', 'black', 'orange', 'pink') >> def some_test_method(self): >> # some check-y stuff > > I can see how each mark consuming a line can be cumbersome. I wonder if > there would be a way to have less line noise, however. For example:: > > @pytest.mark.red.green.blue.black.orange.pink > def test_method(...): > ... I think that using the dot is a bad idea, because it conflicts with the standard meaning that it has in Python, i.e. to "navigate" between layers/levels. If we really want some syntactic sugar, the following is ugly but at least is not as confusing as the one above: m = pytest.mark @m.red/m.green/m.blue def test_method(...): ... Even if I have to admit that in my explicit-is-better-than-implicit opinion, the best ons is still the first. ciao, Anto From andreas at pelme.se Tue Dec 4 09:08:50 2012 From: andreas at pelme.se (Andreas Pelme) Date: Tue, 4 Dec 2012 09:08:50 +0100 Subject: [py-dev] ANN: pytest-django 2.0.0 released - a better Django test runner Message-ID: <9C371C95CA25458F9B0B3541FAF03DE9@pelme.se> Hi, We are pleased to announce the availability of pytest-django 2.0.0. This release is rewritten to make use of fixture API introduced in py.test 2.3, which fixes a lot of bugs and edge cases in previous releases. Get it directly from PyPI or install with pip: pip install -U pytest-django pytest is an innovative testing tool for Python programs. pytest-django is a plugin that provides Django integration and helpers. Improvements over Django's standard test runner includes test database re-use, easier selection of running a subset of test cases. Existing test suites based on Django's own test runner should be compatible out of the box. Documentation & how to get started: http://pytest-django.readthedocs.org/en/latest/ Full changelog: http://pytest-django.readthedocs.org/en/latest/changelog.html Report bugs here: https://github.com/pelme/pytest_django/issues py.test documentation: http://pytest.org/latest/ We aim to make testing Django applications as easy as possible: Please submit suggestions and ideas in the issue tracker! Best regards Andreas Pelme From holger at merlinux.eu Sun Dec 16 12:23:00 2012 From: holger at merlinux.eu (holger krekel) Date: Sun, 16 Dec 2012 11:23:00 +0000 Subject: [py-dev] reversing fixture/xunit setup call order? Message-ID: <20121216112300.GP26599@merlinux.eu> Hi all, Currently, if you define e.g. an autouse fixture function it is going to be called _after_ the xUnit setup functions. This is especially surprising when you do a session-scoped autouse fixture. I am wondering if we could reverse the order, i.e. call fixture functions (including autouse-fixtures of course) ahead of xUnit setup methods. any thoughts? holger From holger at merlinux.eu Sun Dec 16 20:24:18 2012 From: holger at merlinux.eu (holger krekel) Date: Sun, 16 Dec 2012 19:24:18 +0000 Subject: [py-dev] reversing fixture/xunit setup call order? In-Reply-To: <50CDBCDC.4050304@gmx.de> References: <20121216112300.GP26599@merlinux.eu> <50CDBCDC.4050304@gmx.de> Message-ID: <20121216192418.GR26599@merlinux.eu> On Sun, Dec 16, 2012 at 13:21 +0100, Ronny Pfannschmidt wrote: > sounds like the correct curse of action else legacy tests cant be > integrated with fixtures propperly "curse of action" ... like that one :) > i wonder if we should go as far as allowing fixtures to be arguments > to pytest xunit test functions that'd be tricky at least for setup_module and setup_method/function which support a positional argument. Note that you can easily turn your setup function into one that accepts fixtures: @pytest.fixture def setup_method(self, request, tmpdir, ...): ... In this case there is no positional argument but you can get the current function under test via ``request.function``. I think it's clearer to add that extra line. holger > On 12/16/2012 12:23 PM, holger krekel wrote: > >Hi all, > > > >Currently, if you define e.g. an autouse fixture function it is going to > >be called _after_ the xUnit setup functions. This is especially > >surprising when you do a session-scoped autouse fixture. I am wondering > >if we could reverse the order, i.e. call fixture functions (including > >autouse-fixtures of course) ahead of xUnit setup methods. > > > >any thoughts? > > > >holger > >_______________________________________________ > >py-dev mailing list > >py-dev at codespeak.net > >http://codespeak.net/mailman/listinfo/py-dev > From lahwran0 at gmail.com Sun Dec 16 21:49:34 2012 From: lahwran0 at gmail.com (lahwran) Date: Sun, 16 Dec 2012 13:49:34 -0700 Subject: [py-dev] reversing fixture/xunit setup call order? In-Reply-To: <20121216192418.GR26599@merlinux.eu> References: <20121216112300.GP26599@merlinux.eu> <50CDBCDC.4050304@gmx.de> <20121216192418.GR26599@merlinux.eu> Message-ID: I think it's best to think of the xunit-style setup and teardown as part of the actual test - as in, if you're blindly translating this: class SomethingTest(TestCase): def setUp(self): self.doop = 1 def tearDown(self): del self.doop def test_something(self): assert self.doop == 1 def test_something_else(self): assert self.doop != 2 you'd get this: class TestSomething(object): def setUp(self): self.doop = 1 def tearDown(self): del self.doop def test_something(self): self.setUp() try: assert self.doop == 1 finally: self.tearDown() def test_something_else(self): self.setUp() try: assert self.doop != 2 finally: self.tearDown() I'm not sure how to demonstrate an equivalent to setUpClass, but my point is that fixtures shouldn't even be able to tell that setup and teardown aren't part of the actual test method unless they look for it. As for allowing funcargs to the setup functions, I think marking them as @pytest.fixture(autouse=True) would be great. I do think that it'd be more intuitive for people who are used to xunit style if the @pytest.setup thing being discussed on the other thread was made available. On Sun, Dec 16, 2012 at 12:24 PM, holger krekel wrote: > On Sun, Dec 16, 2012 at 13:21 +0100, Ronny Pfannschmidt wrote: > > sounds like the correct curse of action else legacy tests cant be > > integrated with fixtures propperly > > "curse of action" ... like that one :) > > > i wonder if we should go as far as allowing fixtures to be arguments > > to pytest xunit test functions > > that'd be tricky at least for setup_module and setup_method/function which > support a positional argument. Note that you can easily turn your setup > function into one that accepts fixtures: > > @pytest.fixture > def setup_method(self, request, tmpdir, ...): > ... > > In this case there is no positional argument but you can get the > current function under test via ``request.function``. > I think it's clearer to add that extra line. > > holger > > > On 12/16/2012 12:23 PM, holger krekel wrote: > > >Hi all, > > > > > >Currently, if you define e.g. an autouse fixture function it is going to > > >be called _after_ the xUnit setup functions. This is especially > > >surprising when you do a session-scoped autouse fixture. I am wondering > > >if we could reverse the order, i.e. call fixture functions (including > > >autouse-fixtures of course) ahead of xUnit setup methods. > > > > > >any thoughts? > > > > > >holger > > >_______________________________________________ > > >py-dev mailing list > > >py-dev at codespeak.net > > >http://codespeak.net/mailman/listinfo/py-dev > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From holger at merlinux.eu Sun Dec 16 21:55:22 2012 From: holger at merlinux.eu (holger krekel) Date: Sun, 16 Dec 2012 20:55:22 +0000 Subject: [py-dev] reversing fixture/xunit setup call order? In-Reply-To: References: <20121216112300.GP26599@merlinux.eu> <50CDBCDC.4050304@gmx.de> <20121216192418.GR26599@merlinux.eu> Message-ID: <20121216205522.GW26599@merlinux.eu> On Sun, Dec 16, 2012 at 13:49 -0700, lahwran wrote: > I think it's best to think of the xunit-style setup and teardown as part of > the actual test - as in, if you're blindly translating this: > > class SomethingTest(TestCase): > def setUp(self): > self.doop = 1 > > def tearDown(self): > del self.doop > > def test_something(self): > assert self.doop == 1 > > def test_something_else(self): > assert self.doop != 2 > > you'd get this: > > > class TestSomething(object): > def setUp(self): > self.doop = 1 > > def tearDown(self): > del self.doop > > def test_something(self): > self.setUp() > try: > assert self.doop == 1 > finally: > self.tearDown() > > def test_something_else(self): > self.setUp() > try: > assert self.doop != 2 > finally: > self.tearDown() > > I'm not sure how to demonstrate an equivalent to setUpClass, but my point > is that fixtures shouldn't even be able to tell that setup and teardown > aren't part of the actual test method unless they look for it. That speaks for executing fixtures ahead of xunit methods, right? > As for allowing funcargs to the setup functions, I think marking them as > @pytest.fixture(autouse=True) would be great. I do think that it'd be more > intuitive for people who are used to xunit style if the @pytest.setup thing > being discussed on the other thread was made available. Do you know that you can use @pytest.fixture(autouse=True) on xUnit setup methods already with the current release? We could think about introducing a ``@pytest.setup`` shortcut, but i don't think it's worth it. best, holger > On Sun, Dec 16, 2012 at 12:24 PM, holger krekel wrote: > > > On Sun, Dec 16, 2012 at 13:21 +0100, Ronny Pfannschmidt wrote: > > > sounds like the correct curse of action else legacy tests cant be > > > integrated with fixtures propperly > > > > "curse of action" ... like that one :) > > > > > i wonder if we should go as far as allowing fixtures to be arguments > > > to pytest xunit test functions > > > > that'd be tricky at least for setup_module and setup_method/function which > > support a positional argument. Note that you can easily turn your setup > > function into one that accepts fixtures: > > > > @pytest.fixture > > def setup_method(self, request, tmpdir, ...): > > ... > > > > In this case there is no positional argument but you can get the > > current function under test via ``request.function``. > > I think it's clearer to add that extra line. > > > > holger > > > > > On 12/16/2012 12:23 PM, holger krekel wrote: > > > >Hi all, > > > > > > > >Currently, if you define e.g. an autouse fixture function it is going to > > > >be called _after_ the xUnit setup functions. This is especially > > > >surprising when you do a session-scoped autouse fixture. I am wondering > > > >if we could reverse the order, i.e. call fixture functions (including > > > >autouse-fixtures of course) ahead of xUnit setup methods. > > > > > > > >any thoughts? > > > > > > > >holger > > > >_______________________________________________ > > > >py-dev mailing list > > > >py-dev at codespeak.net > > > >http://codespeak.net/mailman/listinfo/py-dev > > > > > > > > _______________________________________________ > py-dev mailing list > py-dev at codespeak.net > http://codespeak.net/mailman/listinfo/py-dev From lahwran0 at gmail.com Mon Dec 17 07:34:22 2012 From: lahwran0 at gmail.com (lahwran) Date: Sun, 16 Dec 2012 23:34:22 -0700 Subject: [py-dev] Making py.test ignore an __init__.py Message-ID: Hi, I've got a bit of a problem related to how pytest determines the fully qualified name for a module. I have a django 1.3 layout project which has an __init__.py at its root, due to oddities in how django functions. so I have something like this layout: ~/project_venv/ # containing the virtualenv for the project ~/project_venv/project/ # the directory I cd to when I work ~/project_venv/project/.git ~/project_venv/project/__init__.py # because django's stuff doesn't work without it ~/project_venv/project/applications/ ~/project_venv/project/applications/__init__.py ~/project_venv/project/applications/projectapp/ ~/project_venv/project/applications/projectapp/__init__.py ~/project_venv/project/applications/projectapp/some_file.py ~/project_venv/project/applications/projectapp/some_other_file.py ~/project_venv/project/applications/projectapp/tests/ ~/project_venv/project/applications/projectapp/tests/__init__.py ~/project_venv/project/applications/projectapp/tests/test_some_file.py ~/project_venv/project/applications/projectapp/tests/test_some_other_file.py in test_some_file.py, I have something like: from applications.projectapp.some_file import Herp, Derp, doop and in test_some_other_file.py, I have something similar: from applications.projectapp.some_other_file import Herk, Derk, foo from applications.projectapp.tests.test_some_file import SomeTestUtilityThingy however, because project/ has an __init__.py, when pytest does its collection, rather than importing applications.projectapp.tests.test_some_file, it imports, project.applications.projectapp.tests.test_some_file! worse, when test_some_other_file.py imports test_some_file, it creates a *duplicate* - the import creates an applications.projectapp.tests.test_some_file when project.applications.projectapp.tests.test_some_file already existed. so, my question is: how can I tell pytest to please just pretend that __init__.py doesn't exist? right now I'm using conftest.py to shove dirname(__file__) onto sys.path, but that was written before I thought to check if there was an __init__.py in the root of the project. -------------- next part -------------- An HTML attachment was scrubbed... URL: From holger at merlinux.eu Mon Dec 17 10:18:41 2012 From: holger at merlinux.eu (holger krekel) Date: Mon, 17 Dec 2012 09:18:41 +0000 Subject: [py-dev] getting syspath handling right (was Re: making py.test ignore an __init__.py) In-Reply-To: References: Message-ID: <20121217091841.GX26599@merlinux.eu> Hi lahwran, all, On Sun, Dec 16, 2012 at 23:34 -0700, lahwran wrote: > Hi, I've got a bit of a problem related to how pytest determines the fully > qualified name for a module. I have a django 1.3 layout project which has > an __init__.py at its root, due to oddities in how django functions. so I > have something like this layout: > > ~/project_venv/ # containing the virtualenv for the project > ~/project_venv/project/ # the directory I cd to when I work > ~/project_venv/project/.git > ~/project_venv/project/__init__.py # because django's stuff doesn't work > without it > ~/project_venv/project/applications/ > ~/project_venv/project/applications/__init__.py > ~/project_venv/project/applications/projectapp/ > ~/project_venv/project/applications/projectapp/__init__.py > ~/project_venv/project/applications/projectapp/some_file.py > ~/project_venv/project/applications/projectapp/some_other_file.py > ~/project_venv/project/applications/projectapp/tests/ > ~/project_venv/project/applications/projectapp/tests/__init__.py > ~/project_venv/project/applications/projectapp/tests/test_some_file.py > ~/project_venv/project/applications/projectapp/tests/test_some_other_file.py > > in test_some_file.py, I have something like: > > from applications.projectapp.some_file import Herp, Derp, doop > > and in test_some_other_file.py, I have something similar: > > from applications.projectapp.some_other_file import Herk, Derk, foo > from applications.projectapp.tests.test_some_file import > SomeTestUtilityThingy > > however, because project/ has an __init__.py, when pytest does its > collection, rather than importing > applications.projectapp.tests.test_some_file, it imports, > project.applications.projectapp.tests.test_some_file! worse, when > test_some_other_file.py imports test_some_file, it creates a *duplicate* - > the import creates an applications.projectapp.tests.test_some_file when > project.applications.projectapp.tests.test_some_file already existed. evil. If it would be some arbitrary project promoting this strange __init__.py file practice i'd be inclined to say "fix it". If a project like Django really promotes this, then i guess we have to deal with it :/ > so, my question is: how can I tell pytest to please just pretend that > __init__.py doesn't exist? right now I'm using conftest.py to shove > dirname(__file__) onto sys.path, but that was written before I thought to > check if there was an __init__.py in the root of the project. It's tricky business to get sys-path manipulations workable in all the different kind of real life situations. I think nose just adds the dir of the test module to sys.path and imports it, and if another test module with the same basename exists in a different directories unloads the module. Or maybe it even always performs reloading, not sure. In nose2 this mechanism is to be dropped, anyway. In python3 unittest discovery requires a "toplevel" directory setting. If you don't set it from the command line the current working dir is used. Unless i am missing something that makes importing between test modules rather fragile. pytest always tries to import under a fully qualified name by walking the directories up that contain an __init__.py. This avoids the reloading business (which i think is not a good idea, nose2/Jason seem to agree) and, unlike unittest, it gives test modules a reliable cross-importing behaviour. Given the above problem, I think we should refine the pytest algorithm to also take existing sys.path settings into account and stop going further up if we find one. In your situation it would stop at "projects/" even though it contains an __init__.py file which would otherwise lead it to go further up. With python3.3 we need to also allow no __init__.py files at all (the new namespace import stuff) and just check directly for a sys.path that fits. Sounds like a plan? Note that i was also considering inifile-settings or new hooks to influence the behaviour. But this is not easy to get right in all situations. At least i didn't manage. Eventually i came up with the above refinement. It's anyway better if things work by default and you don't have to read lengthy explanations like this mail here :) best, holger From tomi.pievilainen at iki.fi Mon Dec 17 11:24:41 2012 From: tomi.pievilainen at iki.fi (Tomi =?iso-8859-1?Q?Pievil=E4inen?=) Date: Mon, 17 Dec 2012 12:24:41 +0200 Subject: [py-dev] Tricky parametrization problem Message-ID: <20121217102441.GK10711@hilla.kapsi.fi> I have a a simulation that has different kinds of forces to simulate, each a different function. I have implemented those functions in several modules: baseline python (Numpy), C (accessed via ctypes) and PyCUDA. I need to make sure I get the same results from the optimized versions compared to the slow but correct python version. Later I will be adding modules/implementations for cython, numba etc. too. Not all modules are always available. For example cuda depends on the current system hardware, so I need to skip some tests. For each opimized module I want to take the func1, func2 and func3, and compare them to the baseline func1, func2 and func3. So I was hoping I could do test functions with @parametrize(('basefunc', 'fastfunc'), [(func1, mod.func1), (func2, mod.func2), (func3, mod.func3)]) @parametrize(('x, 'y'), [(numpy.randn(8), numpy.randn(8)), (numpy.randn(16), numpy.randn(16)), ... ]) def test_random_arrays(x, y, basefunc, fastfunc) assert_arrays_almost_equal(basefunc(x, y), fastfunc(x, y)) and have the mod parametrized. But I can't use @parametrize for that, since the modules can't be always imported. So I was hoping then to have a fixture for the module or the fastfunc, that would call pytest.skip() if the import fails, but I would need the fixture return a bunch of parameters for just one call which I've understood is not possible. I'm not really sure if what I'm aiming at makes any sense, but hopefully someone has an idea on how to do this in a clean way. -- Tomi Pievil?inen, +358 400 487 504 A: Because it disrupts the natural way of thinking. Q: Why is top posting frowned upon? From holger at merlinux.eu Mon Dec 17 13:25:40 2012 From: holger at merlinux.eu (holger krekel) Date: Mon, 17 Dec 2012 12:25:40 +0000 Subject: [py-dev] Tricky parametrization problem In-Reply-To: <20121217102441.GK10711@hilla.kapsi.fi> References: <20121217102441.GK10711@hilla.kapsi.fi> Message-ID: <20121217122540.GZ26599@merlinux.eu> On Mon, Dec 17, 2012 at 12:24 +0200, Tomi Pievil?inen wrote: > I have a a simulation that has different kinds of forces to simulate, > each a different function. I have implemented those functions in > several modules: baseline python (Numpy), C (accessed via ctypes) and > PyCUDA. I need to make sure I get the same results from the optimized > versions compared to the slow but correct python version. Later I will > be adding modules/implementations for cython, numba etc. too. Not all > modules are always available. For example cuda depends on the current > system hardware, so I need to skip some tests. > > For each opimized module I want to take the func1, func2 and func3, > and compare them to the baseline func1, func2 and func3. So I was > hoping I could do test functions with > > @parametrize(('basefunc', 'fastfunc'), > [(func1, mod.func1), > (func2, mod.func2), > (func3, mod.func3)]) > @parametrize(('x, 'y'), > [(numpy.randn(8), numpy.randn(8)), > (numpy.randn(16), numpy.randn(16)), > ... ]) > def test_random_arrays(x, y, basefunc, fastfunc) > assert_arrays_almost_equal(basefunc(x, y), fastfunc(x, y)) > > > and have the mod parametrized. But I can't use @parametrize for > that, since the modules can't be always imported. > > So I was hoping then to have a fixture for the module or the fastfunc, > that would call pytest.skip() if the import fails, but I would need > the fixture return a bunch of parameters for just one call which > I've understood is not possible. > > I'm not really sure if what I'm aiming at makes any sense, but hopefully > someone has an idea on how to do this in a clean way. I came up with a rather simple approach how to handle "optional" imports. See here: http://pytest.org/latest/example/parametrize.html#indirect-parametrization-of-optional-implementations-imports Works for you? Btw, you might also want to checkout the pytest-quickcheck plugin: http://pypi.python.org/pypi/pytest-quickcheck/ best, holger From lahwran0 at gmail.com Mon Dec 17 17:46:26 2012 From: lahwran0 at gmail.com (lahwran) Date: Mon, 17 Dec 2012 09:46:26 -0700 Subject: [py-dev] getting syspath handling right (was Re: making py.test ignore an __init__.py) In-Reply-To: <20121217091841.GX26599@merlinux.eu> References: <20121217091841.GX26599@merlinux.eu> Message-ID: Apparently there is a way to remove the __init__.py and have django function, so rather than making pytest bend over backwards even farther, I'm going to remove the __init__.py. thank heavens for that. On Mon, Dec 17, 2012 at 2:18 AM, holger krekel wrote: > Hi lahwran, all, > > On Sun, Dec 16, 2012 at 23:34 -0700, lahwran wrote: > > Hi, I've got a bit of a problem related to how pytest determines the > fully > > qualified name for a module. I have a django 1.3 layout project which has > > an __init__.py at its root, due to oddities in how django functions. so I > > have something like this layout: > > > > ~/project_venv/ # containing the virtualenv for the project > > ~/project_venv/project/ # the directory I cd to when I work > > ~/project_venv/project/.git > > ~/project_venv/project/__init__.py # because django's stuff doesn't work > > without it > > ~/project_venv/project/applications/ > > ~/project_venv/project/applications/__init__.py > > ~/project_venv/project/applications/projectapp/ > > ~/project_venv/project/applications/projectapp/__init__.py > > ~/project_venv/project/applications/projectapp/some_file.py > > ~/project_venv/project/applications/projectapp/some_other_file.py > > ~/project_venv/project/applications/projectapp/tests/ > > ~/project_venv/project/applications/projectapp/tests/__init__.py > > ~/project_venv/project/applications/projectapp/tests/test_some_file.py > > > ~/project_venv/project/applications/projectapp/tests/test_some_other_file.py > > > > in test_some_file.py, I have something like: > > > > from applications.projectapp.some_file import Herp, Derp, doop > > > > and in test_some_other_file.py, I have something similar: > > > > from applications.projectapp.some_other_file import Herk, Derk, foo > > from applications.projectapp.tests.test_some_file import > > SomeTestUtilityThingy > > > > however, because project/ has an __init__.py, when pytest does its > > collection, rather than importing > > applications.projectapp.tests.test_some_file, it imports, > > project.applications.projectapp.tests.test_some_file! worse, when > > test_some_other_file.py imports test_some_file, it creates a *duplicate* > - > > the import creates an applications.projectapp.tests.test_some_file when > > project.applications.projectapp.tests.test_some_file already existed. > > evil. If it would be some arbitrary project promoting this strange > __init__.py file practice i'd be inclined to say "fix it". If a project > like Django really promotes this, then i guess we have to deal with it :/ > > > so, my question is: how can I tell pytest to please just pretend that > > __init__.py doesn't exist? right now I'm using conftest.py to shove > > dirname(__file__) onto sys.path, but that was written before I thought to > > check if there was an __init__.py in the root of the project. > > It's tricky business to get sys-path manipulations workable in all > the different kind of real life situations. I think nose just adds > the dir of the test module to sys.path and imports it, and if another > test module with the same basename exists in a different directories > unloads the module. Or maybe it even always performs reloading, not sure. > In nose2 this mechanism is to be dropped, anyway. > > In python3 unittest discovery requires a "toplevel" directory setting. > If you don't set it from the command line the current working dir is > used. Unless i am missing something that makes importing between > test modules rather fragile. > > pytest always tries to import under a fully qualified name by walking the > directories up that contain an __init__.py. This avoids the reloading > business (which i think is not a good idea, nose2/Jason seem to agree) > and, unlike unittest, it gives test modules a reliable cross-importing > behaviour. Given the above problem, I think we should refine the pytest > algorithm to also take existing sys.path settings into account and stop > going further up if we find one. In your situation it would stop at > "projects/" even though it contains an __init__.py file which would > otherwise lead it to go further up. > > With python3.3 we need to also allow no __init__.py files at all (the > new namespace import stuff) and just check directly for a sys.path that > fits. > > Sounds like a plan? Note that i was also considering inifile-settings > or new hooks to influence the behaviour. But this is not easy to get > right in all situations. At least i didn't manage. Eventually i came > up with the above refinement. It's anyway better if things work by > default and you don't have to read lengthy explanations like this > mail here :) > > best, > holger > -------------- next part -------------- An HTML attachment was scrubbed... URL: From carl at oddbird.net Mon Dec 17 18:23:45 2012 From: carl at oddbird.net (Carl Meyer) Date: Mon, 17 Dec 2012 10:23:45 -0700 Subject: [py-dev] getting syspath handling right (was Re: making py.test ignore an __init__.py) In-Reply-To: <20121217091841.GX26599@merlinux.eu> References: <20121217091841.GX26599@merlinux.eu> Message-ID: <50CF5521.7080000@oddbird.net> On 12/17/2012 02:18 AM, holger krekel wrote: > evil. If it would be some arbitrary project promoting this strange > __init__.py file practice i'd be inclined to say "fix it". If a project > like Django really promotes this, then i guess we have to deal with it :/ FWIW, Django does not (any longer) promote this silliness. Django 1.4+ handles sys.path sensibly, and a 1.3 project can easily be upgraded to the 1.4-style layout: https://docs.djangoproject.com/en/dev/releases/1.4/#updated-default-project-layout-and-manage-py So I don't think there's any need for pytest to handle this, people should just be pointed to that section in the docs and recommended to update their project. Carl From flub at devork.be Tue Dec 18 21:11:51 2012 From: flub at devork.be (Floris Bruynooghe) Date: Tue, 18 Dec 2012 21:11:51 +0100 Subject: [py-dev] reversing fixture/xunit setup call order? In-Reply-To: <20121216112300.GP26599@merlinux.eu> References: <20121216112300.GP26599@merlinux.eu> Message-ID: On 16 December 2012 12:23, holger krekel wrote: > Currently, if you define e.g. an autouse fixture function it is going to > be called _after_ the xUnit setup functions. This is especially > surprising when you do a session-scoped autouse fixture. I am wondering > if we could reverse the order, i.e. call fixture functions (including > autouse-fixtures of course) ahead of xUnit setup methods. > > any thoughts? Agreed, I think it would be a good idea to have at least autouse fixtures before the xUnit setup. Regards, Floris -- Debian GNU/Linux -- The Power of Freedom www.debian.org | www.gnu.org | www.kernel.org From holger at merlinux.eu Wed Dec 19 09:42:02 2012 From: holger at merlinux.eu (holger krekel) Date: Wed, 19 Dec 2012 08:42:02 +0000 Subject: [py-dev] reversing fixture/xunit setup call order? In-Reply-To: References: <20121216112300.GP26599@merlinux.eu> Message-ID: <20121219084201.GG26599@merlinux.eu> On Tue, Dec 18, 2012 at 21:11 +0100, Floris Bruynooghe wrote: > On 16 December 2012 12:23, holger krekel wrote: > > Currently, if you define e.g. an autouse fixture function it is going to > > be called _after_ the xUnit setup functions. This is especially > > surprising when you do a session-scoped autouse fixture. I am wondering > > if we could reverse the order, i.e. call fixture functions (including > > autouse-fixtures of course) ahead of xUnit setup methods. > > > > any thoughts? > > Agreed, I think it would be a good idea to have at least autouse > fixtures before the xUnit setup. However, i realize we also have scopes. And this is where any attempt to decide ordering between pytest versus xUnit fixtures seesm to break down: - We don't want setup_class execute after a function-scoped pytest fixture. - We don't want class-scoped pytest fixture execute after setup_method. Maybe, we could internally add autouse-fixtures at module/class/function scope which would look for setupX/teardownX and act accordingly. This way xUnit setup/teardown methods would appear as pytest fixtures. Here is an example how it could look like for a mix of xUnit/pytest class/function scoped autouse- and non-autouse fixtures: user-function found by -------------------------------------------------------------------------- ... autoclass(...) # @pytest.fixture(scope="class", autouse=True)) setup_class(cls) # internal @pytest.fixture("class", autouse=True) clsarg(...) # @pytest.fixture("class", autouse=False) funcfixture(...) # @pytest.fixture(scope="function", autouse=True) setup_method() # internal @pytest.fixture("function", autouse=True) arg1 # @pytest.fixture(scope="function", autouse=False)) ... test_function(arg1, clsarg) # teardowns execute in LIFO registration order Makes sense? Ideally, we could produce something like the above output with some command line option to help debugging. best, holger > Regards, > Floris > > > -- > Debian GNU/Linux -- The Power of Freedom > www.debian.org | www.gnu.org | www.kernel.org > From lahwran0 at gmail.com Thu Dec 20 00:27:19 2012 From: lahwran0 at gmail.com (lahwran) Date: Wed, 19 Dec 2012 16:27:19 -0700 Subject: [py-dev] reversing fixture/xunit setup call order? In-Reply-To: <20121219084201.GG26599@merlinux.eu> References: <20121216112300.GP26599@merlinux.eu> <20121219084201.GG26599@merlinux.eu> Message-ID: that looks good to me. I'm not sure I understand the reasoning behind having `clsarg(...) # @pytest.fixture("class", autouse=False)` come after setupClass, though. On Wed, Dec 19, 2012 at 1:42 AM, holger krekel wrote: > On Tue, Dec 18, 2012 at 21:11 +0100, Floris Bruynooghe wrote: > > On 16 December 2012 12:23, holger krekel wrote: > > > Currently, if you define e.g. an autouse fixture function it is going > to > > > be called _after_ the xUnit setup functions. This is especially > > > surprising when you do a session-scoped autouse fixture. I am > wondering > > > if we could reverse the order, i.e. call fixture functions (including > > > autouse-fixtures of course) ahead of xUnit setup methods. > > > > > > any thoughts? > > > > Agreed, I think it would be a good idea to have at least autouse > > fixtures before the xUnit setup. > > However, i realize we also have scopes. And this is where any attempt > to decide ordering between pytest versus xUnit fixtures seesm to break > down: > > - We don't want setup_class execute after a function-scoped pytest fixture. > > - We don't want class-scoped pytest fixture execute after setup_method. > > Maybe, we could internally add autouse-fixtures at module/class/function > scope which would look for setupX/teardownX and act accordingly. This > way xUnit setup/teardown methods would appear as pytest fixtures. > Here is an example how it could look like for a mix of > xUnit/pytest class/function scoped autouse- and non-autouse fixtures: > > user-function found by > > -------------------------------------------------------------------------- > ... > autoclass(...) # @pytest.fixture(scope="class", autouse=True)) > setup_class(cls) # internal @pytest.fixture("class", autouse=True) > clsarg(...) # @pytest.fixture("class", autouse=False) > funcfixture(...) # @pytest.fixture(scope="function", autouse=True) > setup_method() # internal @pytest.fixture("function", autouse=True) > arg1 # @pytest.fixture(scope="function", autouse=False)) > ... > test_function(arg1, clsarg) > # teardowns execute in LIFO registration order > > Makes sense? > > Ideally, we could produce something like the above output with > some command line option to help debugging. > > best, > holger > > > Regards, > > Floris > > > > > > -- > > Debian GNU/Linux -- The Power of Freedom > > www.debian.org | www.gnu.org | www.kernel.org > > > _______________________________________________ > py-dev mailing list > py-dev at codespeak.net > http://codespeak.net/mailman/listinfo/py-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From holger at merlinux.eu Thu Dec 20 10:13:39 2012 From: holger at merlinux.eu (holger krekel) Date: Thu, 20 Dec 2012 09:13:39 +0000 Subject: [py-dev] reversing fixture/xunit setup call order? In-Reply-To: References: <20121216112300.GP26599@merlinux.eu> <20121219084201.GG26599@merlinux.eu> Message-ID: <20121220091339.GM26599@merlinux.eu> On Wed, Dec 19, 2012 at 16:27 -0700, lahwran wrote: > that looks good to me. I'm not sure I understand the reasoning behind > having `clsarg(...) # @pytest.fixture("class", autouse=False)` come after > setupClass, though. The ideas is that implicit fixtures come before explicit ones. All autouse=True fixtures are implicit - they are not requested explicitely through a funcarg or a @pytest.mark.usefixtures(...) decoration. best, holger > On Wed, Dec 19, 2012 at 1:42 AM, holger krekel wrote: > > > On Tue, Dec 18, 2012 at 21:11 +0100, Floris Bruynooghe wrote: > > > On 16 December 2012 12:23, holger krekel wrote: > > > > Currently, if you define e.g. an autouse fixture function it is going > > to > > > > be called _after_ the xUnit setup functions. This is especially > > > > surprising when you do a session-scoped autouse fixture. I am > > wondering > > > > if we could reverse the order, i.e. call fixture functions (including > > > > autouse-fixtures of course) ahead of xUnit setup methods. > > > > > > > > any thoughts? > > > > > > Agreed, I think it would be a good idea to have at least autouse > > > fixtures before the xUnit setup. > > > > However, i realize we also have scopes. And this is where any attempt > > to decide ordering between pytest versus xUnit fixtures seesm to break > > down: > > > > - We don't want setup_class execute after a function-scoped pytest fixture. > > > > - We don't want class-scoped pytest fixture execute after setup_method. > > > > Maybe, we could internally add autouse-fixtures at module/class/function > > scope which would look for setupX/teardownX and act accordingly. This > > way xUnit setup/teardown methods would appear as pytest fixtures. > > Here is an example how it could look like for a mix of > > xUnit/pytest class/function scoped autouse- and non-autouse fixtures: > > > > user-function found by > > > > -------------------------------------------------------------------------- > > ... > > autoclass(...) # @pytest.fixture(scope="class", autouse=True)) > > setup_class(cls) # internal @pytest.fixture("class", autouse=True) > > clsarg(...) # @pytest.fixture("class", autouse=False) > > funcfixture(...) # @pytest.fixture(scope="function", autouse=True) > > setup_method() # internal @pytest.fixture("function", autouse=True) > > arg1 # @pytest.fixture(scope="function", autouse=False)) > > ... > > test_function(arg1, clsarg) > > # teardowns execute in LIFO registration order > > > > Makes sense? > > > > Ideally, we could produce something like the above output with > > some command line option to help debugging. > > > > best, > > holger > > > > > Regards, > > > Floris > > > > > > > > > -- > > > Debian GNU/Linux -- The Power of Freedom > > > www.debian.org | www.gnu.org | www.kernel.org > > > > > _______________________________________________ > > py-dev mailing list > > py-dev at codespeak.net > > http://codespeak.net/mailman/listinfo/py-dev > > > _______________________________________________ > py-dev mailing list > py-dev at codespeak.net > http://codespeak.net/mailman/listinfo/py-dev From holger at merlinux.eu Fri Dec 21 10:51:03 2012 From: holger at merlinux.eu (holger krekel) Date: Fri, 21 Dec 2012 09:51:03 +0000 Subject: [py-dev] yield-tests and fixtures: should they have a future? Message-ID: <20121221095103.GW26599@merlinux.eu> Hi testing folks, hi Jason, i am looking at some recent pytest issues and would like to simplify pytest's internal fixture handling. One obstacle/complication are yield-tests, i.e. the style of producing tests with a generator:: def test_gen(self): for x in range(10): yield check, x This currently produces 10 test items, calling the "check" function with the respective parameter. This by itself is not a big deal to support. However, some people expect fixtures/setup_function/method functions to execute before the generator does and this mixes the collection with the runtest phase. Unfortunately, nose also supports this notion although i am wondering how nose2 is going to deal with it as Jason also plans to separate collection from running. So i am thinking about dropping fixture/setup support for yield-tests in pytest but OTOH i'd like to keep backward and nose compatibility. As far as pytest is concerned, it has many others means of parametrization independently from yield, see e. g. http://pytest.org/latest/fixture.html#fixture-parametrize and http://pytest.org/latest/parametrize.html and more and more people are starting to use those (pytest does not document or recommend yield for 1-2 years now). If anyone has any input/thoughts on this, please shoot. best, holger From holger at merlinux.eu Fri Dec 21 11:00:00 2012 From: holger at merlinux.eu (holger krekel) Date: Fri, 21 Dec 2012 10:00:00 +0000 Subject: [py-dev] this list moving to pytest-dev@python.org Message-ID: <20121221100000.GY26599@merlinux.eu> Hi folks, this list is going to move to move python.org, probably even today. The new list address is: pytest-dev at python.org the commit list will be: pytest-commit at python.org The old addresses (py-dev at codespeak.net,py-svn at codespeak.net) will continue to function so the move shouldn't be disruptive. I am going to send another mail once the move is complete. best, holger From Ronny.Pfannschmidt at gmx.de Fri Dec 21 11:37:57 2012 From: Ronny.Pfannschmidt at gmx.de (Ronny Pfannschmidt) Date: Fri, 21 Dec 2012 11:37:57 +0100 Subject: [py-dev] [TIP] yield-tests and fixtures: should they have a future? In-Reply-To: <20121221095103.GW26599@merlinux.eu> References: <20121221095103.GW26599@merlinux.eu> Message-ID: <50D43C05.4060909@gmx.de> Hi all, My oppinion on yield Tests is, that they should be turned into part of reporting extensions instead of their current place at running/collection i would like to have more than one report for functional/acceptance tests anyway preferably in a way that allows parts to fail but still run the complete test with that in place a yield test would be just a loop running the check of all items without propagating single item failures to the outside also there is another upcoming task yield tests based on the the new async api best, Ronny On 12/21/2012 10:51 AM, holger krekel wrote: > Hi testing folks, hi Jason, > > i am looking at some recent pytest issues and would like to simplify > pytest's internal fixture handling. One obstacle/complication are > yield-tests, i.e. the style of producing tests with a generator:: > > def test_gen(self): > for x in range(10): > yield check, x > > This currently produces 10 test items, calling the "check" function with the > respective parameter. This by itself is not a big deal to support. > However, some people expect fixtures/setup_function/method functions to execute > before the generator does and this mixes the collection with the runtest phase. > Unfortunately, nose also supports this notion although i am wondering how > nose2 is going to deal with it as Jason also plans to separate collection > from running. > > So i am thinking about dropping fixture/setup support for yield-tests in pytest > but OTOH i'd like to keep backward and nose compatibility. As far as pytest > is concerned, it has many others means of parametrization independently > from yield, see e. g. http://pytest.org/latest/fixture.html#fixture-parametrize > and http://pytest.org/latest/parametrize.html and more and more people > are starting to use those (pytest does not document or recommend yield > for 1-2 years now). > > If anyone has any input/thoughts on this, please shoot. > > best, > holger > > _______________________________________________ > testing-in-python mailing list > testing-in-python at lists.idyll.org > http://lists.idyll.org/listinfo/testing-in-python From holger at merlinux.eu Fri Dec 28 22:45:57 2012 From: holger at merlinux.eu (holger krekel) Date: Fri, 28 Dec 2012 21:45:57 +0000 Subject: [pytest-dev] [py-dev] this list moving to pytest-dev@python.org In-Reply-To: <20121221100000.GY26599@merlinux.eu> References: <20121221100000.GY26599@merlinux.eu> Message-ID: <20121228214557.GV26599@merlinux.eu> Hi all, this list has moved to python.org now. time to update your filters and aliases. py-dev at codespeak.net now lives at pytest-dev at python.org py-svn at codespeak.net now lives at pytest-commit at python.org Thanks to Ralf Hildebrandt for performing the move on the python.org side! cheers, holger On Fri, Dec 21, 2012 at 10:00 +0000, holger krekel wrote: > Hi folks, > > this list is going to move to move python.org, probably even today. > The new list address is: > > pytest-dev at python.org > > the commit list will be: > > pytest-commit at python.org > > The old addresses (py-dev at codespeak.net,py-svn at codespeak.net) will > continue to function so the move shouldn't be disruptive. > > I am going to send another mail once the move is complete. > > best, > holger > > _______________________________________________ > py-dev mailing list > py-dev at codespeak.net > http://codespeak.net/mailman/listinfo/py-dev > From dasdasich at gmail.com Sat Dec 29 18:11:41 2012 From: dasdasich at gmail.com (=?UTF-8?Q?Daniel_Neuh=C3=A4user?=) Date: Sat, 29 Dec 2012 18:11:41 +0100 Subject: [pytest-dev] Test parameterization via Fixtures Message-ID: Hello, I have recently re-discovered py.test searching for testing solution for a new project and have started using it in said project. However I have also noticed that fixtures are rather heavily constrained, most importantly in that a fixture cannot have multiple values. To give an example one of the things that are part of my new project is password hashing. What I would like to be able to do with fixtures is have a fixture `password` that creates several random passwords and I would like a test that uses the fixture to be called with each of these passwords. Furthermore I would like to be able to create other fixtures such as a `pw_hash` fixture that depends on the `password` fixture and creates hashes for each password. I would also like to be able to write tests that take multiple fixtures and while being parametrized "naturally". An example would be a function `test_pw_hash_verify(pw_hash, password)` that takes both two fixtures `pw_hash` and `password`. In this case I would like `test_pw_hash_verify` to be called with password and the corresponding `pw_hash`. In more abstract terms I would like to have a fixture system for which the following rules hold: - A fixture `fixture()` represents set of values. - A fixture `fixture(a)` is a n:n (n:m?) mapping of the values represented by `a` to a new set of values. - A fixture `fixture(*fixtures)` is a n:m mapping of the cartesian product of the values represented by each in `fixtures` if they are all independent from one another. If any fixture `a` in `fixtures` is dependent upon one or more other fixtures `b_` in `fixtures` `fixture(*fixtures)` shall only be called with the values of `a` and `b_` so that the values `a` was called with are equal to the ones `fixtures(*fixtures)` is called with. I have created a very simple implementation of this algorithm to play around with as an example, which I hope will better illustrate the idea: https://gist.github.com/4408054 What I would like to know is whether there is any convenient way to do that now without abusing py.test significantly, whether anyone is working on this or something equivalent already or if there is some huge flaw in this idea, that I am missing at the moment. -------------- next part -------------- An HTML attachment was scrubbed... URL: From holger at merlinux.eu Sat Dec 29 19:22:35 2012 From: holger at merlinux.eu (holger krekel) Date: Sat, 29 Dec 2012 18:22:35 +0000 Subject: [pytest-dev] Test parameterization via Fixtures In-Reply-To: References: Message-ID: <20121229182235.GW26599@merlinux.eu> Hi Daniel, On Sat, Dec 29, 2012 at 18:11 +0100, Daniel Neuh?user wrote: > Hello, > > I have recently re-discovered py.test searching for testing solution for a > new project and have started using it in said project. However I have also > noticed that fixtures are rather heavily constrained, most importantly in > that a fixture cannot have multiple values. > > To give an example one of the things that are part of my new project is > password hashing. What I would like to be able to do with fixtures is have > a fixture `password` that creates several random passwords and I would like > a test that uses the fixture to be called with each of these passwords. Did you check out http://pytest.org/latest/fixture.html#parametrizing-a-fixture yet ? > Furthermore I would like to be able to create other fixtures such as a > `pw_hash` fixture that depends on the `password` fixture and creates hashes > for each password. That is directly doable like so: @pytest.fixture(params=["pass1", "pass2"] def password(request): return request.param @pytest.fixture def pw_hash(password): return hash(password) This would make all tests that use "pw_hash" run twice. > I would also like to be able to write tests that take multiple fixtures and > while being parametrized "naturally". An example would be a function > `test_pw_hash_verify(pw_hash, password)` that takes both two fixtures > `pw_hash` and `password`. In this case I would like `test_pw_hash_verify` > to be called with password and the corresponding `pw_hash`. This so far sounds like the current scheme. > > In more abstract terms I would like to have a fixture system for which the > following rules hold: > > - A fixture `fixture()` represents set of values. > - A fixture `fixture(a)` is a n:n (n:m?) mapping of the values represented > by `a` to a new set of values. > - A fixture `fixture(*fixtures)` is a n:m mapping of the cartesian product > of the values represented by each in `fixtures` if they are all independent > from one another. If any fixture `a` in `fixtures` is dependent upon one or > more other fixtures `b_` in `fixtures` `fixture(*fixtures)` shall only be > called with the values of `a` and `b_` so that the values `a` was called > with are equal to the ones `fixtures(*fixtures)` is called with. > > I have created a very simple implementation of this algorithm to play > around with as an example, which I hope will better illustrate the idea: > https://gist.github.com/4408054 This particular API is not supported, i.e. yielding from fixtures. pytest follows a design decision that collection of tests should be independent from running or setting up tests. At collection time pytest has a view on the complete fixture tree neccessary for running the tests and re-orders tests to be grouped by fixtures/scopes. Yielding fixtures would not allow either. > What I would like to know is whether there is any convenient way to do that > now without abusing py.test significantly, whether anyone is working on > this or something equivalent already or if there is some huge flaw in this > idea, that I am missing at the moment. Let us know if the above methods work for you or what is missing! cheers, holger > _______________________________________________ > Pytest-dev mailing list > Pytest-dev at python.org > http://mail.python.org/mailman/listinfo/pytest-dev