From memedough at gmail.com Sun May 1 11:15:16 2011 From: memedough at gmail.com (meme dough) Date: Sun, 1 May 2011 19:15:16 +1000 Subject: [py-dev] conftest global scope Message-ID: Hi, pytest-cov has an issue ticket that conftest global scope has no coverage reported. I don't think it will be possible to cover though. My understanding is that conftest file is local plugin which pytest will load quite early. It then can have hooks in conftest like to add args or change args. Coverage starts at session start hook which I think most appropriate. The hooks in conftest (like configure) could change things to turn coverage on / off or set coverage paths. So starting coverage any earlier than session start may not be good. Is this correct with conftest? It is not reloaded is it? It gets loaded early and hooks in it called to modify pytest operations. So no way we could get coverage at conftest global scope anyway? :) From baptiste.lepilleur at gmail.com Sun May 1 11:59:15 2011 From: baptiste.lepilleur at gmail.com (Baptiste Lepilleur) Date: Sun, 1 May 2011 11:59:15 +0200 Subject: [py-dev] Help understanding pytest_generate_tests related error message In-Reply-To: <1304108704.30044.81.camel@Klappe2> References: <1304071764.30044.18.camel@Klappe2> <1304108704.30044.81.camel@Klappe2> Message-ID: 2011/4/29 Ronny Pfannschmidt > On Fri, 2011-04-29 at 22:18 +0200, Baptiste Lepilleur wrote: > > [...] > > I'm not sure how I would go about that: it would means going through > > the VALIDATION_SCENARIOS test data for each test function, and > > duplicating some setup code of the test function... > > unless i'm misstaken, only the creation of the class is shared > which means that could actually be something like a funcarg or a param > to a funcarg > Indeed. I did the split but I feel that the scenario types/test function association is very clumsy: http://pastebin.com/gTsVJLWK Is there a simpler way to do this? Ideally, I'd rather have a simple way to associate a test function with a test parameter generator. I'm thinking of something like this for example: @parametrized( _generate_property_validation_tests, ['good_values'] ) def test_property_validation_good_values( scenario_type, dbo_class, value, good_value ): ... The parametrized decorator would call _generate_property_validation_tests( metafunc, ['good_values'] ). Is there a way to do that? Thanks, Baptiste. > > -- Ronny > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From baptiste.lepilleur at gmail.com Sun May 1 12:08:16 2011 From: baptiste.lepilleur at gmail.com (Baptiste Lepilleur) Date: Sun, 1 May 2011 12:08:16 +0200 Subject: [py-dev] How to run a given test by name? Message-ID: Hi, typically when troubleshooting multiple test failures, I want to be able to run a single parametrized test case. How can you tell py.test to run only tests matching a specific name. For example, I'd like to be able to run: py.test src --filter-by-name "test_mandatory_property[ZText]" This would run all tests with a name matching "test_mandatory_property[ZText]". Is there a way to do that? Baptiste. -------------- next part -------------- An HTML attachment was scrubbed... URL: From holger at merlinux.eu Sun May 1 12:22:41 2011 From: holger at merlinux.eu (holger krekel) Date: Sun, 1 May 2011 10:22:41 +0000 Subject: [py-dev] How to run a given test by name? In-Reply-To: References: Message-ID: <20110501102241.GB4071@merlinux.eu> On Sun, May 01, 2011 at 12:08 +0200, Baptiste Lepilleur wrote: > typically when troubleshooting multiple test failures, I want to be able to > run a single parametrized test case. > > How can you tell py.test to run only tests matching a specific name. > > For example, I'd like to be able to run: > > py.test src --filter-by-name "test_mandatory_property[ZText]" > > This would run all tests with a name > matching "test_mandatory_property[ZText]". > > Is there a way to do that? two possibilites. You can do a run with "py.test -rf" which will report test IDs for all failures. You can then pass one or more of those IDs to a py.test invocation. Secondly you can use the keyword option try something like "-k ZTEXT", see option help. holger From Ronny.Pfannschmidt at gmx.de Sun May 1 12:29:09 2011 From: Ronny.Pfannschmidt at gmx.de (Ronny Pfannschmidt) Date: Sun, 01 May 2011 12:29:09 +0200 Subject: [py-dev] Help understanding pytest_generate_tests related error message In-Reply-To: References: <1304071764.30044.18.camel@Klappe2> <1304108704.30044.81.camel@Klappe2> Message-ID: <1304245750.30044.89.camel@Klappe2> On Sun, 2011-05-01 at 11:59 +0200, Baptiste Lepilleur wrote: > 2011/4/29 Ronny Pfannschmidt > > On Fri, 2011-04-29 at 22:18 +0200, Baptiste Lepilleur wrote: > > [...] > > I'm not sure how I would go about that: it would means going > through > > the VALIDATION_SCENARIOS test data for each test function, > and > > duplicating some setup code of the test function... > > > unless i'm misstaken, only the creation of the class is shared > which means that could actually be something like a funcarg or > a param > to a funcarg > > > Indeed. I did the split but I feel that the scenario types/test > function association is very clumsy: > http://pastebin.com/gTsVJLWK > > > Is there a simpler way to do this? Ideally, I'd rather have a simple > way to associate a test function with a test parameter generator. I'm > thinking of something like this for example: > > > @parametrized( _generate_property_validation_tests, ['good_values'] ) > def test_property_validation_good_values( scenario_type, dbo_class, > value, good_value ): > ... > > > The parametrized decorator would > call _generate_property_validation_tests( metafunc, ['good_values'] ). > > > Is there a way to do that? you can use the pytest.mark tool to make a decorator which annotates the value types nicely -- Ronny > > > Thanks, > Baptiste. > > > -- Ronny > > -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: This is a digitally signed message part URL: From baptiste.lepilleur at gmail.com Sun May 1 12:41:06 2011 From: baptiste.lepilleur at gmail.com (Baptiste Lepilleur) Date: Sun, 1 May 2011 12:41:06 +0200 Subject: [py-dev] How to run a given test by name? In-Reply-To: <20110501102241.GB4071@merlinux.eu> References: <20110501102241.GB4071@merlinux.eu> Message-ID: options -k was what I was looking for. I somehow managed to miss it. Thanks, Baptiste. 2011/5/1 holger krekel > On Sun, May 01, 2011 at 12:08 +0200, Baptiste Lepilleur wrote: > > typically when troubleshooting multiple test failures, I want to be able > to > > run a single parametrized test case. > > > > How can you tell py.test to run only tests matching a specific name. > > > > For example, I'd like to be able to run: > > > > py.test src --filter-by-name "test_mandatory_property[ZText]" > > > > This would run all tests with a name > > matching "test_mandatory_property[ZText]". > > > > Is there a way to do that? > > two possibilites. You can do a run with "py.test -rf" which will report > test IDs > for all failures. You can then pass one or more of those IDs to a py.test > invocation. > Secondly you can use the keyword option try something like "-k ZTEXT", see > option help. > > holger > -------------- next part -------------- An HTML attachment was scrubbed... URL: From baptiste.lepilleur at gmail.com Sun May 1 13:47:19 2011 From: baptiste.lepilleur at gmail.com (Baptiste Lepilleur) Date: Sun, 1 May 2011 13:47:19 +0200 Subject: [py-dev] Help understanding pytest_generate_tests related error message In-Reply-To: <1304245750.30044.89.camel@Klappe2> References: <1304071764.30044.18.camel@Klappe2> <1304108704.30044.81.camel@Klappe2> <1304245750.30044.89.camel@Klappe2> Message-ID: 2011/5/1 Ronny Pfannschmidt > On Sun, 2011-05-01 at 11:59 +0200, Baptiste Lepilleur wrote: > ... > > @parametrized( _generate_property_validation_tests, ['good_values'] ) > > def test_property_validation_good_values( scenario_type, dbo_class, > > value, good_value ): > > ... > > > > > > The parametrized decorator would > > call _generate_property_validation_tests( metafunc, ['good_values'] ). > > > > > > Is there a way to do that? > you can use the pytest.mark tool to make a decorator which annotates > the value types nicely > Great, this is exactly what I wanted! The pytest_generate_tests function is now generic: def pytest_generate_tests(metafunc): if hasattr( metafunc.function, 'parametrized' ): args = metafunc.function.parametrized.args kwargs = metafunc.function.parametrized.kwargs test_generator, args = args[0], args[1:] test_generator( metafunc, *args, **kwargs ) And the parameter generator function can be specified as I wanted: @py.test.mark.parametrized( _generate_property_validation_tests, ['good_values'] ) def test_property_validation_good_values( dbo_class, value, good_value ): ... Here is the source: http://pastebin.com/kfgvM7L9 The only pitfall to be aware of is that you must always provide two parameters to @py.test.mark.parametrized otherwise MarkDecorator will think that the first parameter, which is callable, must be decorated. Thanks a lot for your help, Baptiste. > > -- Ronny > -------------- next part -------------- An HTML attachment was scrubbed... URL: From schettino72 at gmail.com Mon May 2 05:33:39 2011 From: schettino72 at gmail.com (Eduardo Schettino) Date: Mon, 2 May 2011 11:33:39 +0800 Subject: [py-dev] [ANN] plugin: pytest-incremental In-Reply-To: <20110429180555.GU4071@merlinux.eu> References: <20110429120038.GQ4071@merlinux.eu> <20110429180555.GU4071@merlinux.eu> Message-ID: On Sat, Apr 30, 2011 at 2:05 AM, holger krekel wrote: > On Fri, Apr 29, 2011 at 21:54 +0800, Eduardo Schettino wrote: >> On Fri, Apr 29, 2011 at 8:00 PM, holger krekel wrote: >> > * it's not compatible with pytest-xdist, is it? I gave it a try... Right now it is definitely not compatible. In the plugin test results are saved internally on pytest_runtest_makereport and processed at pytest_sessionfinish. I could see 2 issues: 1) pytest_sessionfinish is called once for every subprocess and once for the main process. So on sessionfinish I need to detect if it is a subprocess and send the result to the runner in the main process. In the main process it would work as it is today. Any clues how to that? I looked into junitxml plugin but it seems it does not work together with xdist also. 2) pytest_runtest_makereport when using xdist "call.result" is None (or not defined) when the test fails. I guess this is a bug on xdist. > > ok, now it's clear that one need to specify file system paths. > Throughout the python world "--XYZ-pkg" might mean a module > import path like "os.path" or a filesystem path. ?Maybe good > to mention this in the help string for the option - i'd probably > rather call it "--watch-path" to disambiguate. thanks. fixed. > > Right. ?Do you plan to implement a manual way to specify deps > for your plugin? I could do that, but I don't know how :) I will join the py.test IRC so we can talk about it. I've been thinking in going the other way. doit calling pytest instead of pytest calling doit in a plugin. I've done that before.. the problem is that I am not sure it can be done without losing all the features from py.test runner. > > sidenote: you may want to announce the next release of the plugin > also to the TIP (testing in python) list - > http://lists.idyll.org/listinfo/testing-in-python - > a number of people are rather following there rather than here. > I announced there. this thread was my motivation to write the plugin (http://lists.idyll.org/pipermail/testing-in-python/2011-April/004083.html) > best, > holger > cheers, eduardo From memedough at gmail.com Mon May 2 12:01:13 2011 From: memedough at gmail.com (meme dough) Date: Mon, 2 May 2011 20:01:13 +1000 Subject: [py-dev] [ANN] plugin: pytest-incremental In-Reply-To: References: <20110429120038.GQ4071@merlinux.eu> <20110429180555.GU4071@merlinux.eu> Message-ID: Hi, pytest-cov has pytest-xdist support. It was a big reason for writing it (pytest-cov that is) and holger kindly added to the protocol between master / slave to allow info to travel that pytest-cov needed. It is generic so you can use it too. Have a look at: https://bitbucket.org/memedough/pytest-cov/src/a7d65cc2458a/pytest_cov.py At pytest_sessionstart you can check if you are a master process or a slave process. The most important thing to look for is if config has slaveinput. If the config has that slaveinput then the process is a slave. My one work with older pytest and 2.x too. So that give you answer to non distributed testing process, master process or slave process. On master pytest_configure_node you get node. On that put info in the node.slaveinput and pytest send it to the slave. Look at DistMaster: https://bitbucket.org/memedough/cov-core/src/b9a3c9756366/cov_core.py On slave at pytest_sessionstart you can get the slaveinput. Look at DistSlave. On slave at pytest_sessionfinish you put info in that config.slaveoutput dict to go back to the master. Then on the master at pytest_testnodedown you get the slaveoutput from the node that went down. So then at pytest_sessionfinish on master you know all slaves have finished and sent back info that been collected by master so can do stuff with it. Feel free to use anything from pytest-cov if useful for you. :) On 2 May 2011 13:33, Eduardo Schettino wrote: > On Sat, Apr 30, 2011 at 2:05 AM, holger krekel wrote: >> On Fri, Apr 29, 2011 at 21:54 +0800, Eduardo Schettino wrote: >>> On Fri, Apr 29, 2011 at 8:00 PM, holger krekel wrote: >>> > * it's not compatible with pytest-xdist, is it? > I gave it a try... Right now it is definitely not compatible. > > In the plugin test results are saved internally on > pytest_runtest_makereport and processed at pytest_sessionfinish. I > could see 2 issues: > > 1) pytest_sessionfinish is called once for every subprocess and once > for the main process. So on sessionfinish I need to detect if it is a > subprocess and send the result to the runner in the main process. In > the main process it would work as it is today. Any clues how to that? > I looked into junitxml plugin but it seems it does not work together > with xdist also. > > 2) pytest_runtest_makereport when using xdist "call.result" is None > (or not defined) when the test fails. I guess this is a bug on xdist. > > >> >> ok, now it's clear that one need to specify file system paths. >> Throughout the python world "--XYZ-pkg" might mean a module >> import path like "os.path" or a filesystem path. ?Maybe good >> to mention this in the help string for the option - i'd probably >> rather call it "--watch-path" to disambiguate. > thanks. fixed. > >> >> Right. ?Do you plan to implement a manual way to specify deps >> for your plugin? > I could do that, but I don't know how :) ? I will join the py.test IRC > so we can talk about it. > > I've been thinking in going the other way. doit calling pytest instead > of pytest calling doit in a plugin. I've done that before.. the > problem is that I am not sure it can be done without losing all the > features from py.test runner. > >> >> sidenote: you may want to announce the next release of the plugin >> also to the TIP (testing in python) list - >> http://lists.idyll.org/listinfo/testing-in-python - >> a number of people are rather following there rather than here. >> > I announced there. this thread was my motivation to write the plugin > (http://lists.idyll.org/pipermail/testing-in-python/2011-April/004083.html) > >> best, >> holger >> > cheers, > ?eduardo > _______________________________________________ > py-dev mailing list > py-dev at codespeak.net > http://codespeak.net/mailman/listinfo/py-dev > From rafalskiy at gmail.com Mon May 2 23:40:33 2011 From: rafalskiy at gmail.com (Vyacheslav Rafalskiy) Date: Mon, 2 May 2011 17:40:33 -0400 Subject: [py-dev] migration from 1.3 to 2.0 In-Reply-To: <20110430142225.GY4071@merlinux.eu> References: <20110405201612.GW16231@merlinux.eu> <20110407061715.GE16231@merlinux.eu> <20110430142225.GY4071@merlinux.eu> Message-ID: On Sat, Apr 30, 2011 at 10:22 AM, holger krekel wrote: > On Thu, Apr 07, 2011 at 12:29 -0400, Vyacheslav Rafalskiy wrote: >> >> >(sidenote: configure and even sessionstart hooks are both a bit >> > not 100% right because they happen even on the master side of a distributed >> > test run and the master side does not collect or run tests at all) >> >> I see. Perhaps something like setup_package() in the top-level __init__.py >> could be a solution? > > I guess you mean an __init__.py file of a test directory. > With a layout of test dirs within an application this might mean > one has to put this setup into the main package __init__.py > and mixing test code and application code is often not a good idea. Yes, exactly. In my case of functional testing I don't even have application code here. When I start the tests I tell the runner where in the network the system under test is. > > So i'd rather put it into a conftest.py file as a normal hook. > Maybe "pytest_pyfunc_setup(request)" would be good where request > is the same object as for the funcarg factories. > > You could then write: > > ? ?# content of conftest.py > ? ?def pytest_pyfunc_setup(request): > ? ? ? ?val = request.cached_setup(setup=makeval, scope="session") > ? ? ? ?# use val for some global setting of the package > > Alternatively we could see to call something like: > > ? ?def pytest_setup_testloop(config): > ? ? ? ?val = makeval() > ? ? ? ?# use val for some global setting of the package > > ? ?def pytest_teardown_testloop(config): > ? ? ? ?... > > which would be called once for a test process. The reason why I suggested setup_package() is that you already have setup_function, setup_method, setup_class and setup_module so the former would just complete the line-up. And the natural place for it would be __init__.py of that package. On the other hand, you can put conftest.py in every folder, which can do precisely the same thing as well as many others. The question is which way is more intuitive and results in cleaner code. The answer is perhaps "It depends". I like setup_module(module) because it lets me dump the configuration straight into the namespace where I use it and setup_package(package) could do the same. But all in all, whatever works is fine with me. Thanks, Vyacheslav > > best, > holger > >> Vyacheslav >> _______________________________________________ >> py-dev mailing list >> py-dev at codespeak.net >> http://codespeak.net/mailman/listinfo/py-dev >> > From holger at merlinux.eu Tue May 3 11:48:14 2011 From: holger at merlinux.eu (holger krekel) Date: Tue, 3 May 2011 09:48:14 +0000 Subject: [py-dev] migration from 1.3 to 2.0 In-Reply-To: References: <20110405201612.GW16231@merlinux.eu> <20110407061715.GE16231@merlinux.eu> <20110430142225.GY4071@merlinux.eu> Message-ID: <20110503094814.GU4071@merlinux.eu> On Mon, May 02, 2011 at 17:40 -0400, Vyacheslav Rafalskiy wrote: > On Sat, Apr 30, 2011 at 10:22 AM, holger krekel wrote: > > On Thu, Apr 07, 2011 at 12:29 -0400, Vyacheslav Rafalskiy wrote: > > >> > >> >(sidenote: configure and even sessionstart hooks are both a bit > >> > not 100% right because they happen even on the master side of a distributed > >> > test run and the master side does not collect or run tests at all) > >> > >> I see. Perhaps something like setup_package() in the top-level __init__.py > >> could be a solution? > > > > I guess you mean an __init__.py file of a test directory. > > With a layout of test dirs within an application this might mean > > one has to put this setup into the main package __init__.py > > and mixing test code and application code is often not a good idea. > > Yes, exactly. In my case of functional testing I don't even have > application code here. > When I start the tests I tell the runner where in the network the > system under test is. > > > > > So i'd rather put it into a conftest.py file as a normal hook. > > Maybe "pytest_pyfunc_setup(request)" would be good where request > > is the same object as for the funcarg factories. > > > > You could then write: > > > > ? ?# content of conftest.py > > ? ?def pytest_pyfunc_setup(request): > > ? ? ? ?val = request.cached_setup(setup=makeval, scope="session") > > ? ? ? ?# use val for some global setting of the package > > > > Alternatively we could see to call something like: > > > > ? ?def pytest_setup_testloop(config): > > ? ? ? ?val = makeval() > > ? ? ? ?# use val for some global setting of the package > > > > ? ?def pytest_teardown_testloop(config): > > ? ? ? ?... > > > > which would be called once for a test process. > > The reason why I suggested setup_package() is that you already have > setup_function, setup_method, setup_class and setup_module so > the former would just complete the line-up. And the natural place > for it would be __init__.py of that package. > > On the other hand, you can put conftest.py in every folder, which > can do precisely the same thing as well as many others. The > question is which way is more intuitive and results in cleaner code. > The answer is perhaps "It depends". > > I like setup_module(module) because it lets me dump the configuration > straight into the namespace where I use it and setup_package(package) > could do the same. good line of reasoning. It's mostly my intuition making me hesitant to add setup_package like you suggest. And i wonder what it is about :) Maybe it's that the root namespace of a test directory is often called "testing" or "tests" (the "test" one is taken by Python stdlib already). And therefore you would end up having to do "import testing" and then use global state with something like "testing.STATE". But i guess this doesn't look so bad to you, does it? :) (The plugin/conftest system is designed such that it hardly requires any imports to manage test state.) Any more opinions on setup_package(package)? If others find it useful as well, i will consider introducing it with pytest-2.1. best, holger From rafalskiy at gmail.com Wed May 4 16:53:12 2011 From: rafalskiy at gmail.com (Vyacheslav Rafalskiy) Date: Wed, 4 May 2011 10:53:12 -0400 Subject: [py-dev] migration from 1.3 to 2.0 In-Reply-To: <20110503094814.GU4071@merlinux.eu> References: <20110405201612.GW16231@merlinux.eu> <20110407061715.GE16231@merlinux.eu> <20110430142225.GY4071@merlinux.eu> <20110503094814.GU4071@merlinux.eu> Message-ID: On Tue, May 3, 2011 at 5:48 AM, holger krekel wrote: > On Mon, May 02, 2011 at 17:40 -0400, Vyacheslav Rafalskiy wrote: >> On Sat, Apr 30, 2011 at 10:22 AM, holger krekel wrote: >> > On Thu, Apr 07, 2011 at 12:29 -0400, Vyacheslav Rafalskiy wrote: >> >> >> >> >> >(sidenote: configure and even sessionstart hooks are both a bit >> >> > not 100% right because they happen even on the master side of a distributed >> >> > test run and the master side does not collect or run tests at all) >> >> >> >> I see. Perhaps something like setup_package() in the top-level __init__.py >> >> could be a solution? >> > >> > I guess you mean an __init__.py file of a test directory. >> > With a layout of test dirs within an application this might mean >> > one has to put this setup into the main package __init__.py >> > and mixing test code and application code is often not a good idea. >> >> Yes, exactly. In my case of functional testing I don't even have >> application code here. >> When I start the tests I tell the runner where in the network the >> system under test is. >> >> > >> > So i'd rather put it into a conftest.py file as a normal hook. >> > Maybe "pytest_pyfunc_setup(request)" would be good where request >> > is the same object as for the funcarg factories. >> > >> > You could then write: >> > >> > ? ?# content of conftest.py >> > ? ?def pytest_pyfunc_setup(request): >> > ? ? ? ?val = request.cached_setup(setup=makeval, scope="session") >> > ? ? ? ?# use val for some global setting of the package >> > >> > Alternatively we could see to call something like: >> > >> > ? ?def pytest_setup_testloop(config): >> > ? ? ? ?val = makeval() >> > ? ? ? ?# use val for some global setting of the package >> > >> > ? ?def pytest_teardown_testloop(config): >> > ? ? ? ?... >> > >> > which would be called once for a test process. >> >> The reason why I suggested setup_package() is that you already have >> setup_function, setup_method, setup_class and setup_module so >> the former would just complete the line-up. And the natural place >> for it would be __init__.py of that package. >> >> On the other hand, you can put conftest.py in every folder, which >> can do precisely the same thing as well as many others. The >> question is which way is more intuitive and results in cleaner code. >> The answer is perhaps "It depends". >> >> I like setup_module(module) because it lets me dump the configuration >> straight into the namespace where I use it and setup_package(package) >> could do the same. > > good line of reasoning. ?It's mostly my intuition making me hesitant > to add setup_package like you suggest. ?And i wonder what it is about :) > Maybe it's that the root namespace of a test directory is often called > "testing" or "tests" ?(the "test" one is taken by Python stdlib already). > And therefore you would end up having to do "import testing" and > then use global state with something like "testing.STATE". > But i guess this doesn't look so bad to you, does it? :) > (The plugin/conftest system is designed such that it hardly > requires any imports to manage test state.) > > Any more opinions on setup_package(package)? If others find it useful > as well, i will consider introducing it with pytest-2.1. I guess I will have to withdraw the idea. Having to explicitly import the test package does not look nice at all. conftest.py rules! As to the two alternatives above I'd rather use pytest_setup_testloop(config) with direct access to config. Regards, Vyacheslav > > best, > holger > From schettino72 at gmail.com Fri May 6 19:29:28 2011 From: schettino72 at gmail.com (Eduardo Schettino) Date: Sat, 7 May 2011 01:29:28 +0800 Subject: [py-dev] [ANN] plugin: pytest-incremental In-Reply-To: References: <20110429120038.GQ4071@merlinux.eu> <20110429180555.GU4071@merlinux.eu> Message-ID: On Mon, May 2, 2011 at 6:01 PM, meme dough wrote: > Hi, > > pytest-cov has pytest-xdist support. It was a big reason for writing > it (pytest-cov that is) and holger kindly added to the protocol > between master / slave to allow info to travel that pytest-cov needed. > It is generic so you can use it too. > > Have a look at: > > https://bitbucket.org/memedough/pytest-cov/src/a7d65cc2458a/pytest_cov.py > > At pytest_sessionstart you can check if you are a master process or a > slave process. The most important thing to look for is if config has > slaveinput. If the config has that slaveinput then the process is a > slave. My one work with older pytest and 2.x too. So that give you > answer to non distributed testing process, master process or slave > process. > > On master pytest_configure_node you get node. On that put info in the > node.slaveinput and pytest send it to the slave. Look at DistMaster: > > https://bitbucket.org/memedough/cov-core/src/b9a3c9756366/cov_core.py > > On slave at pytest_sessionstart you can get the slaveinput. Look at > DistSlave. > > On slave at pytest_sessionfinish you put info in that > config.slaveoutput dict to go back to the master. > > Then on the master at pytest_testnodedown you get the slaveoutput from > the node that went down. > > So then at pytest_sessionfinish on master you know all slaves have > finished and sent back info that been collected by master so can do > stuff with it. > > Feel free to use anything from pytest-cov if useful for you. > > :) > > Thanks Dough, it was very useful... now pytest-incremental works for me with xdist -n. i could also run --load=each but not sure it has the expected behaviour... the code is here https://bitbucket.org/schettino72/pytest-incremental cheers, eduardo -------------- next part -------------- An HTML attachment was scrubbed... URL: From baptiste.lepilleur at gmail.com Wed May 18 17:42:30 2011 From: baptiste.lepilleur at gmail.com (Baptiste Lepilleur) Date: Wed, 18 May 2011 17:42:30 +0200 Subject: [py-dev] What is the recommended way to run test with a coverage report? Message-ID: The pytest documentation page indicates that it is supported, but provides no pointer on how do to this... Doing a search seem to reveal multiple plug-ins to do that. What is the recommended one? I'm working on Windows XP / Python 2.6 & 3.2. By the way, is there a centralized list of useful plug-ins for pytest ? Baptiste. -------------- next part -------------- An HTML attachment was scrubbed... URL: From holger at merlinux.eu Wed May 18 22:04:43 2011 From: holger at merlinux.eu (holger krekel) Date: Wed, 18 May 2011 20:04:43 +0000 Subject: [py-dev] What is the recommended way to run test with a coverage report? In-Reply-To: References: Message-ID: <20110518200443.GA20287@merlinux.eu> Hey Baptiste, On Wed, May 18, 2011 at 17:42 +0200, Baptiste Lepilleur wrote: > The pytest documentation page indicates that it is supported, but provides > no pointer on how do to this... > Doing a search seem to reveal multiple plug-ins to do that. What is the > recommended one? I'm working on Windows XP / Python 2.6 & 3.2. sorry about that. The plugin is named pytest-cov, see here http://pypi.python.org/pypi/pytest-cov If it doesn't work for you i am sure Meme (also here on the list) can answer questions or to issues. > By the way, is there a centralized list of useful plug-ins for pytest ? not really. However, i recommend to install "pip" and then type: pip search pytest to get a good list. best, holger > > Baptiste. > _______________________________________________ > py-dev mailing list > py-dev at codespeak.net > http://codespeak.net/mailman/listinfo/py-dev From baptiste.lepilleur at gmail.com Thu May 19 23:53:39 2011 From: baptiste.lepilleur at gmail.com (Baptiste Lepilleur) Date: Thu, 19 May 2011 23:53:39 +0200 Subject: [py-dev] What is the recommended way to run test with a coverage report? In-Reply-To: <20110518200443.GA20287@merlinux.eu> References: <20110518200443.GA20287@merlinux.eu> Message-ID: Thanks for the pointer concerning pip, I did not know you could search using it. Now, I running into a strange missing import error when trying to the test with coverage: E ImportError: No module named restdoc Any ideas what could be the cause and what this module is? Detail below: py.test --cov src\nitroz\test_collections.py ================================================= test session starts ================================================= platform win32 -- Python 3.2.0 -- pytest-2.0.3 collected 0 items / 1 errors Coverage.py warning: No data was collected. ======================================================= ERRORS ======================================================== _________________________________________________ ERROR collecting . __________________________________________________ C:\Python32\lib\site-packages\py-1.4.3-py3.2.egg\py\_path\common.py:312: in visit > for x in Visitor(fil, rec, ignore, bf, sort).gen(self): C:\Python32\lib\site-packages\py-1.4.3-py3.2.egg\py\_path\common.py:358: in gen > for p in self.gen(subdir): C:\Python32\lib\site-packages\py-1.4.3-py3.2.egg\py\_path\common.py:358: in gen > for p in self.gen(subdir): C:\Python32\lib\site-packages\py-1.4.3-py3.2.egg\py\_path\common.py:358: in gen > for p in self.gen(subdir): C:\Python32\lib\site-packages\py-1.4.3-py3.2.egg\py\_path\common.py:347: in gen > dirs = self.optsort([p for p in entries C:\Python32\lib\site-packages\py-1.4.3-py3.2.egg\py\_path\common.py:348: in > if p.check(dir=1) and (rec is None or rec(p))]) C:\Python32\lib\site-packages\pytest-2.0.3-py3.2.egg\_pytest\main.py:450: in _recurse > ihook.pytest_collect_directory(path=path, parent=self) C:\Python32\lib\site-packages\pytest-2.0.3-py3.2.egg\_pytest\main.py:118: in call_matching_hooks > plugins = self.config._getmatchingplugins(self.fspath) C:\Python32\lib\site-packages\pytest-2.0.3-py3.2.egg\_pytest\config.py:283: in _getmatchingplugins > plugins += self._conftest.getconftestmodules(fspath) C:\Python32\lib\site-packages\pytest-2.0.3-py3.2.egg\_pytest\config.py:185: in getconftestmodules > clist.append(self.importconftest(conftestpath)) C:\Python32\lib\site-packages\pytest-2.0.3-py3.2.egg\_pytest\config.py:221: in importconftest > self._postimport(mod) C:\Python32\lib\site-packages\pytest-2.0.3-py3.2.egg\_pytest\config.py:226: in _postimport > self._onimport(mod) C:\Python32\lib\site-packages\pytest-2.0.3-py3.2.egg\_pytest\config.py:272: in _onimportconftest > self.pluginmanager.consider_conftest(conftestmodule) C:\Python32\lib\site-packages\pytest-2.0.3-py3.2.egg\_pytest\core.py:181: in consider_conftest > self.consider_module(conftestmodule) C:\Python32\lib\site-packages\pytest-2.0.3-py3.2.egg\_pytest\core.py:189: in consider_module > self.import_plugin(spec) C:\Python32\lib\site-packages\pytest-2.0.3-py3.2.egg\_pytest\core.py:202: in import_plugin > return self.import_plugin(modname[7:]) C:\Python32\lib\site-packages\pytest-2.0.3-py3.2.egg\_pytest\core.py:197: in import_plugin > mod = importplugin(modname) C:\Python32\lib\site-packages\pytest-2.0.3-py3.2.egg\_pytest\core.py:323: in importplugin > return __import__(importspec, None, None, '__doc__') E ImportError: No module named restdoc ----------------------------------- coverage: platform win32, python 3.2.0-final-0 ------------------------------------ Name Stmts Miss Cover --------------------------- ============================================== 1 error in 18.02 seconds =============================================== 2011/5/18 holger krekel > Hey Baptiste, > > On Wed, May 18, 2011 at 17:42 +0200, Baptiste Lepilleur wrote: > > The pytest documentation page indicates that it is supported, but > provides > > no pointer on how do to this... > > > Doing a search seem to reveal multiple plug-ins to do that. What is the > > recommended one? I'm working on Windows XP / Python 2.6 & 3.2. > > sorry about that. The plugin is named pytest-cov, see here > > http://pypi.python.org/pypi/pytest-cov > > If it doesn't work for you i am sure Meme (also here on the list) > can answer questions or to issues. > > > By the way, is there a centralized list of useful plug-ins for pytest ? > > not really. However, i recommend to install "pip" and then type: > > pip search pytest > > to get a good list. > > best, > holger > > > > > > > Baptiste. > > > _______________________________________________ > > py-dev mailing list > > py-dev at codespeak.net > > http://codespeak.net/mailman/listinfo/py-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From memedough at gmail.com Sun May 22 06:14:43 2011 From: memedough at gmail.com (meme dough) Date: Sun, 22 May 2011 14:14:43 +1000 Subject: [py-dev] What is the recommended way to run test with a coverage report? In-Reply-To: References: <20110518200443.GA20287@merlinux.eu> Message-ID: You need an arg to --cov which is the source root you want covered. py.test -h coverage reporting with distributed testing support: --cov=path measure coverage for filesystem path (multi-allowed) Try something like: py.test --cov . src\nitroz\test_collections.py py.test --cov src src\nitroz\test_collections.py On 20 May 2011 07:53, Baptiste Lepilleur wrote: > Thanks for the pointer concerning pip, I did not know you could search using > it. > Now, I running into a strange missing import error when trying to the test > with coverage: > E ? ImportError: No module named restdoc > Any ideas what could be the cause and what this module is? > Detail below: > py.test --cov src\nitroz\test_collections.py > ================================================= test session starts > ================================================= > platform win32 -- Python 3.2.0 -- pytest-2.0.3 > collected 0 items / 1 errors > Coverage.py warning: No data was collected. > ======================================================= ERRORS > ======================================================== > _________________________________________________ ERROR collecting . > __________________________________________________ > C:\Python32\lib\site-packages\py-1.4.3-py3.2.egg\py\_path\common.py:312: in > visit >> ? ? ? for x in Visitor(fil, rec, ignore, bf, sort).gen(self): > C:\Python32\lib\site-packages\py-1.4.3-py3.2.egg\py\_path\common.py:358: in > gen >> ? ? ? ? ? ? ? for p in self.gen(subdir): > C:\Python32\lib\site-packages\py-1.4.3-py3.2.egg\py\_path\common.py:358: in > gen >> ? ? ? ? ? ? ? for p in self.gen(subdir): > C:\Python32\lib\site-packages\py-1.4.3-py3.2.egg\py\_path\common.py:358: in > gen >> ? ? ? ? ? ? ? for p in self.gen(subdir): > C:\Python32\lib\site-packages\py-1.4.3-py3.2.egg\py\_path\common.py:347: in > gen >> ? ? ? dirs = self.optsort([p for p in entries > C:\Python32\lib\site-packages\py-1.4.3-py3.2.egg\py\_path\common.py:348: in > >> ? ? ? ? ? ? ? if p.check(dir=1) and (rec is None or rec(p))]) > C:\Python32\lib\site-packages\pytest-2.0.3-py3.2.egg\_pytest\main.py:450: in > _recurse >> ? ? ? ihook.pytest_collect_directory(path=path, parent=self) > C:\Python32\lib\site-packages\pytest-2.0.3-py3.2.egg\_pytest\main.py:118: in > call_matching_hooks >> ? ? ? plugins = self.config._getmatchingplugins(self.fspath) > C:\Python32\lib\site-packages\pytest-2.0.3-py3.2.egg\_pytest\config.py:283: > in _getmatchingplugins >> ? ? ? plugins += self._conftest.getconftestmodules(fspath) > C:\Python32\lib\site-packages\pytest-2.0.3-py3.2.egg\_pytest\config.py:185: > in getconftestmodules >> ? ? ? ? ? ? ? ? ? ? ? clist.append(self.importconftest(conftestpath)) > C:\Python32\lib\site-packages\pytest-2.0.3-py3.2.egg\_pytest\config.py:221: > in importconftest >> ? ? ? ? ? self._postimport(mod) > C:\Python32\lib\site-packages\pytest-2.0.3-py3.2.egg\_pytest\config.py:226: > in _postimport >> ? ? ? ? ? self._onimport(mod) > C:\Python32\lib\site-packages\pytest-2.0.3-py3.2.egg\_pytest\config.py:272: > in _onimportconftest >> ? ? ? self.pluginmanager.consider_conftest(conftestmodule) > C:\Python32\lib\site-packages\pytest-2.0.3-py3.2.egg\_pytest\core.py:181: in > consider_conftest >> ? ? ? ? ? self.consider_module(conftestmodule) > C:\Python32\lib\site-packages\pytest-2.0.3-py3.2.egg\_pytest\core.py:189: in > consider_module >> ? ? ? ? ? ? ? self.import_plugin(spec) > C:\Python32\lib\site-packages\pytest-2.0.3-py3.2.egg\_pytest\core.py:202: in > import_plugin >> ? ? ? ? ? ? ? return self.import_plugin(modname[7:]) > C:\Python32\lib\site-packages\pytest-2.0.3-py3.2.egg\_pytest\core.py:197: in > import_plugin >> ? ? ? ? ? mod = importplugin(modname) > C:\Python32\lib\site-packages\pytest-2.0.3-py3.2.egg\_pytest\core.py:323: in > importplugin >> ? ? ? return __import__(importspec, None, None, '__doc__') > E ? ImportError: No module named restdoc > ----------------------------------- coverage: platform win32, python > 3.2.0-final-0 ------------------------------------ > Name ? ?Stmts ? Miss ?Cover > --------------------------- > ============================================== 1 error in 18.02 seconds > =============================================== > > > 2011/5/18 holger krekel >> >> Hey Baptiste, >> >> On Wed, May 18, 2011 at 17:42 +0200, Baptiste Lepilleur wrote: >> > The pytest documentation page indicates that it is supported, but >> > provides >> > no pointer on how do to this... >> >> > Doing a search seem to reveal multiple plug-ins to do that. What is the >> > recommended one? I'm working on Windows XP / Python 2.6 & 3.2. >> >> sorry about that. ?The plugin is named pytest-cov, see here >> >> ? ?http://pypi.python.org/pypi/pytest-cov >> >> If it doesn't work for you i am sure Meme (also here on the list) >> can answer questions or to issues. >> >> > By the way, is there a centralized list of useful plug-ins for pytest ? >> >> not really. ?However, i recommend to install "pip" and then type: >> >> ? ?pip search pytest >> >> to get a good list. >> >> best, >> holger >> >> >> >> > >> > Baptiste. >> >> > _______________________________________________ >> > py-dev mailing list >> > py-dev at codespeak.net >> > http://codespeak.net/mailman/listinfo/py-dev >> > > > _______________________________________________ > py-dev mailing list > py-dev at codespeak.net > http://codespeak.net/mailman/listinfo/py-dev > > From baptiste.lepilleur at gmail.com Sun May 22 09:56:55 2011 From: baptiste.lepilleur at gmail.com (Baptiste Lepilleur) Date: Sun, 22 May 2011 09:56:55 +0200 Subject: [py-dev] What is the recommended way to run test with a coverage report? In-Reply-To: References: <20110518200443.GA20287@merlinux.eu> Message-ID: That was it! The restdoc mess was coming from an old version of the sources of py-1.3.4 my third-parties sub-directory that py.test was attempting to execute. The pytest-cov plug-in is exactly what I was looking for. By the way, the documentation indicates that it is possible to use standard coverage tool for result data analysis. Do those tool provided more info than the html report? If so could you provide some relevant pointers (I've never use python coverage tool before)? Thanks a lot, Baptiste. 2011/5/22 meme dough > You need an arg to --cov which is the source root you want covered. > > py.test -h > > coverage reporting with distributed testing support: > --cov=path measure coverage for filesystem path (multi-allowed) > > Try something like: > > py.test --cov . src\nitroz\test_collections.py > > py.test --cov src src\nitroz\test_collections.py > > On 20 May 2011 07:53, Baptiste Lepilleur > wrote: > > Thanks for the pointer concerning pip, I did not know you could search > using > > it. > > Now, I running into a strange missing import error when trying to the > test > > with coverage: > > E ImportError: No module named restdoc > > Any ideas what could be the cause and what this module is? > > Detail below: > > py.test --cov src\nitroz\test_collections.py > > ================================================= test session starts > > ================================================= > > platform win32 -- Python 3.2.0 -- pytest-2.0.3 > > collected 0 items / 1 errors > > Coverage.py warning: No data was collected. > > ======================================================= ERRORS > > ======================================================== > > _________________________________________________ ERROR collecting . > > __________________________________________________ > > C:\Python32\lib\site-packages\py-1.4.3-py3.2.egg\py\_path\common.py:312: > in > > visit > >> for x in Visitor(fil, rec, ignore, bf, sort).gen(self): > > C:\Python32\lib\site-packages\py-1.4.3-py3.2.egg\py\_path\common.py:358: > in > > gen > >> for p in self.gen(subdir): > > C:\Python32\lib\site-packages\py-1.4.3-py3.2.egg\py\_path\common.py:358: > in > > gen > >> for p in self.gen(subdir): > > C:\Python32\lib\site-packages\py-1.4.3-py3.2.egg\py\_path\common.py:358: > in > > gen > >> for p in self.gen(subdir): > > C:\Python32\lib\site-packages\py-1.4.3-py3.2.egg\py\_path\common.py:347: > in > > gen > >> dirs = self.optsort([p for p in entries > > C:\Python32\lib\site-packages\py-1.4.3-py3.2.egg\py\_path\common.py:348: > in > > > >> if p.check(dir=1) and (rec is None or rec(p))]) > > C:\Python32\lib\site-packages\pytest-2.0.3-py3.2.egg\_pytest\main.py:450: > in > > _recurse > >> ihook.pytest_collect_directory(path=path, parent=self) > > C:\Python32\lib\site-packages\pytest-2.0.3-py3.2.egg\_pytest\main.py:118: > in > > call_matching_hooks > >> plugins = self.config._getmatchingplugins(self.fspath) > > > C:\Python32\lib\site-packages\pytest-2.0.3-py3.2.egg\_pytest\config.py:283: > > in _getmatchingplugins > >> plugins += self._conftest.getconftestmodules(fspath) > > > C:\Python32\lib\site-packages\pytest-2.0.3-py3.2.egg\_pytest\config.py:185: > > in getconftestmodules > >> clist.append(self.importconftest(conftestpath)) > > > C:\Python32\lib\site-packages\pytest-2.0.3-py3.2.egg\_pytest\config.py:221: > > in importconftest > >> self._postimport(mod) > > > C:\Python32\lib\site-packages\pytest-2.0.3-py3.2.egg\_pytest\config.py:226: > > in _postimport > >> self._onimport(mod) > > > C:\Python32\lib\site-packages\pytest-2.0.3-py3.2.egg\_pytest\config.py:272: > > in _onimportconftest > >> self.pluginmanager.consider_conftest(conftestmodule) > > C:\Python32\lib\site-packages\pytest-2.0.3-py3.2.egg\_pytest\core.py:181: > in > > consider_conftest > >> self.consider_module(conftestmodule) > > C:\Python32\lib\site-packages\pytest-2.0.3-py3.2.egg\_pytest\core.py:189: > in > > consider_module > >> self.import_plugin(spec) > > C:\Python32\lib\site-packages\pytest-2.0.3-py3.2.egg\_pytest\core.py:202: > in > > import_plugin > >> return self.import_plugin(modname[7:]) > > C:\Python32\lib\site-packages\pytest-2.0.3-py3.2.egg\_pytest\core.py:197: > in > > import_plugin > >> mod = importplugin(modname) > > C:\Python32\lib\site-packages\pytest-2.0.3-py3.2.egg\_pytest\core.py:323: > in > > importplugin > >> return __import__(importspec, None, None, '__doc__') > > E ImportError: No module named restdoc > > ----------------------------------- coverage: platform win32, python > > 3.2.0-final-0 ------------------------------------ > > Name Stmts Miss Cover > > --------------------------- > > ============================================== 1 error in 18.02 seconds > > =============================================== > > > > > > 2011/5/18 holger krekel > >> > >> Hey Baptiste, > >> > >> On Wed, May 18, 2011 at 17:42 +0200, Baptiste Lepilleur wrote: > >> > The pytest documentation page indicates that it is supported, but > >> > provides > >> > no pointer on how do to this... > >> > >> > Doing a search seem to reveal multiple plug-ins to do that. What is > the > >> > recommended one? I'm working on Windows XP / Python 2.6 & 3.2. > >> > >> sorry about that. The plugin is named pytest-cov, see here > >> > >> http://pypi.python.org/pypi/pytest-cov > >> > >> If it doesn't work for you i am sure Meme (also here on the list) > >> can answer questions or to issues. > >> > >> > By the way, is there a centralized list of useful plug-ins for pytest > ? > >> > >> not really. However, i recommend to install "pip" and then type: > >> > >> pip search pytest > >> > >> to get a good list. > >> > >> best, > >> holger > >> > >> > >> > >> > > >> > Baptiste. > >> > >> > _______________________________________________ > >> > py-dev mailing list > >> > py-dev at codespeak.net > >> > http://codespeak.net/mailman/listinfo/py-dev > >> > > > > > > _______________________________________________ > > py-dev mailing list > > py-dev at codespeak.net > > http://codespeak.net/mailman/listinfo/py-dev > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From memedough at gmail.com Sun May 22 12:57:45 2011 From: memedough at gmail.com (meme dough) Date: Sun, 22 May 2011 20:57:45 +1000 Subject: [py-dev] What is the recommended way to run test with a coverage report? In-Reply-To: References: <20110518200443.GA20287@merlinux.eu> Message-ID: >From py.test you can get report on stdout with or without the missing line numbers displayed, a html report, an xml report for feeding into something like jenkins or annotated source code. These are the options for reporting that coverage has and pytest-cov allows any combinations of those that you want. Further to that, the coverage data file is called .coverage and will be in the dir that you ran py.test from. You can then use coverage itself to operate on that if you wish. Such as coverage --help coverage report coverage html Ned has docs at http://nedbatchelder.com/code/coverage pytest-cov provides access to all the report types, but having a .coveragerc can be good to get more control over how coverage operates such as turning branch coverage on. On 22 May 2011 17:56, Baptiste Lepilleur wrote: > That was it! > The restdoc mess was coming from an old version of the sources of py-1.3.4 > my third-parties sub-directory that py.test was attempting to execute. > The pytest-cov plug-in is exactly what I was looking for. > By the way, the documentation indicates that it is possible to use standard > coverage tool for result data analysis. Do those tool provided more info > than the html report? If so could you provide some relevant pointers (I've > never use python coverage tool before)? > Thanks a lot, > Baptiste. > 2011/5/22 meme dough >> >> You need an arg to --cov which is the source root you want covered. >> >> py.test -h >> >> ?coverage reporting with distributed testing support: >> ? ?--cov=path ? ? ? ? ?measure coverage for filesystem path >> (multi-allowed) >> >> Try something like: >> >> py.test --cov . src\nitroz\test_collections.py >> >> py.test --cov src src\nitroz\test_collections.py >> >> On 20 May 2011 07:53, Baptiste Lepilleur >> wrote: >> > Thanks for the pointer concerning pip, I did not know you could search >> > using >> > it. >> > Now, I running into a strange missing import error when trying to the >> > test >> > with coverage: >> > E ? ImportError: No module named restdoc >> > Any ideas what could be the cause and what this module is? >> > Detail below: >> > py.test --cov src\nitroz\test_collections.py >> > ================================================= test session starts >> > ================================================= >> > platform win32 -- Python 3.2.0 -- pytest-2.0.3 >> > collected 0 items / 1 errors >> > Coverage.py warning: No data was collected. >> > ======================================================= ERRORS >> > ======================================================== >> > _________________________________________________ ERROR collecting . >> > __________________________________________________ >> > C:\Python32\lib\site-packages\py-1.4.3-py3.2.egg\py\_path\common.py:312: >> > in >> > visit >> >> ? ? ? for x in Visitor(fil, rec, ignore, bf, sort).gen(self): >> > C:\Python32\lib\site-packages\py-1.4.3-py3.2.egg\py\_path\common.py:358: >> > in >> > gen >> >> ? ? ? ? ? ? ? for p in self.gen(subdir): >> > C:\Python32\lib\site-packages\py-1.4.3-py3.2.egg\py\_path\common.py:358: >> > in >> > gen >> >> ? ? ? ? ? ? ? for p in self.gen(subdir): >> > C:\Python32\lib\site-packages\py-1.4.3-py3.2.egg\py\_path\common.py:358: >> > in >> > gen >> >> ? ? ? ? ? ? ? for p in self.gen(subdir): >> > C:\Python32\lib\site-packages\py-1.4.3-py3.2.egg\py\_path\common.py:347: >> > in >> > gen >> >> ? ? ? dirs = self.optsort([p for p in entries >> > C:\Python32\lib\site-packages\py-1.4.3-py3.2.egg\py\_path\common.py:348: >> > in >> > >> >> ? ? ? ? ? ? ? if p.check(dir=1) and (rec is None or rec(p))]) >> > >> > C:\Python32\lib\site-packages\pytest-2.0.3-py3.2.egg\_pytest\main.py:450: in >> > _recurse >> >> ? ? ? ihook.pytest_collect_directory(path=path, parent=self) >> > >> > C:\Python32\lib\site-packages\pytest-2.0.3-py3.2.egg\_pytest\main.py:118: in >> > call_matching_hooks >> >> ? ? ? plugins = self.config._getmatchingplugins(self.fspath) >> > >> > C:\Python32\lib\site-packages\pytest-2.0.3-py3.2.egg\_pytest\config.py:283: >> > in _getmatchingplugins >> >> ? ? ? plugins += self._conftest.getconftestmodules(fspath) >> > >> > C:\Python32\lib\site-packages\pytest-2.0.3-py3.2.egg\_pytest\config.py:185: >> > in getconftestmodules >> >> ? ? ? ? ? ? ? ? ? ? ? clist.append(self.importconftest(conftestpath)) >> > >> > C:\Python32\lib\site-packages\pytest-2.0.3-py3.2.egg\_pytest\config.py:221: >> > in importconftest >> >> ? ? ? ? ? self._postimport(mod) >> > >> > C:\Python32\lib\site-packages\pytest-2.0.3-py3.2.egg\_pytest\config.py:226: >> > in _postimport >> >> ? ? ? ? ? self._onimport(mod) >> > >> > C:\Python32\lib\site-packages\pytest-2.0.3-py3.2.egg\_pytest\config.py:272: >> > in _onimportconftest >> >> ? ? ? self.pluginmanager.consider_conftest(conftestmodule) >> > >> > C:\Python32\lib\site-packages\pytest-2.0.3-py3.2.egg\_pytest\core.py:181: in >> > consider_conftest >> >> ? ? ? ? ? self.consider_module(conftestmodule) >> > >> > C:\Python32\lib\site-packages\pytest-2.0.3-py3.2.egg\_pytest\core.py:189: in >> > consider_module >> >> ? ? ? ? ? ? ? self.import_plugin(spec) >> > >> > C:\Python32\lib\site-packages\pytest-2.0.3-py3.2.egg\_pytest\core.py:202: in >> > import_plugin >> >> ? ? ? ? ? ? ? return self.import_plugin(modname[7:]) >> > >> > C:\Python32\lib\site-packages\pytest-2.0.3-py3.2.egg\_pytest\core.py:197: in >> > import_plugin >> >> ? ? ? ? ? mod = importplugin(modname) >> > >> > C:\Python32\lib\site-packages\pytest-2.0.3-py3.2.egg\_pytest\core.py:323: in >> > importplugin >> >> ? ? ? return __import__(importspec, None, None, '__doc__') >> > E ? ImportError: No module named restdoc >> > ----------------------------------- coverage: platform win32, python >> > 3.2.0-final-0 ------------------------------------ >> > Name ? ?Stmts ? Miss ?Cover >> > --------------------------- >> > ============================================== 1 error in 18.02 seconds >> > =============================================== >> > >> > >> > 2011/5/18 holger krekel >> >> >> >> Hey Baptiste, >> >> >> >> On Wed, May 18, 2011 at 17:42 +0200, Baptiste Lepilleur wrote: >> >> > The pytest documentation page indicates that it is supported, but >> >> > provides >> >> > no pointer on how do to this... >> >> >> >> > Doing a search seem to reveal multiple plug-ins to do that. What is >> >> > the >> >> > recommended one? I'm working on Windows XP / Python 2.6 & 3.2. >> >> >> >> sorry about that. ?The plugin is named pytest-cov, see here >> >> >> >> ? ?http://pypi.python.org/pypi/pytest-cov >> >> >> >> If it doesn't work for you i am sure Meme (also here on the list) >> >> can answer questions or to issues. >> >> >> >> > By the way, is there a centralized list of useful plug-ins for pytest >> >> > ? >> >> >> >> not really. ?However, i recommend to install "pip" and then type: >> >> >> >> ? ?pip search pytest >> >> >> >> to get a good list. >> >> >> >> best, >> >> holger >> >> >> >> >> >> >> >> > >> >> > Baptiste. >> >> >> >> > _______________________________________________ >> >> > py-dev mailing list >> >> > py-dev at codespeak.net >> >> > http://codespeak.net/mailman/listinfo/py-dev >> >> >> > >> > >> > _______________________________________________ >> > py-dev mailing list >> > py-dev at codespeak.net >> > http://codespeak.net/mailman/listinfo/py-dev >> > >> > > > From benjamin at python.org Thu May 26 18:42:36 2011 From: benjamin at python.org (Benjamin Peterson) Date: Thu, 26 May 2011 11:42:36 -0500 Subject: [py-dev] assertion code moving out of pylib Message-ID: Hello everyone, I've been working on refactoring the assertion debugging code for py.test. Part of this has involved moving the assertion code from pylib to py.test proper. I don't think many people are using the assertion related code from pylib, but I wanted to give a heads up I plan to remove it soon from pylib. If you do use pylib assertion code and removing it would be a problem for you, I'd like to hear from you. -- Regards, Benjamin From holger at merlinux.eu Sat May 28 16:56:42 2011 From: holger at merlinux.eu (holger krekel) Date: Sat, 28 May 2011 14:56:42 +0000 Subject: [py-dev] tox 1.0 - rapid multi-python test automation Message-ID: <20110528145642.GU20287@merlinux.eu> tox 1.0: the rapid multi-python test automation =========================================================================== I am happy to announce tox 1.0, a stabilization and maintenance release with some small improvements. tox automates tedious test activities driven from a simple ``tox.ini`` file, including: * creation and management of different virtualenv environments with different Python interpreters * packaging and installing your package into each of them * running your test tool of choice, be it nose, py.test or unittest2 or other tools such as "sphinx" doc checks * testing dev packages against each other without needing to upload to PyPI Docs and examples are now hosted at: http://tox.readthedocs.org Installation or upgrade with: pip install -U tox Note that code hosting and issue tracking has moved from Google to Bitbucket: http://bitbucket.org/hpk42/tox The 1.0 release includes contributions and is based on feedback and work from Chris Rose, Ronny Pfannschmidt, Jannis Leidel, Jakob Kaplan-Moss, Sridhar Ratnakumar, Carl Meyer and others. Many thanks! best, Holger Krekel CHANGES --------------------- - fix issue24: introduce a way to set environment variables for for test commands (thanks Chris Rose) - fix issue22: require virtualenv-1.6.1, obsoleting virtualenv5 (thanks Jannis Leidel) and making things work with pypy-1.5 and python3 more seemlessly - toxbootstrap.py (used by jenkins build slaves) now follows the latest release of virtualenv - fix issue20: document format of URLs for specifying dependencies - fix issue19: substitute Hudson for Jenkins everywhere following the renaming of the project. NOTE: if you used the special [tox:hudson] section it will now need to be named [tox:jenkins]. - fix issue 23 / apply some ReST fixes - change the positional argument specifier to use {posargs:} syntax and fix issues #15 and #10 by refining the argument parsing method (Chris Rose) - remove use of inipkg lazy importing logic - the namespace/imports are anyway very small with tox. - fix a fspath related assertion to work with debian installs which uses symlinks - show path of the underlying virtualenv invocation and bootstrap virtualenv.py into a working subdir - added a CONTRIBUTORS file From holger at merlinux.eu Sun May 29 09:43:27 2011 From: holger at merlinux.eu (holger krekel) Date: Sun, 29 May 2011 07:43:27 +0000 Subject: [py-dev] migration from 1.3 to 2.0 In-Reply-To: References: <20110405201612.GW16231@merlinux.eu> <20110407061715.GE16231@merlinux.eu> <20110430142225.GY4071@merlinux.eu> <20110503094814.GU4071@merlinux.eu> Message-ID: <20110529074327.GX20287@merlinux.eu> Hey Vyacheslav, On Wed, May 04, 2011 at 10:53 -0400, Vyacheslav Rafalskiy wrote: > On Tue, May 3, 2011 at 5:48 AM, holger krekel wrote: > > On Mon, May 02, 2011 at 17:40 -0400, Vyacheslav Rafalskiy wrote: > >> On Sat, Apr 30, 2011 at 10:22 AM, holger krekel wrote: > >> > On Thu, Apr 07, 2011 at 12:29 -0400, Vyacheslav Rafalskiy wrote: > >> > >> >> > >> >> >(sidenote: configure and even sessionstart hooks are both a bit > >> >> > not 100% right because they happen even on the master side of a distributed > >> >> > test run and the master side does not collect or run tests at all) > >> >> > >> >> I see. Perhaps something like setup_package() in the top-level __init__.py > >> >> could be a solution? > >> > > >> > I guess you mean an __init__.py file of a test directory. > >> > With a layout of test dirs within an application this might mean > >> > one has to put this setup into the main package __init__.py > >> > and mixing test code and application code is often not a good idea. > >> > >> Yes, exactly. In my case of functional testing I don't even have > >> application code here. > >> When I start the tests I tell the runner where in the network the > >> system under test is. > >> > >> > > >> > So i'd rather put it into a conftest.py file as a normal hook. > >> > Maybe "pytest_pyfunc_setup(request)" would be good where request > >> > is the same object as for the funcarg factories. > >> > > >> > You could then write: > >> > > >> > ? ?# content of conftest.py > >> > ? ?def pytest_pyfunc_setup(request): > >> > ? ? ? ?val = request.cached_setup(setup=makeval, scope="session") > >> > ? ? ? ?# use val for some global setting of the package > >> > > >> > Alternatively we could see to call something like: > >> > > >> > ? ?def pytest_setup_testloop(config): > >> > ? ? ? ?val = makeval() > >> > ? ? ? ?# use val for some global setting of the package > >> > > >> > ? ?def pytest_teardown_testloop(config): > >> > ? ? ? ?... > >> > > >> > which would be called once for a test process. > >> > >> The reason why I suggested setup_package() is that you already have > >> setup_function, setup_method, setup_class and setup_module so > >> the former would just complete the line-up. And the natural place > >> for it would be __init__.py of that package. > >> > >> On the other hand, you can put conftest.py in every folder, which > >> can do precisely the same thing as well as many others. The > >> question is which way is more intuitive and results in cleaner code. > >> The answer is perhaps "It depends". > >> > >> I like setup_module(module) because it lets me dump the configuration > >> straight into the namespace where I use it and setup_package(package) > >> could do the same. > > > > good line of reasoning. ?It's mostly my intuition making me hesitant > > to add setup_package like you suggest. ?And i wonder what it is about :) > > Maybe it's that the root namespace of a test directory is often called > > "testing" or "tests" ?(the "test" one is taken by Python stdlib already). > > And therefore you would end up having to do "import testing" and > > then use global state with something like "testing.STATE". > > But i guess this doesn't look so bad to you, does it? :) > > (The plugin/conftest system is designed such that it hardly > > requires any imports to manage test state.) > > > > Any more opinions on setup_package(package)? If others find it useful > > as well, i will consider introducing it with pytest-2.1. > > I guess I will have to withdraw the idea. Having to explicitly import > the test package does not look nice at all. > conftest.py rules! > > As to the two alternatives above I'd rather use > pytest_setup_testloop(config) with direct access to config. I am now pondering following your original intention and introduce a "setup_directory" to be put into conftest.py files. You could then access the config object via pytest.config. Would that make sense to you as well? best, holger > Regards, > Vyacheslav > > > > > best, > > holger > > > From holger at merlinux.eu Sun May 29 09:59:38 2011 From: holger at merlinux.eu (holger krekel) Date: Sun, 29 May 2011 07:59:38 +0000 Subject: [py-dev] setup_directory / xUnit extension? Re: migration from 1.3 to 2.0 In-Reply-To: <20110529074327.GX20287@merlinux.eu> References: <20110405201612.GW16231@merlinux.eu> <20110407061715.GE16231@merlinux.eu> <20110430142225.GY4071@merlinux.eu> <20110503094814.GU4071@merlinux.eu> <20110529074327.GX20287@merlinux.eu> Message-ID: <20110529075938.GY20287@merlinux.eu> Hey again, On Sun, May 29, 2011 at 07:43 +0000, holger krekel wrote: > > > good line of reasoning. ?It's mostly my intuition making me hesitant > > > to add setup_package like you suggest. ?And i wonder what it is about :) > > > Maybe it's that the root namespace of a test directory is often called > > > "testing" or "tests" ?(the "test" one is taken by Python stdlib already). > > > And therefore you would end up having to do "import testing" and > > > then use global state with something like "testing.STATE". > > > But i guess this doesn't look so bad to you, does it? :) > > > (The plugin/conftest system is designed such that it hardly > > > requires any imports to manage test state.) > > > > > > Any more opinions on setup_package(package)? If others find it useful > > > as well, i will consider introducing it with pytest-2.1. > > > > I guess I will have to withdraw the idea. Having to explicitly import > > the test package does not look nice at all. > > conftest.py rules! > > > > As to the two alternatives above I'd rather use > > pytest_setup_testloop(config) with direct access to config. > > I am now pondering following your original intention and introduce a > "setup_directory" to be put into conftest.py files. You could then > access the config object via pytest.config. Would that make sense > to you as well? To elaborate a wee bit: * setup_directory would be guranteed to be called for any test (both python, doctest or other test) within the directory hierarchy of the conftest.py dir and before any setup_module/class etc. is called. * teardown_directory would be guranteed to be called when a test is run that is not in the directory hierarchy. * if neccessary one can have setup_directory push test related state to some global module (from which tests could import for example) i am not yet sure about the idea but i guess it would be somewhat natural and complete the setup_*/teardown_* xUnit style fixture methods. holger > best, > holger > > > Regards, > > Vyacheslav > > > > > > > > best, > > > holger > > > > > > From rafalskiy at gmail.com Mon May 30 20:24:57 2011 From: rafalskiy at gmail.com (Vyacheslav Rafalskiy) Date: Mon, 30 May 2011 14:24:57 -0400 Subject: [py-dev] setup_directory / xUnit extension? Re: migration from 1.3 to 2.0 In-Reply-To: <20110529075938.GY20287@merlinux.eu> References: <20110405201612.GW16231@merlinux.eu> <20110407061715.GE16231@merlinux.eu> <20110430142225.GY4071@merlinux.eu> <20110503094814.GU4071@merlinux.eu> <20110529074327.GX20287@merlinux.eu> <20110529075938.GY20287@merlinux.eu> Message-ID: On Sun, May 29, 2011 at 3:59 AM, holger krekel wrote: > Hey again, > > On Sun, May 29, 2011 at 07:43 +0000, holger krekel wrote: >> > > good line of reasoning. ?It's mostly my intuition making me hesitant >> > > to add setup_package like you suggest. ?And i wonder what it is about :) >> > > Maybe it's that the root namespace of a test directory is often called >> > > "testing" or "tests" ?(the "test" one is taken by Python stdlib already). >> > > And therefore you would end up having to do "import testing" and >> > > then use global state with something like "testing.STATE". >> > > But i guess this doesn't look so bad to you, does it? :) >> > > (The plugin/conftest system is designed such that it hardly >> > > requires any imports to manage test state.) >> > > >> > > Any more opinions on setup_package(package)? If others find it useful >> > > as well, i will consider introducing it with pytest-2.1. >> > >> > I guess I will have to withdraw the idea. Having to explicitly import >> > the test package does not look nice at all. >> > conftest.py rules! >> > >> > As to the two alternatives above I'd rather use >> > pytest_setup_testloop(config) with direct access to config. >> >> I am now pondering following your original intention and introduce a >> "setup_directory" to be put into conftest.py files. ?You could then >> access the config object via pytest.config. Would that make sense >> to you as well? > > To elaborate a wee bit: > > * setup_directory would be guranteed to be called for any test > ?(both python, doctest or other test) within the directory hierarchy > ?of the conftest.py dir and before any setup_module/class etc. is called. > > * teardown_directory would be guranteed to be called when a test > ?is run that is not in the directory hierarchy. > > * if neccessary one can have setup_directory push test related > ?state to some global module (from which tests could import for example) > > i am not yet sure about the idea but i guess it would be somewhat natural > and complete the setup_*/teardown_* xUnit style fixture methods. > > holger Sounds great to me. Not much to add. Perhaps a parameter, say directory_config, which can then be imported by a test module like for example from pytest import directory_config This object could have different meaning depending on the directory where the test module resides. Vyacheslav From rafalskiy at gmail.com Mon May 30 21:17:32 2011 From: rafalskiy at gmail.com (Vyacheslav Rafalskiy) Date: Mon, 30 May 2011 15:17:32 -0400 Subject: [py-dev] Decorators and funcargs in py.test Message-ID: Hi Holger, I am trying to make decorators work with test functions, which depend on funcargs. As it stands, they don't. Decorated functions lose funcargs. A workaround would be to decorate an internal function like this: def test_it(funcarg_it): @decorate_it def _test_it(): # test it _test_it() This works, but it is not nice. I'd rather wrote a decorator like def decorate_it(f): def _wrap_it(*args, **kwargs): # wrap f() here _wrap_it._varnames = _pytest.core.varnames(f) return _wrap_it and apply it straight to the test function. After examining the source code I even expected it to just work (magically of course) but it didn't. Do you think it is worthwhile? If so I can enter a feature request. Thanks, Vyacheslav From benjamin at python.org Mon May 30 21:23:12 2011 From: benjamin at python.org (Benjamin Peterson) Date: Mon, 30 May 2011 14:23:12 -0500 Subject: [py-dev] Decorators and funcargs in py.test In-Reply-To: References: Message-ID: 2011/5/30 Vyacheslav Rafalskiy : > Hi Holger, > > I am trying to make decorators work with test functions, which depend > on funcargs. As it stands, they don't. > Decorated functions lose funcargs. A workaround would be to decorate > an internal function like this: > > def test_it(funcarg_it): > ? ?@decorate_it > ? ?def _test_it(): > ? ? ? ?# test it > > ? ?_test_it() > > This works, but it is not nice. I'd rather wrote a decorator like > > def decorate_it(f): > ? ?def _wrap_it(*args, **kwargs): > ? ? ? ?# wrap f() here > > ? ?_wrap_it._varnames = _pytest.core.varnames(f) > ? ?return _wrap_it > > and apply it straight to the test function. > > After examining the source code I even expected it to just work > (magically of course) but it didn't. > Do you think it is worthwhile? If so I can enter a feature request. It's a difficult problem because there's no way to know if a function is being decorated. I suppose py.test could look for a _decorated_ attribute but decorators would have to conform and set that attribute. -- Regards, Benjamin From Ronny.Pfannschmidt at gmx.de Mon May 30 21:55:15 2011 From: Ronny.Pfannschmidt at gmx.de (Ronny Pfannschmidt) Date: Mon, 30 May 2011 21:55:15 +0200 Subject: [py-dev] Decorators and funcargs in py.test In-Reply-To: References: Message-ID: <1306785315.3360.1.camel@Klappe2> Hi, can you try to explain the usecase those decorators are fulfilling, there may be a better integrated way using pytest.mark + setup/teardown hooks -- Ronny On Mon, 2011-05-30 at 15:17 -0400, Vyacheslav Rafalskiy wrote: > Hi Holger, > > I am trying to make decorators work with test functions, which depend > on funcargs. As it stands, they don't. > Decorated functions lose funcargs. A workaround would be to decorate > an internal function like this: > > def test_it(funcarg_it): > @decorate_it > def _test_it(): > # test it > > _test_it() > > This works, but it is not nice. I'd rather wrote a decorator like > > def decorate_it(f): > def _wrap_it(*args, **kwargs): > # wrap f() here > > _wrap_it._varnames = _pytest.core.varnames(f) > return _wrap_it > > and apply it straight to the test function. > > After examining the source code I even expected it to just work > (magically of course) but it didn't. > Do you think it is worthwhile? If so I can enter a feature request. > > Thanks, > Vyacheslav > _______________________________________________ > py-dev mailing list > py-dev at codespeak.net > http://codespeak.net/mailman/listinfo/py-dev -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: This is a digitally signed message part URL: From Ronny.Pfannschmidt at gmx.de Mon May 30 22:38:30 2011 From: Ronny.Pfannschmidt at gmx.de (Ronny Pfannschmidt) Date: Mon, 30 May 2011 22:38:30 +0200 Subject: [py-dev] Decorators and funcargs in py.test In-Reply-To: References: <1306785315.3360.1.camel@Klappe2> Message-ID: <1306787910.3360.15.camel@Klappe2> On Mon, 2011-05-30 at 16:23 -0400, Vyacheslav Rafalskiy wrote: > No problem. Here is my (real life) example. > > My functional test functions may or may not return for different > reasons (like a faulty web application or middleware). I want to > declare a fail if it takes more than so many seconds to complete. So I > write a decorator > run_with_timeout(), which will start the function in a new thread and > abandon it after timeout. This can easily be solved by combining something like pytest.mark('timeout') and a override to pytest_pyfunc_call using it i suppose this could also take a look at extending the "--boxed" mode (which forks for each test and uses subprocesses but doesn?t handle timeouts atm. > > A I stated in OP, it is not that I cannot do it. I can and I do. My > point is that it makes sense to allow decorators and should not be > very difficult (see the example in OP). personally i am opposed to creating new data conventions for problems that can be solved with plain marks + a hook usually there are 2 reasons people use to decorate tests my opinions for implementing those are a) add arguments + their cleanups -> funcargs please, they are made for that b) use more sophisticated call's -> hooks please, maybe add a bug ticket for empowering one to return a exception-info, so stuff like thread-wrappers can pass that more nicely for failures *i* really think it is wrong to decorate test functions that way and expect stuff to work there are already plenty of mechanisms to change the behavior of pytest test function calling in the desired way, none of those require hacks to pass around argspecs -- Ronny > > Thanks, > Vyacheslav > > On Mon, May 30, 2011 at 3:55 PM, Ronny Pfannschmidt > wrote: > > Hi, > > > > can you try to explain the usecase those decorators are fulfilling, > > > > there may be a better integrated way using pytest.mark + setup/teardown > > hooks > > > > -- Ronny > > > > On Mon, 2011-05-30 at 15:17 -0400, Vyacheslav Rafalskiy wrote: > >> Hi Holger, > >> > >> I am trying to make decorators work with test functions, which depend > >> on funcargs. As it stands, they don't. > >> Decorated functions lose funcargs. A workaround would be to decorate > >> an internal function like this: > >> > >> def test_it(funcarg_it): > >> @decorate_it > >> def _test_it(): > >> # test it > >> > >> _test_it() > >> > >> This works, but it is not nice. I'd rather wrote a decorator like > >> > >> def decorate_it(f): > >> def _wrap_it(*args, **kwargs): > >> # wrap f() here > >> > >> _wrap_it._varnames = _pytest.core.varnames(f) > >> return _wrap_it > >> > >> and apply it straight to the test function. > >> > >> After examining the source code I even expected it to just work > >> (magically of course) but it didn't. > >> Do you think it is worthwhile? If so I can enter a feature request. > >> > >> Thanks, > >> Vyacheslav > >> _______________________________________________ > >> py-dev mailing list > >> py-dev at codespeak.net > >> http://codespeak.net/mailman/listinfo/py-dev > > > > -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: This is a digitally signed message part URL: