From issues-reply at bitbucket.org Sat Sep 1 09:18:52 2012 From: issues-reply at bitbucket.org (Alex Gaynor) Date: Sat, 01 Sep 2012 07:18:52 -0000 Subject: [py-dev] [hpk42/pytest] Monkeypatching time causes pytest to break (issue #185) Message-ID: <941a18bc822bf109802fb5ea46bf7ef7@bitbucket.org> --- you can reply above this line --- New issue 185: Monkeypatching time causes pytest to break https://bitbucket.org/hpk42/pytest/issue/185/monkeypatching-time-causes-pytest-to-break Alex Gaynor: The new timing mechanism in pytest requires `time.time()` to work, if you use the `monkeypatch` funcarg, `time.time()` is called before it is torn down, causing things to explode. The following test is enough to break thigns: {{{ #!python import time def test_m(monkeypatch): def f(): raise Exception monkeypatch.setattr(time, "time", f) }}} -- This is an issue notification from bitbucket.org. You are receiving this either because you are the owner of the issue, or you are following the issue. From issues-reply at bitbucket.org Mon Sep 3 10:14:30 2012 From: issues-reply at bitbucket.org (Ronny Pfannschmidt) Date: Mon, 03 Sep 2012 08:14:30 -0000 Subject: [py-dev] [hpk42/pytest] python files in the japanese documentation cause collection errors (issue #186) Message-ID: --- you can reply above this line --- New issue 186: python files in the japanese documentation cause collection errors https://bitbucket.org/hpk42/pytest/issue/186/python-files-in-the-japanese-documentation Ronny Pfannschmidt: -- This is an issue notification from bitbucket.org. You are receiving this either because you are the owner of the issue, or you are following the issue. From issues-reply at bitbucket.org Wed Sep 5 21:39:57 2012 From: issues-reply at bitbucket.org (amcnabb8) Date: Wed, 05 Sep 2012 19:39:57 -0000 Subject: [py-dev] [hpk42/pytest] Using both the --doctest-modules and -m options causes a crash (issue #187) Message-ID: --- you can reply above this line --- New issue 187: Using both the --doctest-modules and -m options causes a crash https://bitbucket.org/hpk42/pytest/issue/187/using-both-the-doctest-modules-and-m amcnabb8: If I use both the --doctest-modules and -m options, I get the error: INTERNALERROR> AttributeError: 'DoctestModule' object has no attribute 'obj' I am attaching the full backtrace. On this system, pytest was installed with the pytest-2.2.4-1.fc17.noarch RPM. -- This is an issue notification from bitbucket.org. You are receiving this either because you are the owner of the issue, or you are following the issue. From issues-reply at bitbucket.org Sat Sep 15 14:57:40 2012 From: issues-reply at bitbucket.org (Ronny Pfannschmidt) Date: Sat, 15 Sep 2012 12:57:40 -0000 Subject: [py-dev] [hpk42/pytest] on python2 we need to call sys.exc_clear or sys.exc_info will stay what it was from the last exception (issue #188) Message-ID: <65bfe5d03ff9cf9e043906a8b56f3f6c@bitbucket.org> --- you can reply above this line --- New issue 188: on python2 we need to call sys.exc_clear or sys.exc_info will stay what it was from the last exception https://bitbucket.org/hpk42/pytest/issue/188/on-python2-we-need-to-call-sysexc_clear-or Ronny Pfannschmidt: this surprised me when testing pdbmm post mortem - and the last test has sys.exc_info being the IndexError of nextitem finding -- This is an issue notification from bitbucket.org. You are receiving this either because you are the owner of the issue, or you are following the issue. From issues-reply at bitbucket.org Tue Sep 18 12:04:32 2012 From: issues-reply at bitbucket.org (Thomas Waldmann) Date: Tue, 18 Sep 2012 10:04:32 -0000 Subject: [py-dev] [hpk42/pytest] crash with non-ascii output (issue #189) Message-ID: <5b82d50e03fbf2f11b1a3ceb83099ad9@bitbucket.org> --- you can reply above this line --- New issue 189: crash with non-ascii output https://bitbucket.org/hpk42/pytest/issue/189/crash-with-non-ascii-output Thomas Waldmann: py.test with pep8 plugin: {{{ ... =================================== FAILURES =================================== PEP8-check(ignoring E121 E122 E123 E124 E125 E126 E127 E128 E262 E301 E302 E401 E501) /home/tw/w/whoosh/src/whoosh/lang/phonetic.py:53:27: E261 at least two spaces before inline comment ("d$", "t"), # Change a trailing d to a t ^ PEP8-check(ignoring E121 E122 E123 E124 E125 E126 E127 E128 E262 E301 E302 E401 E501) Traceback (most recent call last): File "/home/tw/w/whoosh/env/bin/py.test", line 9, in load_entry_point('pytest==2.2.4', 'console_scripts', 'py.test')() File "/home/tw/w/whoosh/env/local/lib/python2.7/site-packages/_pytest/core.py", line 469, in main exitstatus = config.hook.pytest_cmdline_main(config=config) File "/home/tw/w/whoosh/env/local/lib/python2.7/site-packages/_pytest/core.py", line 421, in __call__ return self._docall(methods, kwargs) File "/home/tw/w/whoosh/env/local/lib/python2.7/site-packages/_pytest/core.py", line 432, in _docall res = mc.execute() File "/home/tw/w/whoosh/env/local/lib/python2.7/site-packages/_pytest/core.py", line 350, in execute res = method(**kwargs) File "/home/tw/w/whoosh/env/local/lib/python2.7/site-packages/_pytest/main.py", line 100, in pytest_cmdline_main return wrap_session(config, _main) File "/home/tw/w/whoosh/env/local/lib/python2.7/site-packages/_pytest/main.py", line 92, in wrap_session exitstatus=session.exitstatus or (session._testsfailed and 1)) File "/home/tw/w/whoosh/env/local/lib/python2.7/site-packages/_pytest/core.py", line 421, in __call__ return self._docall(methods, kwargs) File "/home/tw/w/whoosh/env/local/lib/python2.7/site-packages/_pytest/core.py", line 432, in _docall res = mc.execute() File "/home/tw/w/whoosh/env/local/lib/python2.7/site-packages/_pytest/core.py", line 350, in execute res = method(**kwargs) File "/home/tw/w/whoosh/env/local/lib/python2.7/site-packages/_pytest/terminal.py", line 318, in pytest_sessionfinish self.summary_failures() File "/home/tw/w/whoosh/env/local/lib/python2.7/site-packages/_pytest/terminal.py", line 397, in summary_failures self._outrep_summary(rep) File "/home/tw/w/whoosh/env/local/lib/python2.7/site-packages/_pytest/terminal.py", line 418, in _outrep_summary rep.toterminal(self._tw) File "/home/tw/w/whoosh/env/local/lib/python2.7/site-packages/_pytest/runner.py", line 157, in toterminal out.line(str(longrepr)) UnicodeEncodeError: 'ascii' codec can't encode character u'\xc0' in position 2103: ordinal not in range(128) }}} -- This is an issue notification from bitbucket.org. You are receiving this either because you are the owner of the issue, or you are following the issue. From holger at merlinux.eu Tue Sep 18 15:14:40 2012 From: holger at merlinux.eu (holger krekel) Date: Tue, 18 Sep 2012 13:14:40 +0000 Subject: [py-dev] RFC for (hopefully) final commments on pytest-2.3 API Message-ID: <20120918131440.GU26019@merlinux.eu> Hi Brianna, Floris, Carl, Ronny, all, first, let me thank for all the useful feedback you already provided. I am back trying to finalize the next pytest release -- FYI trunk is actually fully functional and implements all previously discussed features. However, I am still pondering Brianna's feedback (that that she found it surprising that there is a @pytest.setup but no teardown) and a few other little naming and protocol issues. If you find some time to state your opinion on one or another issue i'd very much appreciate it: - rename @pytest.setup to @pytest.fixture and the docs would talk about fixture instead of setup functions. (See http://pytest.org/dev/setup.html for the current docs) This would induce less of a question about @pytest.teardown?! - rename @pytest.factory to @pytest.funcarg - because with pytest-trunk it's already the case that the former pytest_funcarg__NAME is just a shortcut - there is no internal distinction anymore if you use the decorator or the prefix or both. Removing the "pytest_funcarg__" prefix and using a @pytest.funcarg(...) declaration looks naturally related?! - introduce a "resource-centric" convenience protocol for fixtures and funcarg factories. It would be used if you decorate a class instead of a function and would lead to the calling of (optional) setup/teardown methods. Here is an example: # content of test_module.py import pytest @pytest.funcarg class db: def setup(self, funcarg1, funcarg2): # called during the setup phase of each test using the "db" # funcarg. this setup function will be called multiple # times in case db.params is defined def teardown(self): # called during the teardown phase of a test which # previously saw a successful db.setup() call - moreover, there could optionally be a class-method "configure" which is called ahead of any setup() and would allow to dynamically compute the scope and params attribute and influence them e. g. from command line options:: @pytest.funcarg class db: @classmethod def configure(cls, config): scope = config.option.scope params = ... ... Dynamic scoping/parametrization is used today if you want to have broadly-scoped less-parametrized resources during development but use more function-scope more-parametrized resources during CI runs. - the same class convenience protocol would work for fixtures: import pytest @pytest.fixture class transact: def setup(self, db): db.begin() self.db = db def teardown(self): self.db.commit() best, holger From Ronny.Pfannschmidt at gmx.de Tue Sep 18 21:56:35 2012 From: Ronny.Pfannschmidt at gmx.de (Ronny Pfannschmidt) Date: Tue, 18 Sep 2012 21:56:35 +0200 Subject: [py-dev] RFC for (hopefully) final commments on pytest-2.3 API In-Reply-To: <20120918131440.GU26019@merlinux.eu> References: <20120918131440.GU26019@merlinux.eu> Message-ID: <5058D1F3.4090604@gmx.de> Hi Holger, the proposed class centric convenience protocol strikes me as strange/odd/confusing - i'd like to avoid to be finalized on such an api the rest of the changes looks reasonable/good -- Ronny On 09/18/2012 03:14 PM, holger krekel wrote: > Hi Brianna, Floris, Carl, Ronny, all, > > first, let me thank for all the useful feedback you already provided. > I am back trying to finalize the next pytest release -- FYI trunk is actually > fully functional and implements all previously discussed features. > > However, I am still pondering Brianna's feedback (that that she found it > surprising that there is a @pytest.setup but no teardown) and a few > other little naming and protocol issues. If you find some time to state > your opinion on one or another issue i'd very much appreciate it: > > - rename @pytest.setup to @pytest.fixture and the docs would talk > about fixture instead of setup functions. (See > http://pytest.org/dev/setup.html for the current docs) > This would induce less of a question about @pytest.teardown?! > > - rename @pytest.factory to @pytest.funcarg - because with pytest-trunk > it's already the case that the former pytest_funcarg__NAME is just > a shortcut - there is no internal distinction anymore if you use > the decorator or the prefix or both. Removing the "pytest_funcarg__" prefix > and using a @pytest.funcarg(...) declaration looks naturally related?! > > - introduce a "resource-centric" convenience protocol for fixtures and > funcarg factories. It would be used if you decorate a class instead of > a function and would lead to the calling of (optional) setup/teardown > methods. Here is an example: > > # content of test_module.py > > import pytest > > @pytest.funcarg > class db: > def setup(self, funcarg1, funcarg2): > # called during the setup phase of each test using the "db" > # funcarg. this setup function will be called multiple > # times in case db.params is defined > > def teardown(self): > # called during the teardown phase of a test which > # previously saw a successful db.setup() call > > - moreover, there could optionally be a class-method "configure" which is > called ahead of any setup() and would allow to dynamically compute the scope > and params attribute and influence them e. g. from command line options:: > > @pytest.funcarg > class db: > @classmethod > def configure(cls, config): > scope = config.option.scope > params = ... > > ... > > Dynamic scoping/parametrization is used today if you want to have > broadly-scoped less-parametrized resources during development but > use more function-scope more-parametrized resources during CI runs. > > > - the same class convenience protocol would work for fixtures: > > import pytest > @pytest.fixture > class transact: > def setup(self, db): > db.begin() > self.db = db > def teardown(self): > self.db.commit() > > best, > holger > _______________________________________________ > py-dev mailing list > py-dev at codespeak.net > http://codespeak.net/mailman/listinfo/py-dev From flub at devork.be Wed Sep 19 02:00:15 2012 From: flub at devork.be (Floris Bruynooghe) Date: Wed, 19 Sep 2012 01:00:15 +0100 Subject: [py-dev] pytest-timeout 0.3: py.test plugin to abort long-running tests Message-ID: Hello all, I am pleased to announce the release of pytest-timeout 0.3. http://pypi.python.org/pypi/pytest-timeout/0.3 pytest-timeout is a plugin for py.test which allows you to abort tests after a specific timeout. This is useful to e.g. handle hanging tests on a continuous integration server. When tests are aborted you will get stack dumps of all threads in the python process to help you debug why the test hung. All feedback is welcome. Please use the bug tracker on bitbucket to report any issues: https://bitbucket.org/flub/pytest-timeout/ Changelog since 0.2: * Added the PYTEST_TIMEOUT environment variable as a way of specifying the timeout (closes issue #2). * More flexible marker argument parsing: you can now specify the method using a positional argument. * The plugin is now enabled by default. There is no longer a need to specify ``timeout=0`` in the configuration file or on the command line simply so that a marker would work. Regards, Floris From issues-reply at bitbucket.org Fri Sep 21 23:22:15 2012 From: issues-reply at bitbucket.org (Alex Okrushko) Date: Fri, 21 Sep 2012 21:22:15 -0000 Subject: [py-dev] [hpk42/pytest] Naming test methods "setup" causes different behavior (issue #190) Message-ID: --- you can reply above this line --- New issue 190: Naming test methods "setup" causes different behavior https://bitbucket.org/hpk42/pytest/issue/190/naming-test-methods-setup-causes-different Alex Okrushko: depending on the name of the setup-method the behavior is completely different: 1-st scenario: {{{ #!python class TestMyProp(object): @pytest.setup(scope = "class") def stp(self): self.prop = "init" print self.prop def test_prop(self): TestMyProp.prop = "changed" print self.prop def test_prop_again(self): print self.prop }}} OUTPUT: py.test -sq {{{ #!output ============================= test session starts ============================== platform win32 -- Python 2.7.3 -- pytest-2.3.0.dev15 collecting ... collected 2 items my_test.py init changed .changed . =========================== 2 passed in 0.01 seconds =========================== }}} This is the expected result. NOW change the name of the method to "setup": {{{ #!python class TestMyProp(object): @pytest.setup(scope = "class") def setup(self): self.prop = "init" print self.prop def test_prop(self): TestMyProp.prop = "changed" print self.prop def test_prop_again(self): print self.prop }}} and the output becomes: {{{ #!output ============================= test session starts ============================== platform win32 -- Python 2.7.3 -- pytest-2.3.0.dev15 collecting ... collected 2 items my_test.py init init init .init init . =========================== 2 passed in 0.02 seconds =========================== }}} -- This is an issue notification from bitbucket.org. You are receiving this either because you are the owner of the issue, or you are following the issue. From issues-reply at bitbucket.org Sat Sep 22 16:47:19 2012 From: issues-reply at bitbucket.org (Anonymous) Date: Sat, 22 Sep 2012 14:47:19 -0000 Subject: [py-dev] [hpk42/pytest] Support for unittests TestCase.runTest syntax (issue #191) Message-ID: --- you can reply above this line --- New issue 191: Support for unittests TestCase.runTest syntax https://bitbucket.org/hpk42/pytest/issue/191/support-for-unittests-testcaseruntest Anonymous: I'm currently maintaining a collection of unittest.TestCases with most of them containing runTest methods. This syntax seems unfortunately to be unsupported by py.test. Subclassing from TestCase and extending this class with some redirect from TestSubclass.test_run to TestSubclass.runTest to provide compatibility, is possible, but can require some rewrites. Monkeypatching of TestCase with such a function within a generated runtests script seems to work, but is also not the prettiest solution. It would really be nice, if this quite usual syntax of unittest would also be supported out of the box. -- This is an issue notification from bitbucket.org. You are receiving this either because you are the owner of the issue, or you are following the issue. From flub at devork.be Sun Sep 23 18:45:25 2012 From: flub at devork.be (Floris Bruynooghe) Date: Sun, 23 Sep 2012 17:45:25 +0100 Subject: [py-dev] Inspecting requested funcargs Message-ID: Hi Holger, The new api tried to make it possible to know in advance which funcargs are required for a test function which is a very nice thing to have. However I'm not sure how to get to the full list: @pytest.factory() def arg0(): pass @pytest.factory() def arg1(arg0): pass @pytest.setup() def setup(request): print request.node.funcargnames def test_foo(arg1): pass When executing this with py.test -s this only prints ['arg1'] and I have no idea how to find out that arg0 has also been created/requested. Is there a way to know all the requested funcargs? Floris From flub at devork.be Sun Sep 23 22:23:11 2012 From: flub at devork.be (Floris Bruynooghe) Date: Sun, 23 Sep 2012 21:23:11 +0100 Subject: [py-dev] Ordering of setup and funcargs Message-ID: Hello, As I understand it the order of setup functions being called is: pytest_runtest_setup funcarg resources @setup marked functions And if you mark pytest_runtest_setup with trylast it will be moved down to the bottom. But it is impossible to get a @setup marked function to be called before the funcarg resources are called. This means that in order to be able to influence what is possible inside funcargs [0] you still need to use pytest_runtest_setup. I have no particular opinion on this (yet) but was surprised about this so thought I'd point it out since it still forces plugin writers to use pytest_runtest_setup in some cases. [0] In my case monkeypatch django so you can't get access to it's database using all of django's global state. Regards, Floris From holger at merlinux.eu Mon Sep 24 10:40:21 2012 From: holger at merlinux.eu (holger krekel) Date: Mon, 24 Sep 2012 08:40:21 +0000 Subject: [py-dev] Ordering of setup and funcargs In-Reply-To: References: Message-ID: <20120924084021.GC27773@merlinux.eu> Hi Floris, On Sun, Sep 23, 2012 at 21:23 +0100, Floris Bruynooghe wrote: > Hello, > > As I understand it the order of setup functions being called is: > > pytest_runtest_setup > funcarg resources > @setup marked functions > > And if you mark pytest_runtest_setup with trylast it will be moved > down to the bottom. But it is impossible to get a @setup marked > function to be called before the funcarg resources are called. This > means that in order to be able to influence what is possible inside > funcargs [0] you still need to use pytest_runtest_setup. I have no > particular opinion on this (yet) but was surprised about this so > thought I'd point it out since it still forces plugin writers to use > pytest_runtest_setup in some cases. I just made sure setups are called ahead of the funcarg factories of the main function. I think it makes more sense this way. For example, my setup functions sometimes decide to skip a test and in this case there is no need to further setup the main function and its funcargs. best, holger > > [0] In my case monkeypatch django so you can't get access to it's > database using all of django's global state. > > Regards, > Floris > _______________________________________________ > py-dev mailing list > py-dev at codespeak.net > http://codespeak.net/mailman/listinfo/py-dev > From holger at merlinux.eu Mon Sep 24 17:11:11 2012 From: holger at merlinux.eu (holger krekel) Date: Mon, 24 Sep 2012 15:11:11 +0000 Subject: [py-dev] Inspecting requested funcargs In-Reply-To: References: Message-ID: <20120924151111.GE27773@merlinux.eu> Hi Floris, On Sun, Sep 23, 2012 at 17:45 +0100, Floris Bruynooghe wrote: > Hi Holger, > > The new api tried to make it possible to know in advance which > funcargs are required for a test function which is a very nice thing > to have. However I'm not sure how to get to the full list: > > @pytest.factory() > def arg0(): > pass > > @pytest.factory() > def arg1(arg0): > pass > > @pytest.setup() > def setup(request): > print request.node.funcargnames > > def test_foo(arg1): > pass > > When executing this with py.test -s this only prints ['arg1'] and I > have no idea how to find out that arg0 has also been > created/requested. Is there a way to know all the requested funcargs? This is solved now if you access "request.funcargnames" which contains (along with metafunc.funcargnames) the full closure of all used funcargs for the test function in question. If one wants to just see the test function's funcargnames there is request.node.funcargnames. best, holger From flub at devork.be Wed Sep 26 01:52:46 2012 From: flub at devork.be (Floris Bruynooghe) Date: Wed, 26 Sep 2012 00:52:46 +0100 Subject: [py-dev] RFC for (hopefully) final commments on pytest-2.3 API In-Reply-To: <20120918131440.GU26019@merlinux.eu> References: <20120918131440.GU26019@merlinux.eu> Message-ID: Hi Only providing feedback on the naming issues so far. After some usage of this API I have changed my mind slightly on these. On 18 September 2012 14:14, holger krekel wrote: > - rename @pytest.setup to @pytest.fixture +1 > - rename @pytest.factory to @pytest.funcarg I would rather like to keep @pytest.factory. While writing plugin code I find it much clearer thinking about a resource rather then a funcarg. And @factory is short for "resource factory" so that works. I think consistently using the name resource in the documentation and only mention pytest_funcarg__foo as a legacy aproach will be clearer. I think it is also clearer to explain a resource and a fixture next to each other, while for a funcarg the explanation would become longer. Lastly I'd probably vote for "testcontext" as name for the funcarg currently provided by "request". This is a bit more annoying to type but less likely to conflict with user provided resources. Thanks for all the work this, and for putting up with all my requests and questions. Floris From issues-reply at bitbucket.org Thu Sep 27 11:52:00 2012 From: issues-reply at bitbucket.org (Alex Greif) Date: Thu, 27 Sep 2012 09:52:00 -0000 Subject: [py-dev] [hpk42/pytest] looponfailing mode not working in 2.2.4 (issue #192) Message-ID: <2418a531ab008d4bfb16f91038be72c2@bitbucket.org> --- you can reply above this line --- New issue 192: looponfailing mode not working in 2.2.4 https://bitbucket.org/hpk42/pytest/issue/192/looponfailing-mode-not-working-in-224 Alex Greif: Hi, The documented feature in the 2.2.4 docs does not work. $ py.test -f returns: py.test: error: no such option: -f -- This is an issue notification from bitbucket.org. You are receiving this either because you are the owner of the issue, or you are following the issue. From issues-reply at bitbucket.org Thu Sep 27 14:54:45 2012 From: issues-reply at bitbucket.org (Anonymous) Date: Thu, 27 Sep 2012 12:54:45 -0000 Subject: [py-dev] [hpk42/pytest] os module has no attribute 'fork' on Windows (issue #193) Message-ID: <5ab2090aee1a5cc8d999e9cc51625747@bitbucket.org> --- you can reply above this line --- New issue 193: os module has no attribute 'fork' on Windows https://bitbucket.org/hpk42/pytest/issue/193/os-module-has-no-attribute-fork-on-windows Anonymous: executing py.test --boxed on Windows failed with the error: Traceback (most recent call last): File "c:\python27\lib\site-packages\_pytest\main.py", line 74, in wrap_session doit(config, session) File "c:\python27\lib\site-packages\_pytest\main.py", line 106, in _main config.hook.pytest_runtestloop(session=session) File "c:\python27\lib\site-packages\_pytest\core.py", line 421, in __call__ return self._docall(methods, kwargs) File "c:\python27\lib\site-packages\_pytest\core.py", line 432, in _docall res = mc.execute() File "c:\python27\lib\site-packages\_pytest\core.py", line 350, in execute res = method(**kwargs) File "c:\python27\lib\site-packages\_pytest\main.py", line 119, in pytest_runtestloop item.config.hook.pytest_runtest_protocol(item=item, nextitem=nextitem) File "c:\python27\lib\site-packages\_pytest\core.py", line 421, in __call__ return self._docall(methods, kwargs) File "c:\python27\lib\site-packages\_pytest\core.py", line 432, in _docall res = mc.execute() File "c:\python27\lib\site-packages\_pytest\core.py", line 350, in execute res = method(**kwargs) File "c:\python27\lib\site-packages\pytest_xdist-1.8-py2.7.egg\xdist\plugin.py", line 85, in pytest_runtest_protocol reports = forked_run_report(item) File "c:\python27\lib\site-packages\pytest_xdist-1.8-py2.7.egg\xdist\plugin.py", line 105, in forked_run_report ff = py.process.ForkedFunc(runforked) File "c:\python27\lib\site-packages\py\_process\forkedfunc.py", line 30, in __init__ pid = os.fork() AttributeError: 'module' object has no attribute 'fork' -- This is an issue notification from bitbucket.org. You are receiving this either because you are the owner of the issue, or you are following the issue. From issues-reply at bitbucket.org Sat Sep 29 19:00:31 2012 From: issues-reply at bitbucket.org (Ronny Pfannschmidt) Date: Sat, 29 Sep 2012 17:00:31 -0000 Subject: [py-dev] [hpk42/pytest] node.keywords no longer writable (issue #194) Message-ID: <9a0b98475c523a9bb1e136b0562702db@bitbucket.org> --- you can reply above this line --- New issue 194: node.keywords no longer writable https://bitbucket.org/hpk42/pytest/issue/194/nodekeywords-no-longer-writable Ronny Pfannschmidt: due to vars(self.markers) being a nonwritable dictview now -- This is an issue notification from bitbucket.org. You are receiving this either because you are the owner of the issue, or you are following the issue. From flub at devork.be Sun Sep 30 13:57:28 2012 From: flub at devork.be (Floris Bruynooghe) Date: Sun, 30 Sep 2012 12:57:28 +0100 Subject: [py-dev] performance timing and timer pausing in pytest-timeout Message-ID: Hi all, While pytest-timeout was targeting hanging/deadlocked code there have been various requests which relate to failing tests based on performance. Another issue which came up is the control of timeouts during fixtures (funcarg/resource) setup. So instead of saying that this is not in the scope of pytest-timeout as I have been doing so far here is a proposal of how to address these requirements. I'd be very grateful on any feedback you could provide. Firstly I'd like to introduce a timeout funcarg resource which allows you do to things like this:: @pytest.funcarg() def shared_resource(timeout): with timeout.pause(), timeout.skip(30): return acquire_shared_resource() Some details: timeout.pause: context manager to pause the normal timeout timer if one is active. timeout.skip and timeout.fail: These are context managers which will try to "soft" timeout the code block within them. This means if reasonably possible it will try to interrupt the code block after the timeout, but if not it will just run the code block until it finished. Either way when exiting the context manager either Skip or Fail will be raised. Also no extra information like e.g. thread and stack details will be provided. It's full signature will probably something like timeout.skip(timeout, timer=time.time, msg=None) where the idea is to provide some timers in the pytest namespace like pytest.timer_ms, pytest.timer_pystone. This can be useful in both fixtures as well as test functions. Lastly I'm thinking of providing a way of getting a timer which uses pytest-cache and to measure the duration of a test (in pystones) and will fail the test if it takes longer then the previous pystones value. This would only be reset by clearing the cache. I'm not sure whether to do this as part of the new timeout funcarg resource or as a new marker,so either:: def test_foo(timeout): with timeout.noregress(): performance_critical_func() or:: @pytest.mark.timeout_noregress def test_foo(): performance_coritical_func() Both version could probably take a timer argument too re-using the pytest.timer_ms, pytest.timer_pystones etc timers. I'm slightly in favour of the context manager since it makes the timed section more explicit. But it might possibly only be allowed once per test function (or have a "key" argument). I'm very keen to hear other people's input on these ideas before implementing them. Especially since they solve problems I don't really have personally. Regards, Floris From anton7811 at gmail.com Sun Sep 30 18:16:33 2012 From: anton7811 at gmail.com (Anton P) Date: Sun, 30 Sep 2012 19:16:33 +0300 Subject: [py-dev] using logging in py.test Message-ID: Hi All, I'd like to use standard logging module with py.test. Messages generated from test functions are visible on stdout if -s option is set. But log messages generated from conftest.py or other custom modules used inside test function are not visible on stdout but visible in captured stdout in case -s option isn't set. I want to see all the messages from all modules on stdout, not only from test function. Could anybody point me to the root couse of the issue? Thank you in advance! -Anton From holger at merlinux.eu Sun Sep 30 20:46:13 2012 From: holger at merlinux.eu (holger krekel) Date: Sun, 30 Sep 2012 18:46:13 +0000 Subject: [py-dev] performance timing and timer pausing in pytest-timeout In-Reply-To: References: Message-ID: <20120930184613.GP27773@merlinux.eu> Hi Floris, On Sun, Sep 30, 2012 at 12:57 +0100, Floris Bruynooghe wrote: > Hi all, > > While pytest-timeout was targeting hanging/deadlocked code there have > been various requests which relate to failing tests based on > performance. Another issue which came up is the control of timeouts > during fixtures (funcarg/resource) setup. So instead of saying that > this is not in the scope of pytest-timeout as I have been doing so far > here is a proposal of how to address these requirements. I'd be very > grateful on any feedback you could provide. > > > Firstly I'd like to introduce a timeout funcarg resource which allows > you do to things like this:: > > @pytest.funcarg() > def shared_resource(timeout): > with timeout.pause(), timeout.skip(30): > return acquire_shared_resource() > > Some details: > > timeout.pause: context manager to pause the normal timeout timer if > one is active. > > timeout.skip and timeout.fail: These are context managers which will > try to "soft" timeout the code block within them. This means if > reasonably possible it will try to interrupt the code block after the > timeout, but if not it will just run the code block until it finished. > Either way when exiting the context manager either Skip or Fail will > be raised. Should skip be raised because the system/network whatever is deemed to slow to handle the tests? Usually skip (as oposed to xfail) are there for a dependency or environment miss/mismatch so that the test cannot be run at all. In any case, I also don't see why timeout neeeds to be a funcarg. Could it not work like this:: @pytest.funcarg() def shared_resource(): with pytest.timeout(30, ontimeout=lambda: pytest.fail("too slow")): ... and such a timeout helper could also be used in test functions. Of course it doesn't even need to be pytest specific timeout helper - it could already exist in some little library? > Also no extra information like e.g. thread and stack > details will be provided. It's full signature will probably something > like timeout.skip(timeout, timer=time.time, msg=None) where the idea > is to provide some timers in the pytest namespace like > pytest.timer_ms, pytest.timer_pystone. This can be useful in both > fixtures as well as test functions. For me, passing in a string like "10mp" (10 megapystones) would be fine enough. I did something like for timeout-specs in the PyPy test suite several centuries ago. > Lastly I'm thinking of providing a way of getting a timer which uses > pytest-cache and to measure the duration of a test (in pystones) and > will fail the test if it takes longer then the previous pystones > value. This would only be reset by clearing the cache. I'm not sure > whether to do this as part of the new timeout funcarg resource or as a > new marker,so either:: > > def test_foo(timeout): > with timeout.noregress(): > performance_critical_func() > > or:: > > @pytest.mark.timeout_noregress > def test_foo(): > performance_coritical_func() The "timeout" name here makes not much sense to me here. I wonder if the performance related thgoughts (which i think are very interesting) shouldn't better go to a "pytest-speed" or perf plugin with according markers/funcarg names. The above two ideas otherwise sound both potentially useful to me. In addition, running and recording timings for all test functions of the last release and then comparing against my current trunk would be useful - something like a report on "top 10 slowed down test funcs, top 10 sped up test funcs" or so. I think anyway, that the real work with such a plugin/approach lies in the reporting part. As an initial use i'd rather be interested in this last top10 bit as a general starting point (you have to do almost nothing other than re-running your code with different checkouts and get a nice report). Then i probably discover i want to do something more fine-grained like the above ... > Both version could probably take a timer argument too re-using the > pytest.timer_ms, pytest.timer_pystones etc timers. > > I'm slightly in favour of the context manager since it makes the timed > section more explicit. But it might possibly only be allowed once per > test function (or have a "key" argument). > > I'm very keen to hear other people's input on these ideas before > implementing them. Especially since they solve problems I don't > really have personally. Heh, maybe you should try to identify an area where you do care about speed regressions :) And/or invite the people who wanted to have features from pytest-timeout to comment and/or detail their thoughts on what they actually want to have :) best, holger P.S.: I think that many pytest users have subscribed rather to Testing-in-Python rather than the py-dev mailing list because TIP is mentioned first on the web page for a year now. So CCing TIP is maybe not a bad idea, especially if the topic (performance measuring etc.) is of general interest. From holger at merlinux.eu Sun Sep 30 20:56:43 2012 From: holger at merlinux.eu (holger krekel) Date: Sun, 30 Sep 2012 18:56:43 +0000 Subject: [py-dev] using logging in py.test In-Reply-To: References: Message-ID: <20120930185643.GQ27773@merlinux.eu> Hi Anton, On Sun, Sep 30, 2012 at 19:16 +0300, Anton P wrote: > Hi All, > > I'd like to use standard logging module with py.test. Messages > generated from test functions are visible on stdout if -s option is > set. But log messages generated from conftest.py or other custom > modules used inside test function are not visible on stdout but > visible in captured stdout in case -s option isn't set. > > I want to see all the messages from all modules on stdout, not only > from test function. An initial conftest.py file is loaded very early and pytest uses unconditional capturing. This goes back to a request from the PyPy folks who start GCC at this stage and otherwise see lots of stuff. If this proves to be a problem we have to see how to make it conditional. Problem is that the conftest.py often add command line options and thus need to to be loaded before opts are parsed (and thus "-s"). If logging is active during conftest.py import-time it will see the captured stream and probably record it and use it from there on. As the captured stream is not analyzed/used further in the case of "-s" it is kind of lost, loosing the logging messages. Potential solutions: - try to avoid doing calls/imports of logging during conftest.py import-time. - try out pytest-logging plugin to see if it helps - program a PYTEST_NOCAPTURE environement setting to pytest to allow switching off capturing totally best, holger That > Thank you in advance! > -Anton > _______________________________________________ > py-dev mailing list > py-dev at codespeak.net > http://codespeak.net/mailman/listinfo/py-dev >