Mailman 3 May 2015 - pytest-dev

Working with other dependency injection systems
by Kai Groner Sept. 10, 2015

Sept. 10, 2015

Hi testing people! I'm trying to figure out how I can test a code base that uses an existing dependency injection system. I've run into two problems, and I have solutions for each of them but I think maybe there is a better way, so I'm looking for some advice. The first problem is how do I override how a test using this injector is called? I think I want to use the pytest_runtest_call hook, but it still tries to invoke the test without the injector. My current solution is to use the experimental hookwrapper mechanism to replace item.obj with a partially bound version, then restore it afterward. The second problem I'm having is the py.test fixture system is trying to resolve arguments that are provided by this other injector. How can I tell it that some of these don't need to be provided? My current solution is to blind py.test with a wrapper function with a *a, **kw signature. I use functools.wraps, so annotations are introspectable for the other injector, and delete the __wrapped__ attribute to prevent py.test from introspecting it. Is there a nicer, and perhaps less blunt, way to influence the funcarg fixture behaviors? I've tried a couple things with the pytest_collection_modifyitems hook, but haven't gotten anything that works yet. Other details to know about this injection system: - we want to build and teardown the injector with each test - we may want to configure the injector differently for some tests (possibly with fixture data from py.test) Thanks, Kai

2 6

Fixture override not working
by Ken Petti June 3, 2015

June 3, 2015

Hello all, Last week I spoke to Ronny on the IRC channel about a fixture issue, and he recommended I send out an email to the larger group. Here is a gist of the issuehttps://gist.github.com/citizenken/861055b8778c9cce343a I have a fixture on the module level that I want to override on a per-test basis. This module fixture handles app log in and is both autoused and function scoped. I override it with a tuple value, to use non-default credentials, and then use the new fixture in an alternate login fixture. What I expect to happen is that the module fixture it overridden with my static values, the alternate login fixture then uses those static values and logs in, then control goes to my tests, which executes as expected. What is happening is that the module fixture is never overridden, so the app logs in with the wrong credentials and the test fails. Ronny suggested modifying my gist and using hidden fixtures like so: https://gist.github.com/RonnyPfannschmidt/3c17b07aa56d6231d38d This did not work. Instead, the hidden fixture was executed with the static values, then, without closing the browser, the original login_by_account fixture is called. This is an excerpt from my chat with Ronny: [13:25] <ronny> citken: i think you missunderstood me, the idea is to separate th data fixture and the autouse fixture, since it seems all autouse instaces are made n all cases [13:27] <citken> ronny: can you show me an example/comment on the gist what i should do? [13:28] <ronny> citken: https://gist.github.com/RonnyPfannschmidt/3c17b07aa56d6231d38d [13:30] <citken> ok let me try that now [13:30] <citken> thanks, that makes more sense [13:31] <ronny> citken: could you please repost an issue about autouse fixture overriding issues [13:34] <citken> yeah that isn't working either [13:34] <citken> my fixture is overriding for all test cases [13:35] <citken> it should only override for the testcase that is marked parameterize [13:35] <citken> making the hidden fixture, as in your gist, does not work correctly [13:43] <ronny> then im missing something [13:44] <ronny> citken: what is the problem with the hidden fixture/ [14:01] <citken> ronny: so i don't think your gist is handling the override correctly [14:02] <ronny> citken: im probably missing something, what exactly is happening how? [14:04] <citken> its weird... it looks like _ensure_login is hit first (expected, with autouse=True), and that has the value of my override. My app logs in with the override credentials. Then, without closing the browser or anything, the original login_by_account fixture is called, the one with the scope of "function" [14:04] <citken> is the stack trace like this?: [14:05] <citken> override -> autouse fixture -> function scope fixture -> test? [14:07] <ronny> citken: not sure, i dont know the exact details, holger and bubenkoff worked with fixture oerrides in-depth, and currently im at the end of what i can help with If anyone could give me a hand that would be great! Thanks! -- *Ken Petti* Software Development Engineer in Test E-Mail: ken.petti(a)cloudlock.com <uche(a)cloudlock.com> <http://www.cloudlock.com/> <http://www.twitter.com/cloudlock> <http://www.facebook.com/cloudlock> <http://www.linkedin.com/company/cloudlock> <https://plus.google.com/106755360058827077409/posts> *CloudLock provides compliance and security solutions for enterprises using public cloud applications like Salesforce and Google Apps. The largest organizations in the world trust CloudLock to secure their data*.

2 1

Test support for Python micro versions?
by Dinu Gherman June 3, 2015

June 3, 2015

Hello! I'm looking for information about how to run py.test (or rather tox) on micro versions of Python releases (the micro version being part C in the full versioning scheme A.B.C, i.e. major.minor.micro, see also https://docs.python.org/devguide/devcycle.html). I've put a well-known search engine at work and have searched on pytest.org, but haven't been really successful. Is this a non-feature or just considered a non-issue? The latter would be surprising to me as Python micro releases have recently shown what I'd call significant changes, e.g. those concerning SSL between 2.7.8 and 2.7.9. So AFAICT it would be helpful to be able to run something like this: tox -e py278 py279 py2710 Any clues are highly appreciated, thanks! Cheers, Dinu

2 1

tox-2.0.0: plugin support, platforms, env isolation
by holger krekel May 22, 2015

May 22, 2015

tox-2.0: plugins, platform, env isolation ========================================== tox-2.0 was released to pypi, a major new release with *mostly* backward-compatible enhancements and fixes: - experimental support for plugins, see https://testrun.org/tox/dev/plugins.html which includes also a refined internal registration mechanism for new testenv ini options. You can now ask tox which testenv ini parameters exist with ``tox --help-ini``. - ENV isolation: only pass through very few environment variables from the tox invocation to the test environments. This may break test runs that previously worked with tox-1.9 -- you need to either use the ``setenv`` or ``passenv`` ini variables to set appropriate environment variables. - PLATFORM support: you can set ``platform=REGEX`` in your testenv sections which lets tox skip the environment if the REGEX does not match ``sys.platform``. - tox now stops execution of test commands if the first of them fails unless you set ``ignore_errors=True``. Thanks to Volodymyr Vitvitski, Daniel Hahler, Marc Abramowitz, Anthon van der Neuth and others for contributions. More documentation about tox in general: http://tox.testrun.org/ Installation: pip install -U tox code hosting and issue tracking on bitbucket: https://bitbucket.org/hpk42/tox What is tox? ---------------- tox standardizes and automates tedious test activities driven from a simple ``tox.ini`` file, including: * creation and management of different virtualenv environments with different Python interpreters * packaging and installing your package into each of them * running your test tool of choice, be it nose, py.test or unittest2 or other tools such as "sphinx" doc checks * testing dev packages against each other without needing to upload to PyPI best, Holger Krekel, merlinux GmbH -- about me: http://holgerkrekel.net/about-me/ contracting: http://merlinux.eu 2.0.0 ----------- - (new) introduce environment variable isolation: tox now only passes the PATH and PIP_INDEX_URL variable from the tox invocation environment to the test environment and on Windows also ``SYSTEMROOT``, ``PATHEXT``, ``TEMP`` and ``TMP`` whereas on unix additionally ``TMPDIR`` is passed. If you need to pass through further environment variables you can use the new ``passenv`` setting, a space-separated list of environment variable names. Each name can make use of fnmatch-style glob patterns. All environment variables which exist in the tox-invocation environment will be copied to the test environment. - a new ``--help-ini`` option shows all possible testenv settings and their defaults. - (new) introduce a way to specify on which platform a testenvironment is to execute: the new per-venv "platform" setting allows to specify a regular expression which is matched against sys.platform. If platform is set and doesn't match the platform spec in the test environment the test environment is ignored, no setup or tests are attempted. - (new) add per-venv "ignore_errors" setting, which defaults to False. If ``True``, a non-zero exit code from one command will be ignored and further commands will be executed (which was the default behavior in tox < 2.0). If ``False`` (the default), then a non-zero exit code from one command will abort execution of commands for that environment. - show and store in json the version dependency information for each venv - remove the long-deprecated "distribute" option as it has no effect these days. - fix issue233: avoid hanging with tox-setuptools integration example. Thanks simonb. - fix issue120: allow substitution for the commands section. Thanks Volodymyr Vitvitski. - fix issue235: fix AttributeError with --installpkg. Thanks Volodymyr Vitvitski. - tox has now somewhat pep8 clean code, thanks to Volodymyr Vitvitski. - fix issue240: allow to specify empty argument list without it being rewritten to ".". Thanks Daniel Hahler. - introduce experimental (not much documented yet) plugin system based on pytest's externalized "pluggy" system. See tox/hookspecs.py for the current hooks. - introduce parser.add_testenv_attribute() to register an ini-variable for testenv sections. Can be used from plugins through the tox_add_option hook. - rename internal files -- tox offers no external API except for the experimental plugin hooks, use tox internals at your own risk. - DEPRECATE distshare in documentation

4 6

pytest-2.7.1: bugfix release, wheel support, yield fixtures
by Floris Bruynooghe May 19, 2015

May 19, 2015

pytest-2.7.1: bugfix release, wheel support and yield fixtures ============================================================== pytest is a mature Python testing tool with more than a 1100 tests against itself, passing on many different interpreters and platforms. This release is supposed to be drop-in compatible to 2.7.0. The most user-visible changes are that wheels are now available on pypi and yield fixtures are no longer marked as experimental, they are here to stay. See below for the changes and see docs at: http://pytest.org As usual, you can upgrade from pypi via:: pip install -U pytest Thanks to all who contributed to this release, among them: Bruno Oliveira Holger Krekel Ionel Maries Cristian Floris Bruynooghe Happy testing, The py.test Development Team 2.7.1 (compared to 2.7.0) ------------------------- - fix issue731: do not get confused by the braces which may be present and unbalanced in an object's repr while collapsing False explanations. Thanks Carl Meyer for the report and test case. - fix issue553: properly handling inspect.getsourcelines failures in FixtureLookupError which would lead to to an internal error, obfuscating the original problem. Thanks talljosh for initial diagnose/patch and Bruno Oliveira for final patch. - fix issue660: properly report scope-mismatch-access errors independently from ordering of fixture arguments. Also avoid the pytest internal traceback which does not provide information to the user. Thanks Holger Krekel. - streamlined and documented release process. Also all versions (in setup.py and documentation generation) are now read from _pytest/__init__.py. Thanks Holger Krekel. - fixed docs to remove the notion that yield-fixtures are experimental. They are here to stay :) Thanks Bruno Oliveira. - Support building wheels by using environment markers for the requirements. Thanks Ionel Maries Cristian. - fixed regression to 2.6.4 which surfaced e.g. in lost stdout capture printing when tests raised SystemExit. Thanks Holger Krekel. - reintroduced _pytest fixture of the pytester plugin which is used at least by pytest-xdist.

1 0

Re: [pytest-dev] [TIP] tox-2.0.0: plugin support, platforms, env isolation
by holger krekel May 12, 2015

May 12, 2015

Hi Alex, On Tue, May 12, 2015 at 18:51 -0400, Alex Gaynor wrote: > Hi Holger, > > First: congrats on shipping 2.0, I'm very excited to explore some of the > new features. > > Second: > > Did tox drop Python 2.6 support? It started failing on our CI, so far the > error I see is that tox imports argparse, but I guess it's setup.py does > not depend on it. Can this be fixed? Hum, that's probably a problem of the wheel format. Can you try installing with "--no-use-wheel" and see if that helps? holger > Thanks! > Alex > > On Tue, May 12, 2015 at 5:46 PM, holger krekel <holger(a)merlinux.eu> wrote: > > > tox-2.0: plugins, platform, env isolation > > ========================================== > > > > tox-2.0 was released to pypi, a major new release with *mostly* > > backward-compatible enhancements and fixes: > > > > - experimental support for plugins, see > > https://testrun.org/tox/dev/plugins.html > > which includes also a refined internal registration mechanism for new > > testenv > > ini options. You can now ask tox which testenv ini parameters exist > > with ``tox --help-ini``. > > > > - ENV isolation: only pass through very few environment variables from the > > tox invocation to the test environments. This may break test runs that > > previously worked with tox-1.9 -- you need to either use the > > ``setenv`` or ``passenv`` ini variables to set appropriate environment > > variables. > > > > - PLATFORM support: you can set ``platform=REGEX`` in your testenv sections > > which lets tox skip the environment if the REGEX does not match > > ``sys.platform``. > > > > - tox now stops execution of test commands if the first of them fails > > unless > > you set ``ignore_errors=True``. > > > > Thanks to Volodymyr Vitvitski, Daniel Hahler, Marc Abramowitz, Anthon van > > der Neuth and others for contributions. > > > > More documentation about tox in general: > > > > http://tox.testrun.org/ > > > > Installation: > > > > pip install -U tox > > > > code hosting and issue tracking on bitbucket: > > > > https://bitbucket.org/hpk42/tox > > > > What is tox? > > ---------------- > > > > tox standardizes and automates tedious test activities driven from a > > simple ``tox.ini`` file, including: > > > > * creation and management of different virtualenv environments > > with different Python interpreters > > * packaging and installing your package into each of them > > * running your test tool of choice, be it nose, py.test or unittest2 or > > other tools such as "sphinx" doc checks > > * testing dev packages against each other without needing to upload to PyPI > > > > best, > > Holger Krekel, merlinux GmbH > > > > -- > > about me: http://holgerkrekel.net/about-me/ > > contracting: http://merlinux.eu > > > > 2.0.0 > > ----------- > > > > - (new) introduce environment variable isolation: > > tox now only passes the PATH and PIP_INDEX_URL variable from the tox > > invocation environment to the test environment and on Windows > > also ``SYSTEMROOT``, ``PATHEXT``, ``TEMP`` and ``TMP`` whereas > > on unix additionally ``TMPDIR`` is passed. If you need to pass > > through further environment variables you can use the new ``passenv`` > > setting, > > a space-separated list of environment variable names. Each name > > can make use of fnmatch-style glob patterns. All environment > > variables which exist in the tox-invocation environment will be copied > > to the test environment. > > > > - a new ``--help-ini`` option shows all possible testenv settings and > > their defaults. > > > > - (new) introduce a way to specify on which platform a testenvironment is > > to > > execute: the new per-venv "platform" setting allows to specify > > a regular expression which is matched against sys.platform. > > If platform is set and doesn't match the platform spec in the test > > environment the test environment is ignored, no setup or tests are > > attempted. > > > > - (new) add per-venv "ignore_errors" setting, which defaults to False. > > If ``True``, a non-zero exit code from one command will be ignored and > > further commands will be executed (which was the default behavior in > > tox < > > 2.0). If ``False`` (the default), then a non-zero exit code from one > > command > > will abort execution of commands for that environment. > > > > - show and store in json the version dependency information for each venv > > > > - remove the long-deprecated "distribute" option as it has no effect these > > days. > > > > - fix issue233: avoid hanging with tox-setuptools integration example. > > Thanks simonb. > > > > - fix issue120: allow substitution for the commands section. Thanks > > Volodymyr Vitvitski. > > > > - fix issue235: fix AttributeError with --installpkg. Thanks > > Volodymyr Vitvitski. > > > > - tox has now somewhat pep8 clean code, thanks to Volodymyr Vitvitski. > > > > - fix issue240: allow to specify empty argument list without it being > > rewritten to ".". Thanks Daniel Hahler. > > > > - introduce experimental (not much documented yet) plugin system > > based on pytest's externalized "pluggy" system. > > See tox/hookspecs.py for the current hooks. > > > > - introduce parser.add_testenv_attribute() to register an ini-variable > > for testenv sections. Can be used from plugins through the > > tox_add_option hook. > > > > - rename internal files -- tox offers no external API except for the > > experimental plugin hooks, use tox internals at your own risk. > > > > - DEPRECATE distshare in documentation > > > > > > _______________________________________________ > > testing-in-python mailing list > > testing-in-python(a)lists.idyll.org > > http://lists.idyll.org/listinfo/testing-in-python > > > > > > -- > "I disapprove of what you say, but I will defend to the death your right to > say it." -- Evelyn Beatrice Hall (summarizing Voltaire) > "The people's good is the highest law." -- Cicero > GPG Key fingerprint: 125F 5C67 DFE9 4084 -- about me: http://holgerkrekel.net/about-me/ contracting: http://merlinux.eu

1 0

[ANN] plugin pytest-incremental 0.4 released
by Eduardo Schettino May 8, 2015

May 8, 2015

Hi, pytest-incremental is a plugin that aims to give faster feedback for test results. It analyses the source code (dependency) structure and keeps track of file modifications to: - re-order tests into a more relevant order than alphabetical order - de-select (skip) tests when changes since last execution dont affect the test This is the first release that features test re-ordering and adds python3 support. Though it is still in "alpha" stage (works for me), please let me know if you find any issues. pypi: https://pypi.python.org/pypi/pytest-incremental docs: http://pytest-incremental.readthedocs.org source: https://github.com/pytest-dev/pytest-incremental Regards, Eduardo

1 0

Re: [pytest-dev] adopt pytest month is over!
by Aryé H May 7, 2015

May 7, 2015

Hi, Our question is on the right approach to testing such a framework ? 1) start with low level unit tests and strive to mock out the DB 2) continue with higher level integration tests with a DB and data fixtures More specifically (1) What is the best way to write unit tests that need model definitions (without database) and (2) integration tests that need database access in a framework that relies on Pyramid+SQLA but doesn't have any models defined in the code. Thanks for the follow up! Any advice would be greatly appreciated. > Message: 1 > Date: Wed, 6 May 2015 11:38:34 +0000 > From: holger krekel <holger(a)merlinux.eu> > To: Brianna Laugher <brianna.laugher(a)gmail.com> > Cc: "pytest-dev(a)python.org" <pytest-dev(a)python.org>, Chris Hart > <chris(a)brandicted.com> > Subject: Re: [pytest-dev] adopt pytest month is over! > Message-ID: <20150506113834.GP25957(a)merlinux.eu> > Content-Type: text/plain; charset=us-ascii > > Hi Brianna! > > thanks for this great initiative! I am also looking forward to the > survey. > > Note sure i can help myself but are there some notes/summary of what is > the problem with Nefertari currently? > > best, > holger > >

2 1

adopt pytest month is over!
by Brianna Laugher May 6, 2015

May 6, 2015

Hi all, So adopt pytest month is officially over! I am working on surveys for the helpers and projects and will send them out this week. I hope everyone enjoyed it and maybe even learned something. One thing, the helpers for Nefertari (REST API framework built on Pyramid and ElasticSearch) are a bit stuck and could do with some more experienced eyes, specifically about how to approach testing a framework. Although the month is over Nefertari don't have much to show for it yet and it would be nice to see if we can help them get going with the basics. Would anyone be willing to help there? cheers, Brianna -- They've just been waiting in a mountain for the right moment: http://modernthings.org/

2 1