Mailman 3 March 2015 - pytest-dev

release checklist / de-monopolizing release process
by holger krekel April 2, 2015

April 2, 2015

Hi committers and friends of pytest :) to de-monopolize the knowledge of releasing pytest i just created a PR with a first stab at a release checklist: https://bitbucket.org/pytest-dev/pytest/pull-request/266/add-a-release-chec… It doesn't explain how to use ``devpi``, maybe this tutorial helps: http://doc.devpi.net/latest/quickstart-releaseprocess.html Ideally more of the release process would be automated, am open to PRs and scripts doing it. Also, i'd like to add some more people's SSH key to the "pytest-dev" account on ``pytest.org``. Brianna, Ronny and me can currently "make install" the docs there. Floris, Anatoly, Andreas, Bruno: please send me your public ssh-key and your PYPI handle so you are technically able to do a release. Anyone else who wants to and is in the current pytest-dev committer group is invited as well :) best, holger -- about me: http://holgerkrekel.net/about-me/ contracting: http://merlinux.eu

3 7

moving pytest-timeout to pytest-dev
by Floris Bruynooghe April 1, 2015

April 1, 2015

Hi all, If no objections I'd like to move pytest-timeout to the pytest-dev team on bitbucket as well. Regards, Floris

3 2

Move pytest-faulthandler to pytest-dev@github
by Bruno Oliveira March 31, 2015

March 31, 2015

Hi all, I would to like to know if there are any objections to moving the pytest-faulthandler (https://github.com/nicoddemus/pytest-faulthandler) plugin to the pytest-dev@github organization? Cheers,

3 3

pytest, or pytest plug-in dogfooding
by Tibor Arpas March 31, 2015

March 31, 2015

Hi, is it difficult to set-up development environment for hacking on pytest itself or a plug-in where you use a stable version as a test runner but the code under test is a different one? Or in other words 'import pytest_myplugin' has to pick up the stable source code on pytest start-up, but all the tests (and their subprocesses) doing 'import pytest_myplugin' have to pick-up the bleading edge-version. Thanks! Tibor

2 1

Re: [pytest-dev] pytest-2.7.0 release package
by Peter March 30, 2015

March 30, 2015

Outside of issue 701 that I reported just prior to the 2.7.0 release, I found no regressions in my usage of pytest (which I believe differs from the norm - see below). I use python with pytest to check for observational equivalence between actual and expected results of a business critical C#.NET application. My conftest.py script customizes py.test with fixtures that automate the running of selected tests and their subsequent analysis. This includes the driving and coordination of CLI and GUI .NET applications that were not designed to support automated testing. This automated test facility also includes a package to extend pytest's functionality. For example: - some fields (e.g. processing date) change with each run of the .NET application, which necessitates that certain fields in the actual and previously generated expected results are masked prior to using the built-in pytest assertions; I created a package to declaratively specify what fields are masked for various versioned record structures - the actual masking happens prior to using pytest assertions - the record structures for this .NET application are required via legacy downstream systems and are at least 5000 bytes long; if there are only one or two differences between the actual and expected results and the differences are close together, the built-in pytest assertions produce a nice report; if there are multiple differences and they are far from each other, the built-in pytest assertion report isn't very helpful; I added functionality to produce a list of the indices where values differ, which greatly simplifies troubleshooting The automated test facility that I built also includes a python application to automatically create automated test repository content for each test (i.e. test data, expected results, and the required py.test scripts). The benefits of using pytest in this manner are as follows: 1) Creating good unit tests requires additional software development work - including testing the unit tests themselves. That's fine, if you have sufficiently competent developers and the necessary time to do so. The automated test facility that I built doesn't require any additional programming (beyond what I've already done); therefore, even business users and less experienced developers can create test data for use by the automated test system. 2) Supports rapid regression testing while refactoring the existing .NET code base. 3) Manual testing and analysis that often took days (sometimes weeks) can now be done in minutes. 4) Troubleshooting regression failures has been greatly simplified. Thank you for producing a truly great piece of software! Peter Santoro

3 2

Re: [pytest-dev] re-running pytest on file changes
by Tom Viner March 28, 2015

March 28, 2015

I find it really helps TDD to have my tests run automatically whenever I change either a test or the code itself. I set my editor (Sublime Text 3) to auto save upon defocus, so it's actually impossible to avoid a test rerun whenever I alter a file and move on or hit save. For this, I regularly use Jonathan Hartley's rerun Python library[0] which gives the ability to wrap my py.test or tox command like so: rerun --ignore sqlite.db "py.test tests/" The --ignore part is very important, as different test suites end up altering various working directory files, and if you can't select files/folders (file patterns would be even better) to ignore, you end up in an un-pausing loop of test reruns. I hadn't heard of looponchange can it ignore certain files/folders? [0] https://pypi.python.org/pypi/rerun > ---------------------------------------------------------------------- > > Message: 1 > Date: Fri, 27 Mar 2015 14:55:31 +0000 > From: Tibor Arpas <tibor.arpas(a)infinit.sk> > To: "pytest-dev(a)python.org" <pytest-dev(a)python.org> > Subject: [pytest-dev] re-running pytest on file changes > Message-ID: > < > CADi_WY3cbDAOfDS8+oV5iRoF6qzvjsX+_PSmcBK-Ss1uObGCxA(a)mail.gmail.com> > Content-Type: text/plain; charset="utf-8" > > Hi, > > I'm trying to wrap my head around this subject. > > Is there a strong reason why the -f looponfail option is inside a plug-in? > My hunch would be to do it as a separate, flexible, command, but that's > just feelings without any experience. > > Poll: Do you guys use looponfail? Has it been working good? Does it play > nice with other tasks you might want to do on file changes? What other > tools do you use for repeated tasks on source code changes? > > What's the situation at the moment with --looponfail and --looponchange? Is > it going to be inside pytest_cache which will go into core? Or it will be a > different plug-in? > > I've seen > https://bitbucket.org/pytest-dev/pytest/commits/b5727cc6632c/ > > and the "take in cache" PR. > > > Tibor > P.S. some more thoughts about the same functionality are here: > https://github.com/tarpas/testmon/issues/4 > and a script whith callbacks for every success and failure: > https://github.com/blueyed/testmon/blob/tmon-improvements/testmon/tmon.py >

1 0

re-running pytest on file changes
by Tibor Arpas March 27, 2015

March 27, 2015

Hi, I'm trying to wrap my head around this subject. Is there a strong reason why the -f looponfail option is inside a plug-in? My hunch would be to do it as a separate, flexible, command, but that's just feelings without any experience. Poll: Do you guys use looponfail? Has it been working good? Does it play nice with other tasks you might want to do on file changes? What other tools do you use for repeated tasks on source code changes? What's the situation at the moment with --looponfail and --looponchange? Is it going to be inside pytest_cache which will go into core? Or it will be a different plug-in? I've seen https://bitbucket.org/pytest-dev/pytest/commits/b5727cc6632c/ and the "take in cache" PR. Tibor P.S. some more thoughts about the same functionality are here: https://github.com/tarpas/testmon/issues/4 and a script whith callbacks for every success and failure: https://github.com/blueyed/testmon/blob/tmon-improvements/testmon/tmon.py

1 0

pytest-2.7.0 release package / please test
by holger krekel March 26, 2015

March 26, 2015

Hi everybody, You can install either form trunk or with: pip install https://devpi.net/hpk/dev/+f/14c/de91b1106c937/pytest-2.7.0.tar.gz Please install it and work with it today and let me know any issues, regressions. I am going to release it verbatim this afternoon otherwise. thanks, holger -- about me: http://holgerkrekel.net/about-me/ contracting: http://merlinux.eu

7 6

pytest-2.7.0: fixes, features and more speed
by holger krekel March 26, 2015

March 26, 2015

pytest-2.7.0: fixes, features, speed improvements =========================================================================== pytest is a mature Python testing tool with more than a 1100 tests against itself, passing on many different interpreters and platforms. This release is supposed to be drop-in compatible to 2.6.X. See below for the changes and see docs at: http://pytest.org As usual, you can upgrade from pypi via:: pip install -U pytest Thanks to all who contributed, among them: Anatoly Bubenkoff Floris Bruynooghe Brianna Laugher Eric Siegerman Daniel Hahler Ronny Pfannschmidt Charles Cloud Tom Viner Holger Peters Ldiary Translations almarklein have fun, holger krekel 2.7.0 (compared to 2.6.4) ----------------------------- - fix issue435: make reload() work when assert rewriting is active. Thanks Daniel Hahler. - fix issue616: conftest.py files and their contained fixutres are now properly considered for visibility, independently from the exact current working directory and test arguments that are used. Many thanks to Eric Siegerman and his PR235 which contains systematic tests for conftest visibility and now passes. This change also introduces the concept of a ``rootdir`` which is printed as a new pytest header and documented in the pytest customize web page. - change reporting of "diverted" tests, i.e. tests that are collected in one file but actually come from another (e.g. when tests in a test class come from a base class in a different file). We now show the nodeid and indicate via a postfix the other file. - add ability to set command line options by environment variable PYTEST_ADDOPTS. - added documentation on the new pytest-dev teams on bitbucket and github. See https://pytest.org/latest/contributing.html . Thanks to Anatoly for pushing and initial work on this. - fix issue650: new option ``--docttest-ignore-import-errors`` which will turn import errors in doctests into skips. Thanks Charles Cloud for the complete PR. - fix issue655: work around different ways that cause python2/3 to leak sys.exc_info into fixtures/tests causing failures in 3rd party code - fix issue615: assertion re-writing did not correctly escape % signs when formatting boolean operations, which tripped over mixing booleans with modulo operators. Thanks to Tom Viner for the report, triaging and fix. - implement issue351: add ability to specify parametrize ids as a callable to generate custom test ids. Thanks Brianna Laugher for the idea and implementation. - introduce and document new hookwrapper mechanism useful for plugins which want to wrap the execution of certain hooks for their purposes. This supersedes the undocumented ``__multicall__`` protocol which pytest itself and some external plugins use. Note that pytest-2.8 is scheduled to drop supporting the old ``__multicall__`` and only support the hookwrapper protocol. - majorly speed up invocation of plugin hooks - use hookwrapper mechanism in builtin pytest plugins. - add a doctest ini option for doctest flags, thanks Holger Peters. - add note to docs that if you want to mark a parameter and the parameter is a callable, you also need to pass in a reason to disambiguate it from the "decorator" case. Thanks Tom Viner. - "python_classes" and "python_functions" options now support glob-patterns for test discovery, as discussed in issue600. Thanks Ldiary Translations. - allow to override parametrized fixtures with non-parametrized ones and vice versa (bubenkoff). - fix issue463: raise specific error for 'parameterize' misspelling (pfctdayelise). - On failure, the ``sys.last_value``, ``sys.last_type`` and ``sys.last_traceback`` are set, so that a user can inspect the error via postmortem debugging (almarklein). -- about me: http://holgerkrekel.net/about-me/ contracting: http://merlinux.eu

1 0

Website sidebar links are out of date
by Brianna Laugher March 25, 2015

March 25, 2015

Hi, I noticed that the issue tracker and source links on the website are linking to hpk42 rather than pytest-dev. It's worse for the issue tracker as it just has a 404 and no link or information about the moved repo. I see the links were corrected already here https://bitbucket.org/pytest-dev/pytest/pull-request/261/update-website-sid… - is it just waiting for a docs push or did something else happen? Brianna -- They've just been waiting in a mountain for the right moment: http://modernthings.org/

3 2