Mailman 3 March 2018 - pytest-dev

Pytest hoodie
by Oleg S March 28, 2018

March 28, 2018

2 1

pytest-flask maintainer unreachable
by Bruno Oliveira March 27, 2018

March 27, 2018

Hi everyone, People have expressed concern over pytest-flask because there has not been a release in over 2 years (see https://github.com/pytest-dev/pytest-flask/issues/72) and Vital Kudzelka (the maintainer) has not been active in the repository for some time, with PRs waiting to be merged. Does anybody have other means to contact Vital Kudzelka other than GitHub? He doesn't seem to be commenting/responding to issues and PRs for awhile (the last comment I can see is from Nov 6, 2016: https://github.com/pytest-dev/pytest-flask/issues/52#issuecomment-258704636, but it might be that I'm just not good at using GH's issue search). I've created an issue on the repository to track this discussion: https://github.com/pytest-dev/pytest-flask/issues/76 Any suggestions from the core devs of other actions that can/should be taken in situations like this, according to our pytest-dev policies? Cheers, Bruno.

2 3

Re: [pytest-dev] Custom reporting for asserts without comparison operators?
by Shawn Brown March 26, 2018

March 26, 2018

Ah. It's good to see that this has been thought about before. My motivation for asking this question was to perform my due diligence and make sure I wasn't missing something before moving ahead. My immediate need is handled by using assert_myfunc() to raise its own error internally--same as Floris' example. Though, it's not ideal. I know my examples have been vague as I've stripped the specifics of my project to focus the question specifically on pytest's behavior and I greatly appreciate everyone who is giving some thought to this. As Ronny mentioned, I'm sure it's possible to address this without user-facing AST manipulation. But I'm not familiar enough with the code base to see where I can best hack on the representations. However, I do have a working AST-based demonstration (below). This uses a fragile monkey-patch that is just asking for trouble so please take this for the experimental hack it is... FILE "conftest.py": import ast import _pytest def my_ast_prerewrite_hook(ast_assert): """Modifies AST of certain asserts before pytest-rewriting.""" # Demo AST-tree manipulation (actual implemenation # would need to be more careful than this). if (isinstance(ast_assert.test, ast.Call) and isinstance(ast_assert.test.func, ast.Name) and ast_assert.test.func.id == 'myfunc'): ast_assert.test.func = ast.Name('assert_myfunc', ast.Load()) return ast_assert # UNDESIRABLE MONKEY PATCHING!!! class ModifiedRewriter(_pytest.assertion.rewrite.AssertionRewriter): def visit_Assert(self, assert_): assert_ = my_ast_prerewrite_hook(assert_) # <- PRE-REWRITE HOOK return super(ModifiedRewriter, self).visit_Assert(assert_) def rewrite_asserts(mod, module_path=None, config=None): ModifiedRewriter(module_path, config).run(mod) _pytest.assertion.rewrite.rewrite_asserts = rewrite_asserts FILE "test_ast_hook_approach.py": import pytest # Test helpers. def myfunc(x): return x == 42 def assert_myfunc(x): __tracebackhide__ = True if not myfunc(x): msg = 'custom report\nmulti-line output\nmyfunc({0}) failed' raise AssertionError(msg.format(x)) return True # Test cases. def test_1passing(): assert myfunc(42) def test_2passing(): assert myfunc(41) is False def test_3passing(): with pytest.raises(AssertionError) as excinfo: assert myfunc(41) assert 'custom report' in str(excinfo.value) def test_4failing(): assert myfunc(41) Running the above test gives 3 passing cases and 1 failing case (which uses the custom report). Also, test_2passing() checks for "is False" instead of just "== False" which I think would be wonderful to support as it removes all caveats for the user (so users get a real False when they expect False, instead of a Falsey alternative). Also, if I were going to use AST manipulation like this, I would probably reference assert_myfunc() by attaching it as a private attribute to myfunc() itself -- and then reference it with ast.Attribute() node instead of an ast.Name(). But again, solving this without AST manipulation could be better in many ways. --Shawn On Mon, Mar 19, 2018 at 1:59 PM, Ronny Pfannschmidt < ich(a)ronnypfannschmidt.de> wrote: > hi everyone, > > this is just about single value assertion helpers > > i logged an feature request about that a few year back > see https://github.com/pytest-dev/pytest/issues/95 - > > so basically this use-case was known since 2011 ^^ and doesn't require > ast rewriting lice macros, > just proper engineering of the representation and handling of single > values in the assertion rewriter. > > -- Ronny > > > Am 19.03.2018 um 15:13 schrieb holger krekel: > > On Mon, Mar 19, 2018 at 15:03 +0100, Floris Bruynooghe wrote: > >> On Sun, Mar 18 2018, Shawn Brown wrote: > >>> Unfortunately, this does not solve my usecase. I'm trying to handle > cases > >>> where the following statement would pass: > >>> > >>> assert myfunc(failing_input) == False > >>> > >>> But where this next statement would fail using my custom report: > >>> > >>> assert myfunc(failing_input) > >>> > >>> Calling myfunc() needs to return True or False (or at least Truthy or > >>> Falsy)--this is locked-in behavior. > >> I'm not sure if this is compatible with Python's semantics really. If I > >> understand correctly you're asking for a full-on macro implementation on > >> Python or something. Which in theory you could do with an AST > >> NodeVisitor, but really Python isn't made for this -- sounds like you'd > >> enjoy lisp! ;-) > >> > >> The best thing I can suggest is to make use of the:: > >> > >> assert myfunc(failing_input), repr(myfunc(failing_input())) > > i wonder if one could try to rewrite the ast for "assert myfunc(x)" to > > "assert __pytest_funcrepr_helper(myfunc(x), 'myfunc(x)')" with > something like: > > > > class __pytest_funcrepr_helper: > > def __init__(self, val, source): > > self.val = val > > self.source = source > > def __bool__(self): > > return bool(self.val) > > def __repr__(self): > > return "{!r} returned non-true {!r}".format(self.source, > self.val) > > > > but maybe i am not grasping all details involved. It's been a while since > > i looked into ast-rewriting ... > > > > holger > > > > > >> functionality to also get a custom error message. Here your myfunc() > >> whould have to return some object which both implements __bool__ as well > >> as __repr__ I guess. > >> > >> Maybe there's a feature request in here for something like this:: > >> > >> class Foo: > >> def __bool__(self): > >> return False > >> > >> def __repr__(self): > >> return 'multiline\nstring' > >> > >> assert Foo() > >> > >> To actually show the repr in the error message, which it currently > >> doesn't. I'd like to know what other people think of such a feature > >> though, and haven't thought through all the implications yet. But I'm > >> curious, would something like that solve your case? > >> > >> Cheers, > >> Floris > >> _______________________________________________ > >> pytest-dev mailing list > >> pytest-dev(a)python.org > >> https://mail.python.org/mailman/listinfo/pytest-dev > > _______________________________________________ > > pytest-dev mailing list > > pytest-dev(a)python.org > > https://mail.python.org/mailman/listinfo/pytest-dev >

4 6

pytest 3.5.0 released!
by Bruno Oliveira March 22, 2018

March 22, 2018

The pytest team is proud to announce the 3.5.0 release! This release contains a large number of new command-line options, a lot of them made by new contributors which makes this all the more reason for celebration, make sure to take a look at the CHANGELOG to see what's new: http://doc.pytest.org/en/latest/changelog.html For complete documentation, please visit: http://docs.pytest.org As usual, you can upgrade from pypi via: pip install -U pytest Thanks to all who contributed to this release, among them: * Allan Feldman * Brian Maissy * Bruno Oliveira * Carlos Jenkins * Daniel Hahler * Florian Bruhin * Jason R. Coombs * Jeffrey Rackauckas * Jordan Speicher * Julien Palard * Kale Kundert * Kostis Anagnostopoulos * Kyle Altendorf * Maik Figura * Pedro Algarvio * Ronny Pfannschmidt * Tadeu Manoel * Tareq Alayan * Thomas Hisch * William Lee * codetriage-readme-bot * feuillemorte * joshm91 * mike Happy testing, The Pytest Development Team

1 0

License for pytest-dev/pytest-design
by Bruno Oliveira March 21, 2018

March 21, 2018

Hi folks, pytest-design is the repository used to store logo and t-shirt designs from our 2016 sprint, but it currently it isn't under any LICENSE. I've been asked if it was OK to use the logos to make a t-shirt (and of course I know it is), but we should add a license to the repository allowing just that. What license should we add? Cheers, Bruno.

3 3

Fixture ordering and scopes
by Bruno Oliveira March 20, 2018

March 20, 2018

Hi everyone and Holger, Looking at the code below: data = {} @pytest.fixture(scope='session')def clean_data(): data.clear() @pytest.fixture(autouse=True)def add_data(): data['value'] = 1 @pytest.mark.usefixtures('clean_data')def test_foo(): assert data.get('value') Should test_foo fail or pass? Keep your answer in mind before proceeding. :) ------------------------------ I ask this because *unrelated* fixtures, i.e. which don’t depend on each other, are executed in the order they are declared in the test function signature, regardless of scope. The example above *fails*, because the fixtures are executed in (add_data, clean_data) order: add_data, being *autouse*, is added to the beginning of the argument list, and afterwards clean_data is inserted because of the usefixtures mark. This came up in #2405 <https://github.com/pytest-dev/pytest/issues/2405>, where Jason Coombs assumed that clean_data, being session-scoped, would always be executed first. I wonder if the current state of things is by design, or just an accident of how things work? I opened up a PR <https://github.com/pytest-dev/pytest/pull/3306> which does sort parameters by scope while keeping the relative order of fixtures of same scope intact, and the test suite passes without failures so if the current behavior is by design there are not tests enforcing it. Using code in the PR, then things might also be surprising: @pytest.fixture(scope='session')def s(): pass @pytest.fixture(scope='function')def f(): pass def test_foo(f, s): pass s and f are unrelated and execute in (f, s) order in master, but in (s, f) order in my branch. Would like to hear what people think about this matter, as I think it is an important one, specially *Holger* if this is a design decision or just an accident of implementation, and if we should change it. Also, please feel free to just reply with what you thought what should be the behavior of the first sample. Cheers, Bruno.

5 16

mark fix-up major milestone, please review
by Ronny Pfannschmidt March 20, 2018

March 20, 2018

Hi everyone, in https://github.com/pytest-dev/pytest/pull/3317 i introduced a new way to store marks and aslo consistently use it (this fixed a few bugs pytest dragged along since years) some of the things i had to do is introduce a internal FunctionDefinition node that's currently just being used by the collectors to give metafunc correct access to marks. i'm hoping to eventually elevate that in future, but the amount of spaghetti makes that a major undertaking, pytest is just so tightly coupled all over the place that it is suffocating and really error inducing I also did deprecate markinfo attributes, so everyone using them will get deprecation warnings. as far as i'm concerned, markinfo attributes where never ever a correct way to handle distinct marks and quite a few bugs in pytest came from using combined marks in markinfo instead of distinct marks. i also changed get_marker, and starting to realize that i need to make it return a MarkInfo again (at least one with correct data this time around) (that's upcoming in a few) other than that please take a look and find details to scrutinize -- Ronny -- Red Hat GmbH, http://www.de.redhat.com/, Registered seat: Grasbrunn, Commercial register: Amtsgericht Muenchen, HRB 153243, Managing Directors: Charles Cachera, Michael Cunningham, Michael O'Neill, Eric Shander

3 8

Custom reporting for asserts without comparison operators?
by Shawn Brown March 19, 2018

March 19, 2018

I understand how to use pytest_assertrepr_compare() to return custom assertion reports--but this interface requires a comparison operator. I'm hoping to write a plugin that makes custom reports for statements like: assert myfunc(myobj) Where myfunc() returns True or False. But without an operator (e.g. "=="), I can't intercept the results to build my custom report. While I *could* use a comparison operator, doing so is semantically wrong for most of my use cases! And in the spirit of pytest and Pythonic code, I'm hoping to handle the "assert myfunc(myobj)" case as written because it's clean and caters to the test writer. I investigated the idea of trying to manipulate the AST before the tests are executed by using other pytest-supported hooks but this doesn't seem possible. I also looked at the idea of subclassing pytests's AssertionRewriter but there doesn't seem to be a mechanism for supporting this within the context of non-invasive plugin behavior--it looks like I would have to monkey patch pytest itself which isn't a route I want to take. I've also looked at returning a subclassed False instance that contains my report info and uses a custom repr for displaying it. But this was too much of a hack and caused problems. My current current "solution" is to have an assert_myfunc() function that raises errors directly so the statements read "assert_myfunc(myobj)". But I'd really like to use Python's native assert for this. Are there other approaches I might take to rewrite an assertion report for statements like "assert myfunc(myobj)" in a pytest-friendly/non-fragile way?

4 6

Parametrized autouse fixtures
by Thomas De Schampheleire March 17, 2018

March 17, 2018

Hello, The Kallithea project is a repository hosting and review system, currently supporting git and hg. We currently have some test cases that need to be run for these two version control systems. Previously this was done with some Python magic, which was now simplified and made more explicit in commit: https://kallithea-scm.org/repos/kallithea/changeset/45a281a0f36ff59ffaa4aa0… but I assume it can be made more automatic with pytest fixtures. I was thinking to use an autouse, parametrized fixture to set the backend to 'hg' and 'git' respectively. But I can't make it work. Here is the change I did on top of the mentioned commit: diff --git a/kallithea/tests/vcs/base.py b/kallithea/tests/vcs/base.py --- a/kallithea/tests/vcs/base.py +++ b/kallithea/tests/vcs/base.py @@ -5,6 +5,7 @@ InMemoryChangeset class is working prope import os import time import datetime +import pytest from kallithea.lib import vcs from kallithea.lib.vcs.nodes import FileNode @@ -27,6 +28,11 @@ class _BackendTestMixin(object): """ recreate_repo_per_test = True + @pytest.fixture(autouse=True, + params=['hg', 'git']) + def set_backend_alias(cls, request): + cls.backend_alias = request.param + @classmethod def get_backend(cls): return vcs.get_backend(cls.backend_alias) diff --git a/kallithea/tests/vcs/test_archives.py b/kallithea/tests/vcs/test_archives.py --- a/kallithea/tests/vcs/test_archives.py +++ b/kallithea/tests/vcs/test_archives.py @@ -14,7 +14,7 @@ from kallithea.tests.vcs.base import _Ba from kallithea.tests.vcs.conf import TESTS_TMP_PATH -class ArchivesTestCaseMixin(_BackendTestMixin): +class TestArchivesTestCaseMixin(_BackendTestMixin): @classmethod def _get_commits(cls): @@ -95,11 +95,3 @@ class ArchivesTestCaseMixin(_BackendTest def test_archive_prefix_with_leading_slash(self): with pytest.raises(VCSError): self.tip.fill_archive(prefix='/any') - - -class TestGitArchive(ArchivesTestCaseMixin): - backend_alias = 'git' - - -class TestHgArchive(ArchivesTestCaseMixin): - backend_alias = 'hg' but when running this I get: $ pytest kallithea/tests/vcs/test_archives.py Test session starts (platform: linux2, Python 2.7.14, pytest 3.4.2, pytest-sugar 0.9.1) benchmark: 3.1.1 (defaults: timer=time.time disable_gc=False min_rounds=5 min_time=0.000005 max_time=1.0 calibration_precision=10 warmup=False warmup_iterations=100000) rootdir: /home/tdescham/repo/contrib/kallithea/kallithea-review, inifile: pytest.ini plugins: sugar-0.9.1, localserver-0.4.1, benchmark-3.1.1 ―――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――― ERROR at setup of TestArchivesTestCaseMixin.test_archive_zip[hg] ――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――― kallithea/tests/vcs/base.py:70: in setup_class Backend = cls.get_backend() kallithea/tests/vcs/base.py:38: in get_backend return vcs.get_backend(cls.backend_alias) E AttributeError: type object 'TestArchivesTestCaseMixin' has no attribute 'backend_alias' 7% ▊ ―――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――― ERROR at setup of TestArchivesTestCaseMixin.test_archive_zip[git] ―――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――― kallithea/tests/vcs/base.py:70: in setup_class Backend = cls.get_backend() kallithea/tests/vcs/base.py:38: in get_backend return vcs.get_backend(cls.backend_alias) E AttributeError: type object 'TestArchivesTestCaseMixin' has no attribute 'backend_alias' 14% █▌ [..] So the parametrization seems to work because each test is run twice, but I can't seem to find the right way to set the backend_alias, such that a later call from a classmethod works correctly. I found some possibly useful magic by nicoddemus but could not make that work either: https://github.com/pytest-dev/pytest/issues/2618#issuecomment-318519875 Is it possible to achieve what I want, without changing each test method to take a fixture explicitly? Thanks, Thomas

3 4

pytest-commit emails
by Bruno Oliveira March 17, 2018

March 17, 2018

Hi everyone, I receive an email from pytest-commit(a)python.org whenever new merges happen on a branch of the pytest repository, but it seems it happens only for my own merges. It was my understanding I should receive for any merge, not just my own. Does this happen for anybody else or is it just me? Cheers, Bruno

2 5