Mailman 3 November 2018 - pytest-dev

enhancing the assertion rewriting experience with custom integration/workflows
by Ronny Pfannschmidt July 29, 2019

July 29, 2019

Hi everyone, at a project at work which integrates pytest we did build a custom workflow around invoking tests and pytest. This in turn triggers a lot of warnings from the assertion rewriter about things that have been already imported. at first glance fixing this seems rather problematic, as the configuration of the assertion rewriting is deeply bound to the pytest configuration and it seems we cant do something simple and crude like letting a pth file fix the issue. i believe opening up the hooking system could be best done by moving it to a package with some sort of singleton control anyway - then pytest would be just a consumer of that api enabling if not enabled and push in its own "import roots". i'd like ot hear other impressions and ideas on this as well. -- Ronny

2 2

Looking for additional maintainers for pytest plugins
by Dave Hunt Dec. 3, 2018

Dec. 3, 2018

Due to a change in my role, I’m no longer going to be able to spend as much time as I have with maintaining the plugins listed below. The maintenance burden is very light, and releases are handled through Travis CI. I’ve already had some excellent help from Jim Brännlund (BeyondEvil), and have added him as a developer to a few of these. I’d appreciate any additional help from the community to support these plugins, and if you’re aware of somebody who would like to contribute please point them in my direction. https://github.com/pytest-dev/pytest-base-url <https://github.com/pytest-dev/pytest-base-url> - A very small plugin, no commits in over a year https://github.com/pytest-dev/pytest-selenium <https://github.com/pytest-dev/pytest-selenium> - Jim is a developer, and helps out a lot with this one, he's taken care of much of the recent development and responding to issues https://github.com/pytest-dev/pytest-html <https://github.com/pytest-dev/pytest-html> - This is probably the most popular plugin I’ve been maintaining, and I think there’s a lot of potential for enhancements (great ideas already in the tracker) https://github.com/pytest-dev/pytest-metadata <https://github.com/pytest-dev/pytest-metadata> - Split off as a dependency for pytest-html, relatively small plugin and minimal maintenance needs. Jim is a developer for this plugin too https://github.com/pytest-dev/pytest-variables <https://github.com/pytest-dev/pytest-variables> - Another small plugin with very few maintenance needs https://github.com/pytest-dev/pytest-repeat <https://github.com/pytest-dev/pytest-repeat> - I don’t really maintain this one currently, but feel that it’s worth mentioning because I was one of the early contributors I’m happy to answer any questions, and keen to hear any suggestions for handling any handover I’m sure these are in safe hands, and I will continue to use them myself, and promote the use of them to others! Cheers, Dave

5 6

pyi stubs (PEP 561/ PEP-484) for pytest
by Ilya Kazakevich Nov. 26, 2018

Nov. 26, 2018

Hello, My name is Ilya, I develop pytest integration in PyCharm IDE. There is a PEP-484 that covers optional type hinting for Python an PEP-561 that describes how to distribute this information as separate package with .pyi files (also called stubs). Type hinting is huge benefit for any static analysis tool. At this moment it is suported by mypy (http://mypy-lang.org/) and PyCharm. My question is do you have any plans to create pyi stubs for pytest? .pyi will improve code insight / code completion in PyCharm and give users ability to validate their code with mypy. We can create this package by ourselves (physically we create pytest-subs package on pypi, see PEP-561) but at least we need approvement from pytest devs, and any contributions are welcome) Creating pyi files is not hard, and they could be used by any PEP-561 compatible project. Ilya.

3 3

pytest 4.0.1
by Bruno Oliveira Nov. 24, 2018

Nov. 24, 2018

pytest 4.0.1 has just been released to PyPI! This is a bug-fix release, being a drop-in replacement. To upgrade:: pip install --upgrade pytest The full changelog is available at https://docs.pytest.org/en/latest/changelog.html. Thanks to all who contributed to this release, among them: * Anthony Sottile * Bruno Oliveira * Daniel Hahler * Michael D. Hoyle * Ronny Pfannschmidt * Slam Happy testing, The pytest Development Team

1 0

pytest-harvest
by Sylvain MARIE Nov. 14, 2018

Nov. 14, 2018

Dear all I am very glad to announce my new plugin pytest-harvest https://smarie.github.io/python-pytest-harvest/ , designed to collect selected fixtures and applicative test results so as to create synthesis tables. A typical use-case (at least the reason why I built it and will use it :) ) is to easily create benchmarks for data science algorithms. I now plan to work on a "pytest-patterns" project that will host reference examples for some design patterns. The example that I want to create first is (how surprising...) a combination of pytest-cases, pytest-steps and pytest-harvest. Maybe at some point I will move the 4 projects under a dedicated "pytest-patterns" github organization, I don't know yet. Best regards and happy pytest 4.x ! (and do not hesitate to provide feedback on the issues page!) Sylvain -----Message d'origine----- De : pytest-dev [mailto:pytest-dev-bounces+sylvain.marie=se.com@python.org] De la part de Sylvain MARIE Envoyé : lundi 5 novembre 2018 22:26 À : pytest-dev <pytest-dev(a)python.org> Objet : [pytest-dev] parameters and fixtures, storage and benchmarks: feedback appreciated [External email: Use caution with links and attachments] ________________________________ Dear pytest development team In our data science team we develop a lot of machine learning/statistics components and a recurrent need is benchmarking codes against many reference datasets. By benchmarking we often mean "applicative" benchmarking (such as algorithm accuracy), even if execution time may also be looked at as a secondary target. For this reason https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.c… does not seem to suit our needs. I developed several versions of a benchmarking toolkit in the past years for this purpose but I managed to port it entirely to pytest, so as to leverage all the great built-in mechanisms (namely, parameters and fixtures). It is mostly for this reason that I broke down the problem into smaller pieces and open-sourced each piece as a separate mechanism. You already know pytest-cases (to separate the test logic from the test cases/datasets) and pytest-steps (to break down the test logic into pieces while keeping it quite readable), the last piece I completed is about storing test results so as to get a true "benchmark" functionality with synthesis reports at the end. I broke down the mechanisms into two parts and would like to have your opinion: - one part will be a decorator for function-scoped fixtures, so as to say "please store all these fixture's instances" (in a dictionary-like storage object under key=test id) - another part will be a special fixture, decorated with the above, allowing people to easily inject "results bags" into their test functions to save results and retrieve them at the end Do you think that this (in particular the first item to store fixture) is compliant with pytest ? From investigating, I found out that params are currently stored in session but not fixtures (purposedly). It therefore seems natural to propose a way for users to store very specific fixtures on a declarative basis. Concerning the place where to store them (the global storage object), I am currently allowing users either to rely on a global variable or on a fixture. Would there be another preferred way to store information in pytest, across all tests ? Of course compliance with pytest-xdist will be a holy grail in the future, I do not want to shoot at it directly but pick the 'least bad' choices. Thanks in advance for your kind review and advices ! Best regards -- Sylvain _______________________________________________ pytest-dev mailing list pytest-dev(a)python.org https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmail.pyt… ______________________________________________________________________ This email has been scanned by the Symantec Email Security.cloud service. ______________________________________________________________________

1 0

pytest 4.0.0 released!
by Bruno Oliveira Nov. 14, 2018

Nov. 14, 2018

The pytest team is proud to announce the 4.0.0 release! This version is near identical to 3.10 feature-wise, except now pytest's own deprecated features generate errors, but they can be turned back into warnings as a stop gap measure until 4.1. Users are *strongly* encouraged to take a look at the CHANGELOG for instructions and rationale about the deprecated functionality: https://docs.pytest.org/en/latest/changelog.html For complete documentation, please visit: https://docs.pytest.org/en/latest/ As usual, you can upgrade from PyPI via: pip install -U pytest Thanks to all who contributed to this release! Happy testing, The Pytest Development Team

1 0

another "i have had it moment" and a stern request ro review our interaction with backward compatibility
by Ronny Pfannschmidt Nov. 14, 2018

Nov. 14, 2018

Hi everyone, a few days i was trying to replicate an issue in the python shell, and quickly realized, that outside of actually doing a complete pytest.main invocation there is simply no practical and sane way to get a config object that is not in a utterly broken unusable state to begin with. Again and again i keep running into things that have deep and nastyly hidden dependencies that are seemingly impossible to resolve. And we get clear resutls from that - again and again certain features simply cant be implemented or fearlessly delivered as the shoddy house of cards falls appart with spooky actions at a distance. Be it marks that smear over class hierachies or fixture definitions that blow up for undiscoverable reasons when trying to introduce the multi scope fixture system. Again and again many of us run into massive impediments when trying to develop features simply for strange actions at distance. This is an accumulative cost in many ways - in particular for a project driven primarily by volunteers - every hour we sacrafice to this altar of technical debt is an hour lost for improving the project and/or working on actual features. >From my perspective this issue directly grows out of driving a large part of pytests development by examples and tests, but precluding stepping back and doing actual design. The history of marks makes a perfect example of such an horror storry. Each iteration adding a new minimal element while adding a huge factor to the structural error as it grew. I really want to hammer down there that "minimal" is very different from "minimal viable" - leaving design uncontested for too long is an ramp up for unsustainable technical debt. Its aslo critical to note that for **me** there is a inherent dishonesty in trying to stay backward compatible and not putting forward a design and development process that deeply supports it - right now we can observe a state of fragility from pytest where it feels like every feature release triggers some regression - while at the same time we keep and keep shipping features that are structurally and fundamentally broken since years (yield tests, config init, ...). This setup, which gives me the impression it is "designed" to make the project fail its user (aka its broken to begin with and now it will break some more), is a massive emotional drain. It painfully twists around the sense of responsibility simply due to the perceived inherent base-level of failure - and **I** want to do and feel better. However with the current flow of releases and with our backward compatibility policies in place i am under a very general impression that i cant really do anything about it, and even if i try it would drag on for years to generate even basic positive results - this impression is a killer to my motivation - as it **feels** like it would take years to even get a initial iteration out to the users - and the process of fixing the issues in general would drag on over not only years - but decades. A timeflow like that would commpletely disconnect me from perceptions of reward or archivement - in turn making it even less likely to succeed or even start. This is a open source project that volunteer driven - fixing deeper issues should connect to feeling an archivement - but with out current setup what seems to be in for **me** is more like dread, fear and depression - a general defeat. Thats not something i want to allow anymore. **I** want to feel good about my work on pytest. **I** want to feel the archivements i have on pytest. and because of that **I** have to make sure the project can support that. So **I** want to invite all of you, Lets make pytest a project again where * we claim suport for backward compatibility and our actions and results show for it * we can enjoy the act of adding new features and enhancing the ecosystem * we can have better and quicker feedback, not just from the fabulous people that work on it - but also the users. **I** strongly beleive that this is doable, but it will require cutting some bad ends. We need to enhance layering, it doesnt have to be perfect but we do have to be able to keep things appart sanely and we need to have objects in valid states per default (invalid states should only be possible deliberately, not accidentially) We need to talk/write about design more - the difference between minimal and viable after all only shows itself in the heat and fruction of different perspectives and oppinions clashing. We need a conversation and interaction with our advanced users, so they dont end up on old and dead pytest versions (like pypy for example). We need a pervasive chain reason to bring trough the period of papercuts where we do shift around layering so the ecosystem can keep up (saving the structural integrity of pytest at the expense of destroying the ecosystem would be a dissaster). of course there is a obligatory white on red "make pytest great again" cap in planning ;P -- Ronny

5 4

pytest 3.10.1 released
by Bruno Oliveira Nov. 11, 2018

Nov. 11, 2018

pytest 3.10.1 has just been released to PyPI. This is a bug-fix release, being a drop-in replacement. To upgrade:: pip install --upgrade pytest The full changelog is available at https://docs.pytest.org/en/latest/changelog.html. Thanks to all who contributed to this release, among them: * Anthony Sottile * Boris Feld * Bruno Oliveira * Daniel Hahler * Fabien ZARIFIAN * Jon Dufresne * Ronny Pfannschmidt Happy testing, The pytest Development Team

1 0

Deprecating "calling" Nodes
by RonnyPfannschmidt Nov. 7, 2018

Nov. 7, 2018

Hi everyone, i'm facing a really harsh dilemma with the the way we initialize Nodes and their subclasses the __init__ methods are just plain "wrong" in terms of doing a lot of logic and "Magically" putting things in place. this makes both structural changes and porting to attrs practically impossible In order to elevate this structural problem i'd like to sort out the way we invoke them and deprecate calling them main reason being that if direct "calling" is no longer valid behavior for clients and down streams, the logic can be moved to more controllable and testable places in order to facilitate this, i'd create a NodeMetaClass which has support methods for instance creation and allows for custom functions/methods to be called. -- Ronny

2 1

breaking releases required - sorting out FunctionDefinition and Compatproperties
by Ronny Pfannschmidt Nov. 7, 2018

Nov. 7, 2018

Hi everyone, introducing FunctionDefinition to the collectiontree will trigger indirect api breaches for the compat property setup we have in place to support node.CollectorClass im unwilling to spend a hard to estimate amount of time on a feature that is indeed deprecated since years and will be removed in near future, so i rather want it gone and out of the way now so i can actually work on structral cleanup of things that will stay rather than resurrecting the dead. i propose we prepare an actual breraking release since its about time for that since quite a while. -- Ronny

3 2