From ran at unusedvar.com  Sat Apr  3 18:13:00 2021
From: ran at unusedvar.com (Ran Benita)
Date: Sun, 04 Apr 2021 01:13:00 +0300
Subject: [pytest-dev] pytest 6.2.3
Message-ID: <b7bbd023-78d1-4d97-aea2-df07a123547d@www.fastmail.com>

pytest 6.2.3 has just been released to PyPI.

This is a bug-fix release, being a drop-in replacement. To upgrade::

  pip install --upgrade pytest

The full changelog is available at https://docs.pytest.org/en/stable/changelog.html.

Thanks to all of the contributors to this release:

* Bruno Oliveira
* Ran Benita

Happy testing,
The pytest Development Team

From christopher.eliot at nagrastar.com  Thu Apr 15 15:56:18 2021
From: christopher.eliot at nagrastar.com (Eliot, Christopher)
Date: Thu, 15 Apr 2021 19:56:18 +0000
Subject: [pytest-dev] How can I suppress cleanups when a test fails
Message-ID: <a2689edb70734c81b15644180838c408@nagrastar.com>

My test suit generates some intermediate files and other resources which typically would be deleted upon termination of the test.  However, if there is a failure, I would like to leave them in place to help in diagnosing the failure.

Is there a clean way to do this?

I'm already using a fixture to do cleanup, so I'm prepared to use some aspect of the fixture if that's appropriate.

Thanks,
Topher Eliot

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.python.org/pipermail/pytest-dev/attachments/20210415/089aea5a/attachment.html>

From christopher.eliot at nagrastar.com  Thu Apr 15 17:00:46 2021
From: christopher.eliot at nagrastar.com (Eliot, Christopher)
Date: Thu, 15 Apr 2021 21:00:46 +0000
Subject: [pytest-dev] How can I suppress cleanups when a test fails
In-Reply-To: <79054d9a-506b-9d62-4534-1e03f0c1fa43@ronnypfannschmidt.de>
References: <a2689edb70734c81b15644180838c408@nagrastar.com>
 <79054d9a-506b-9d62-4534-1e03f0c1fa43@ronnypfannschmidt.de>
Message-ID: <42ac5bda9d974c1b9f7aa6ed0faff444@nagrastar.com>

Thank you.  This might be helpful, but I also have other resources that are not files that I would like to selectively clean up or not.  These resources are in a relational DB, not a file system.  And no, I can't just wrap everything in a big transaction and then roll it back, unfortunately.

Topher Eliot

From: Ronny Pfannschmidt <opensource at ronnypfannschmidt.de>

Hi Christopher,

if you use the tmpdir/tmp_path fixtures pytest provides,
just don't do additional cleanup, pytest keeps the last 3 basetemps around precisely for that use-case and drops older ones

-- Ronny
Am 15.04.21 um 21:56 schrieb Eliot, Christopher:
My test suit generates some intermediate files and other resources which typically would be deleted upon termination of the test.  However, if there is a failure, I would like to leave them in place to help in diagnosing the failure.

Is there a clean way to do this?

I'm already using a fixture to do cleanup, so I'm prepared to use some aspect of the fixture if that's appropriate.

Thanks,
Topher Eliot
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.python.org/pipermail/pytest-dev/attachments/20210415/b0358b73/attachment.html>

From opensource at ronnypfannschmidt.de  Thu Apr 15 16:57:01 2021
From: opensource at ronnypfannschmidt.de (Ronny Pfannschmidt)
Date: Thu, 15 Apr 2021 22:57:01 +0200
Subject: [pytest-dev] How can I suppress cleanups when a test fails
In-Reply-To: <a2689edb70734c81b15644180838c408@nagrastar.com>
References: <a2689edb70734c81b15644180838c408@nagrastar.com>
Message-ID: <79054d9a-506b-9d62-4534-1e03f0c1fa43@ronnypfannschmidt.de>

Hi Christopher,

if you use the tmpdir/tmp_path fixtures pytest provides,
just don't do additional cleanup, pytest keeps the last 3 basetemps 
around precisely for that use-case and drops older ones


-- Ronny

Am 15.04.21 um 21:56 schrieb Eliot, Christopher:
>
> My test suit generates some intermediate files and other resources 
> which typically would be deleted upon termination of the test.? 
> However, if there is a failure, I would like to leave them in place to 
> help in diagnosing the failure.
>
> Is there a clean way to do this?
>
> I?m already using a fixture to do cleanup, so I?m prepared to use some 
> aspect of the fixture if that?s appropriate.
>
> Thanks,
>
> Topher Eliot
>
>
> _______________________________________________
> pytest-dev mailing list
> pytest-dev at python.org
> https://mail.python.org/mailman/listinfo/pytest-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.python.org/pipermail/pytest-dev/attachments/20210415/51eaeedb/attachment.html>

From opensource at ronnypfannschmidt.de  Thu Apr 15 17:05:08 2021
From: opensource at ronnypfannschmidt.de (Ronny Pfannschmidt)
Date: Thu, 15 Apr 2021 23:05:08 +0200
Subject: [pytest-dev] How can I suppress cleanups when a test fails
In-Reply-To: <42ac5bda9d974c1b9f7aa6ed0faff444@nagrastar.com>
References: <a2689edb70734c81b15644180838c408@nagrastar.com>
 <79054d9a-506b-9d62-4534-1e03f0c1fa43@ronnypfannschmidt.de>
 <42ac5bda9d974c1b9f7aa6ed0faff444@nagrastar.com>
Message-ID: <eef2c6c9-9b9d-f122-aad6-19098feb3a83@ronnypfannschmidt.de>

in case you use a orm/abstraction layer, sqlite in a tmpdir can be a 
wonderful companion

in case you use another db, you can actually have failures do a db/data 
dump if you like,

unfortunately i'm not aware of a tmp_path alike db setup/teardown tool

--Ronny

Am 15.04.21 um 23:00 schrieb Eliot, Christopher:
>
> Thank you. This might be helpful, but I also have other resources that 
> are not files that I would like to selectively clean up or not.? These 
> resources are in a relational DB, not a file system.? And no, I can?t 
> just wrap everything in a big transaction and then roll it back, 
> unfortunately.
>
> Topher Eliot
>
> *From:* Ronny Pfannschmidt <opensource at ronnypfannschmidt.de>
>
> Hi Christopher,
>
> if you use the tmpdir/tmp_path fixtures pytest provides,
> just don't do additional cleanup, pytest keeps the last 3 basetemps 
> around precisely for that use-case and drops older ones
>
> -- Ronny
>
> Am 15.04.21 um 21:56 schrieb Eliot, Christopher:
>
>     My test suit generates some intermediate files and other resources
>     which typically would be deleted upon termination of the test.?
>     However, if there is a failure, I would like to leave them in
>     place to help in diagnosing the failure.
>
>     Is there a clean way to do this?
>
>     I?m already using a fixture to do cleanup, so I?m prepared to use
>     some aspect of the fixture if that?s appropriate.
>
>     Thanks,
>
>     Topher Eliot
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.python.org/pipermail/pytest-dev/attachments/20210415/8b94a981/attachment-0001.html>

From christopher.eliot at nagrastar.com  Thu Apr 15 18:16:55 2021
From: christopher.eliot at nagrastar.com (Eliot, Christopher)
Date: Thu, 15 Apr 2021 22:16:55 +0000
Subject: [pytest-dev] How can I suppress cleanups when a test fails
In-Reply-To: <eef2c6c9-9b9d-f122-aad6-19098feb3a83@ronnypfannschmidt.de>
References: <a2689edb70734c81b15644180838c408@nagrastar.com>
 <79054d9a-506b-9d62-4534-1e03f0c1fa43@ronnypfannschmidt.de>
 <42ac5bda9d974c1b9f7aa6ed0faff444@nagrastar.com>
 <eef2c6c9-9b9d-f122-aad6-19098feb3a83@ronnypfannschmidt.de>
Message-ID: <980d60cf4e264be2bc188bf58de81c2e@nagrastar.com>

Thank you again.  What I really want is a way to ask Pytest, from inside a fixture or other python class, "hey Pytest, have you encountered any assertion failures during this session?"

Topher Eliot

From: Ronny Pfannschmidt <opensource at ronnypfannschmidt.de>
Sent: Thursday, April 15, 2021 3:05 PM
To: Eliot, Christopher <christopher.eliot at nagrastar.com>; pytest-dev at python.org
Subject: Re: [pytest-dev] How can I suppress cleanups when a test fails


in case you use a orm/abstraction layer, sqlite in a tmpdir can be a wonderful companion

in case you use another db, you can actually have failures do a db/data dump if you like,

unfortunately i'm not aware of a tmp_path alike db setup/teardown tool

--Ronny
Am 15.04.21 um 23:00 schrieb Eliot, Christopher:
Thank you.  This might be helpful, but I also have other resources that are not files that I would like to selectively clean up or not.  These resources are in a relational DB, not a file system.  And no, I can't just wrap everything in a big transaction and then roll it back, unfortunately.

Topher Eliot

From: Ronny Pfannschmidt <opensource at ronnypfannschmidt.de><mailto:opensource at ronnypfannschmidt.de>

Hi Christopher,

if you use the tmpdir/tmp_path fixtures pytest provides,
just don't do additional cleanup, pytest keeps the last 3 basetemps around precisely for that use-case and drops older ones

-- Ronny
Am 15.04.21 um 21:56 schrieb Eliot, Christopher:
My test suit generates some intermediate files and other resources which typically would be deleted upon termination of the test.  However, if there is a failure, I would like to leave them in place to help in diagnosing the failure.

Is there a clean way to do this?

I'm already using a fixture to do cleanup, so I'm prepared to use some aspect of the fixture if that's appropriate.

Thanks,
Topher Eliot
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.python.org/pipermail/pytest-dev/attachments/20210415/e6d0d102/attachment.html>

From nicoddemus at gmail.com  Fri Apr 16 20:41:22 2021
From: nicoddemus at gmail.com (Bruno Oliveira)
Date: Fri, 16 Apr 2021 21:41:22 -0300
Subject: [pytest-dev] How can I suppress cleanups when a test fails
In-Reply-To: <980d60cf4e264be2bc188bf58de81c2e@nagrastar.com>
References: <a2689edb70734c81b15644180838c408@nagrastar.com>
 <79054d9a-506b-9d62-4534-1e03f0c1fa43@ronnypfannschmidt.de>
 <42ac5bda9d974c1b9f7aa6ed0faff444@nagrastar.com>
 <eef2c6c9-9b9d-f122-aad6-19098feb3a83@ronnypfannschmidt.de>
 <980d60cf4e264be2bc188bf58de81c2e@nagrastar.com>
Message-ID: <CA+RQFfeM5dDTFDPspU=scEh5hEodxj6Czg+k3D8O8FPugf2McQ@mail.gmail.com>

Hi Eliot,

The only way to detect failures is using the pytest_runtest_logreport hook,
where you will obtain the test report for each test during each phase
(setup/call/teardown). You will need to store that somewhere so you can
reuse it later, possibly in a global variable or in self if you implement a
plugin class and register it during pytest_configure.

Here?s the example using a global variable, and printing the failed nodes
at the end of the session:

failed_tests = []
def pytest_runtest_logreport(report):
    if report.outcome == "failed":
        failed_tests.append(report.nodeid)
def pytest_terminal_summary(terminalreporter):
    terminalreporter.section("my custom report")
    terminalreporter.line(f"detected {len(failed_tests)} failures")

You can read more about hooks here:
https://docs.pytest.org/en/stable/writing_plugins.html#writing-hook-functions

Hope this helps,
Bruno.

On Thu, Apr 15, 2021 at 7:17 PM Eliot, Christopher <
christopher.eliot at nagrastar.com> wrote:

> Thank you again.  What I really want is a way to ask Pytest, from inside a
> fixture or other python class, ?hey Pytest, have you encountered any
> assertion failures during this session??
>
>
>
> Topher Eliot
>
>
>
> *From:* Ronny Pfannschmidt <opensource at ronnypfannschmidt.de>
> *Sent:* Thursday, April 15, 2021 3:05 PM
> *To:* Eliot, Christopher <christopher.eliot at nagrastar.com>;
> pytest-dev at python.org
> *Subject:* Re: [pytest-dev] How can I suppress cleanups when a test fails
>
>
>
> in case you use a orm/abstraction layer, sqlite in a tmpdir can be a
> wonderful companion
>
> in case you use another db, you can actually have failures do a db/data
> dump if you like,
>
> unfortunately i'm not aware of a tmp_path alike db setup/teardown tool
>
> --Ronny
>
> Am 15.04.21 um 23:00 schrieb Eliot, Christopher:
>
> Thank you.  This might be helpful, but I also have other resources that
> are not files that I would like to selectively clean up or not.  These
> resources are in a relational DB, not a file system.  And no, I can?t just
> wrap everything in a big transaction and then roll it back, unfortunately.
>
>
>
> Topher Eliot
>
>
>
> *From:* Ronny Pfannschmidt <opensource at ronnypfannschmidt.de>
> <opensource at ronnypfannschmidt.de>
>
> Hi Christopher,
>
> if you use the tmpdir/tmp_path fixtures pytest provides,
> just don't do additional cleanup, pytest keeps the last 3 basetemps around
> precisely for that use-case and drops older ones
>
> -- Ronny
>
> Am 15.04.21 um 21:56 schrieb Eliot, Christopher:
>
> My test suit generates some intermediate files and other resources which
> typically would be deleted upon termination of the test.  However, if there
> is a failure, I would like to leave them in place to help in diagnosing the
> failure.
>
>
>
> Is there a clean way to do this?
>
>
>
> I?m already using a fixture to do cleanup, so I?m prepared to use some
> aspect of the fixture if that?s appropriate.
>
>
>
> Thanks,
>
> Topher Eliot
>
> _______________________________________________
> pytest-dev mailing list
> pytest-dev at python.org
> https://mail.python.org/mailman/listinfo/pytest-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.python.org/pipermail/pytest-dev/attachments/20210416/c328bf00/attachment.html>

From schettino72 at gmail.com  Fri Apr 23 04:32:21 2021
From: schettino72 at gmail.com (Eduardo Schettino)
Date: Fri, 23 Apr 2021 16:32:21 +0800
Subject: [pytest-dev] proposal: plugin metadata
Message-ID: <CAHSrCY0V+LuYMYtBp_+LoO1-yRTxadWvyk1rbFKjuDt5=kCRYA@mail.gmail.com>

Hi,

I would like to propose a mechanism to improve the plugin list page.
https://docs.pytest.org/en/latest/reference/plugin_list.html

Currently there are over 800 plugins and relying on someone manually
curated them is not really feasible.
So the idea is to allow plugin authors to set some metadata that could in
turn be used to generate a more informative plugins page.

More specifically it should:

Clearly state to which version of pytest the plugin is compatible.
Currently it shows the last release date and minimum required version.
But it is not really clear if the latest version of pytest is
supported/tested.

Define one or more categories. Going through a list of 800+ plugins is a
lot of work.
Could define categories like:
 - framework-integration: i.e. pytest-airflow, pytest-django
 - output/report: i.e. pytest-clarity
 - coverage
 - ordering
 - etc..

In future could also add some more stuff, like download stats, github
stars, etc.


So the first step would be to define a schema and agree on a format (JSON?).
What do you guys think?

As a reference see how grafana does it:
https://grafana.com/docs/grafana/latest/developers/plugins/metadata/

Regards
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.python.org/pipermail/pytest-dev/attachments/20210423/3bdbb2e0/attachment.html>

From flub at devork.be  Sat Apr 24 12:21:32 2021
From: flub at devork.be (Floris Bruynooghe)
Date: Sat, 24 Apr 2021 18:21:32 +0200
Subject: [pytest-dev] proposal: plugin metadata
In-Reply-To: <CAHSrCY0V+LuYMYtBp_+LoO1-yRTxadWvyk1rbFKjuDt5=kCRYA@mail.gmail.com>
References: <CAHSrCY0V+LuYMYtBp_+LoO1-yRTxadWvyk1rbFKjuDt5=kCRYA@mail.gmail.com>
Message-ID: <CAAWk_Dx3YoVg4Co10nx1O59xwK9bann4n55ZTeSR+m2t=Gm_+w@mail.gmail.com>

Hi,

Previously there was https://plugincompat.herokuapp.com/ for testing
versions but eventually that was stopped as too much work to maintain
as I understand it.

As for further categorising, generally I think this could be nice but
you'd need a standard place for plugins to add this.  There's
currently the "Framework :: Pytest" PyPI classifier, maybe you can
convince PyPI to add more sub-classifiers, some other projects seem to
have this.  But maybe there's better ways these days, I've not kept up
to speed with python packaging recently.

Cheers,
Floris

On Fri, 23 Apr 2021 at 10:32, Eduardo Schettino <schettino72 at gmail.com> wrote:
>
> Hi,
>
> I would like to propose a mechanism to improve the plugin list page.
> https://docs.pytest.org/en/latest/reference/plugin_list.html
>
> Currently there are over 800 plugins and relying on someone manually curated them is not really feasible.
> So the idea is to allow plugin authors to set some metadata that could in turn be used to generate a more informative plugins page.
>
> More specifically it should:
>
> Clearly state to which version of pytest the plugin is compatible.
> Currently it shows the last release date and minimum required version.
> But it is not really clear if the latest version of pytest is supported/tested.
>
> Define one or more categories. Going through a list of 800+ plugins is a lot of work.
> Could define categories like:
>  - framework-integration: i.e. pytest-airflow, pytest-django
>  - output/report: i.e. pytest-clarity
>  - coverage
>  - ordering
>  - etc..
>
> In future could also add some more stuff, like download stats, github stars, etc.
>
>
> So the first step would be to define a schema and agree on a format (JSON?).
> What do you guys think?
>
> As a reference see how grafana does it:
> https://grafana.com/docs/grafana/latest/developers/plugins/metadata/
>
> Regards
> _______________________________________________
> pytest-dev mailing list
> pytest-dev at python.org
> https://mail.python.org/mailman/listinfo/pytest-dev