On 2 May 2019, at 11:28, Florian Bruhin wrote:

Hey,

On Thu, May 02, 2019 at 11:22:08AM +0200, Florian Schulze wrote:

...

Is there a way to run a bunch of tests like with --lf but don't clear the cache when they pass? This would be mighty helpful for git bisect. For now I helped myself by copying the test ids, but that requires editing the list copied from the terminal before being able to paste them to the command line.

I've needed something similar before (like when I want to run a set of failing tests, but maybe run some other set in between).

What would be nice is something like a --save-collected <name> and --load-collected <name>. Then you could do "--lf --save-collected bisect" once, and do "--load-collected bisect" to rerun exactly those tests.

I wanted to write a plugin for that for months (if not years) now, but never got around to it. So far I've also done what you described.

Any hints on which hooks would be most useful for gathering the failing tests (or maybe a hint for existing code to look at)? Then I might write that plugin :) Regards, Florian Schulze

On 02.05.19 11:28, Florian Bruhin wrote:

...

Is there a way to run a bunch of tests like with --lf but don't clear the cache when they pass?

What would be nice is something like a --save-collected <name> and --load-collected <name>. Then you could do "--lf --save-collected bisect" once, and do "--load-collected bisect" to rerun exactly those tests.

I wanted to write a plugin for that for months (if not years) now, but never got around to it. So far I've also done what you described.

See https://github.com/pytest-dev/pytest/issues/5073 for general improvements for the last-failed plugin, and https://github.com/pytest-dev/pytest/issues/1017 from 2016 for this specific issue. I think this should be addressed/fixed there, instead of creating a new plugin. As for workarounds, I've used to do `cp -a .pytest_cache/v/cache/lastfailed{,.stash}` (and the other way around) also. But a better workaround is using a wrapper that puts the cache in its own Git directory, allowing for `p-cache commit -a -m saved`, `p-cache checkout .` etc then. (it also clearly showed a problem with the --newest-first plugin (#5206)). ``` #!/bin/sh # Wrapper around git in .pytest_cache/v/cache. set -e # TODO: could use `m=((../)#.pytest_cache(N))` in Zsh for upward search. root=.pytest_cache/v/cache if ! [ -d "$root" ]; then error "Directory $root does not exist" fi cd "$root" if ! [ -e ".git" ]; then git init . git commit --allow-empty -m "init" git add --all . git commit -m "initial state" fi git "$@" ```