[pytest-dev] pytest-xdist-1.11: restarting crash nodes, fixture cache fix

[holger krekel]

Hi Andreas, all,

On Fri, Sep 19, 2014 at 09:06 +0200, Andreas Pelme wrote:
> Hi,
> 
> First: Thanks for the fixes and release, my test suite runs fine with the new version!
> 
> On 18 sep 2014, at 20:56, holger krekel <holger at merlinux.eu> wrote:
> > On Thu, Sep 18, 2014 at 20:51 +0200, Anatoly Bubenkov wrote:
> >> hmm, but that grouping is not important for session scoped fixtures, right?
> >> i mean, what's the sense of tearing down the session-scoped fixture, even
> >> if most likely it will be not used by any test, earlier than the end of the
> >> session?
> > 
> > Parametrized session-scoped fixtures, for example, may depend on being torn
> > down, not all be active at the same time.  In any case, it's part of a
> > general scheme to teardown fixtures as early as possible, not as late as
> > possible.  We can discuss/play with that and introduce an option which
> > prohibits it, possibly.
> 
> 
> I’d like an option/hint (in a sense similar to pytest.mark.trylast) to tell pytest to tear down fixtures as late as possible. Or a higher level scope than session, that is cached until the end of the entire test run.

Interesting idea.  Maybe we could add something to the fixture decorator, e.g.:

    @pytest.fixture(scope="session", lateteardown=True)

> I’ve was not aware of this situation until I hit issue 503, and have always assumed that session fixtures cannot be teared down more than once.
> 
> pytest-django’s internal fixtures and other fixtures I’ve written depends on having a single invocation for session fixtures. The teardown bugs in these fixtures can (and should) of course be fixed to work properly multiple times.
> 
> Even if these fixtures were correct, from a performance point of view, tearing down some of these fixtures is a big issue, since session fixtures are typically very slow in the first place.

I'd like to see if with pytest-xdist-1.11 there still is a problem.
Let's try to make everything work by default without adding new options.
We might just allow non-parametrized session fixtures to survive
until the end of a session (or rather a process: i now think that
"session" should have been named "process" because it better avoids
misconceptions when things run distributed).  A "non parametrized
session fixture" means that it is itself not parametrized and none of
its fixture dependencies is, neither (recursive definition).  We should
be able to determine this property at collection time and thus postpone 
teardown accordingly.

I presume that with module and class scope people don't presume that
a fixture survives until the end of a process so eager teardown is less
of a problem, there, right?

best,
holger