[pytest-dev] pytest-xdist-1.11: restarting crash nodes, fixture cache fix

[Anatoly Bubenkov]

On 19 September 2014 09:52, holger krekel <holger at merlinux.eu> wrote:

> Hi Andreas, all,
>
> On Fri, Sep 19, 2014 at 09:06 +0200, Andreas Pelme wrote:
> > Hi,
> >
> > First: Thanks for the fixes and release, my test suite runs fine with
> the new version!
> >
> > On 18 sep 2014, at 20:56, holger krekel <holger at merlinux.eu> wrote:
> > > On Thu, Sep 18, 2014 at 20:51 +0200, Anatoly Bubenkov wrote:
> > >> hmm, but that grouping is not important for session scoped fixtures,
> right?
> > >> i mean, what's the sense of tearing down the session-scoped fixture,
> even
> > >> if most likely it will be not used by any test, earlier than the end
> of the
> > >> session?
> > >
> > > Parametrized session-scoped fixtures, for example, may depend on being
> torn
> > > down, not all be active at the same time.  In any case, it's part of a
> > > general scheme to teardown fixtures as early as possible, not as late
> as
> > > possible.  We can discuss/play with that and introduce an option which
> > > prohibits it, possibly.
> >
> >
> > I’d like an option/hint (in a sense similar to pytest.mark.trylast) to
> tell pytest to tear down fixtures as late as possible. Or a higher level
> scope than session, that is cached until the end of the entire test run.
>
> Interesting idea.  Maybe we could add something to the fixture decorator,
> e.g.:
>
>     @pytest.fixture(scope="session", lateteardown=True)
>
how about:
@pytest.fixture(scope="<scope>", teardown='<(early or demand (default)) |
(session or late, not sure)>')

the teardown 'session' will clearly tell the user when this fixture will be
teared down
'late' also works, but not so clear

teardown 'demand' clearly tells that it's teardown will depend on the
demand of the fixture from the tests
'early' - same as with 'late'


>
> > I’ve was not aware of this situation until I hit issue 503, and have
> always assumed that session fixtures cannot be teared down more than once.
> >
> > pytest-django’s internal fixtures and other fixtures I’ve written
> depends on having a single invocation for session fixtures. The teardown
> bugs in these fixtures can (and should) of course be fixed to work properly
> multiple times.
> >
> > Even if these fixtures were correct, from a performance point of view,
> tearing down some of these fixtures is a big issue, since session fixtures
> are typically very slow in the first place.
>
> I'd like to see if with pytest-xdist-1.11 there still is a problem.
> Let's try to make everything work by default without adding new options.
> We might just allow non-parametrized session fixtures to survive
> until the end of a session (or rather a process: i now think that
> "session" should have been named "process" because it better avoids
> misconceptions when things run distributed).  A "non parametrized
> session fixture" means that it is itself not parametrized and none of
> its fixture dependencies is, neither (recursive definition).  We should
> be able to determine this property at collection time and thus postpone
> teardown accordingly.
>
session is rather good name, because if it will be the process, a lot of
questions will arise why pytest has session everywhere internally :)
also sometimes it's probably needed to start multiple sessions, who knows?

>
> I presume that with module and class scope people don't presume that
> a fixture survives until the end of a process so eager teardown is less
> of a problem, there, right?
>
> agree


-- 
Anatoly Bubenkov
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pytest-dev/attachments/20140919/4ea95154/attachment.html>