[pytest-dev] Case for @fixture to support yield-style fixtures directly

[Vladimir Keleshev]

I would like to make a case for making yield-style fixtures
to be supported directly by the @fixture decorator.

But before that I want to remind some of you on what I am
talking about.

Yield-sytle fixtures are planned to be added in pytest 2.4,
but their exact API is under discussion.

Let me remind you that yield-style fixture looks like:

    @fixture
    def empty_db():
        db = DB()           # Setup code.
        yield db
        db.clear().close()  # Teardown code.

Now contrast with pytest 2.3 fixtures:

    @fixture
    def empty_db(request):
        db = DB()
        request.addfinalizer(lambda: db.clear().close())
        return db

Both create an instance of a hypothetical database,
return it as a fixture, then perform cleanup. For a similar
real-life example take a look here:
http://github.com/halst/urlbeat/blob/master/test_urlbeat.py

Current docs draft (http://pytest.org/dev/yieldfixture.html)
states some advantages and disadvantages which I want to
comment on.

1.1 Streamlined control flow.

Callbacks are known to (visually) break the order of
control flow, which contributes to their poor readability.
At the same time, having tear-down code after `yield` might
seem unusuall at first, but I think that such fixtures are
so readable to the point of being called beautiful.  You
can read them from top to bottom: first, set-up code, then
the object is yielded, then the tear-down follows.

1.2 No need for `request` argument.

In my experience with pytest 2.3 and before, the `request`
argument of fixtures is used for defining finalizers in
about 90% of the cases.  Other uses are relatively rare.
Yield-sytle fixtures allow to simplify this by allowing 
to drop that argument.

1.3. Simple fixtures that use context managers.

This is important; yield-style fixtures allow to use
context managers (think `with` statement) inside fixtures.
For example, if a `DB` class is designed to be a context
manager, you would write the following yield-style fixture:

    @fixture
    def emtpy_db():
        with DB() as db:
            yield db

However, with current pytest 2.3 fixtures that would 
approximately be:

    @fixture
    def empty_db(request):
        db = DB.__enter__()
        request.addfinalizer(
            lambda: db.__exit__(None, None, None))
        return db

What I want to stress is that context managers were
specifically introduced for this purpose: handle resources
that need some set-up and tear-down.  Almost all such
resources now support them.  That is one of the reasons,
I think, that yield-style fixtures are such a huge
improvement.

* * *

To sum up the advantages: you can argue about readability,
but you can't argue that support for context-managers is
an extremely important use-case.  Both fixtures and context
managers are made primarily to set-up and tear-down
resources. Increasing numbers of API support context
managers, and I'm sure that most of existing fixtures
(that use `.close()` and `.clear()` and similar methods)
could be rewritten more elegantly (and more uniformly) to
use yield-style fixtures with context managers.

* * *

Now, to comment on disadvantages:

2.1. Existing fixtures might use `yield` for ad-hoc
parametrization.

That means that yield-sytle fixtures might break existing
code that uses generators for parametrization like this:

    @fixture
    def all_backends(request):
        for Backend in [MemoryBackend,
                        BinaryBackend,
                        CSVBackend]:
            backend = Backend()
            yield backend
            request.addfinalizer(backend.clear)

    def test_backends(all_backends):
        for backend in all_backends:
            assert backend.put('key', 42).get('key') == 42

For one, I have never seen such code, and for two, the same
could be achieved more elegantly with parametrization API:

    @fixture(params=[MemroyBackend,
                     BinaryBackend,
                     CSVBackend])
    def backend(request):
        backend = request.param()
        request.addfinilizer(backend.clear)
        return backend

    def test_backends(backend):
        assert backend.put('key', 42).get('key') == 42

Plus, as of pytest 2.4.0.dev11, the user of such ad-hoc
generator fixture will see a very clear error message:

    fixture function backends has more than one `yield`

Which could elaborate even more if necessary:

    Fixture function `backends` has more than one `yield`.
    Since pytest 2.4 fixtures with `yield` statement are
    treated differently: <link>.  You might want to use
    parametrization API instead: <link>.

2.2 Yield-style fixtures introduce more that one way to do
fixtures (together with current return-style fixtures)

Yield-style fixtures present us with a superset of
functionality of what return-style fixtures can achieve.
I would even like to say that except for backwards-compatibility
and user habit concerns, there is no need for return-style
fixtures at all, if yield-style is available.  So I would
even argue that yield-style fixtures should take the main
place in fixture documentation, and return-style fixtures
mentioned for backwards compatibility (as funcargs are).

* * *

To sum up, the only real disadvantage of yield-style
fixtures is that they can break a single hypothetical
use-case, which is either rare or nonexistent, and for 
which a clear error-message is provided.

On the other hand, they allow to use context managers for
finalization, which were specifically designed to hanldle
this use-case elegantly.

I'm pretty sure that 100% of fixtures that use
`request.addfinalizer` will benefit from yield-style
fixtures, and that most of them will be able to use
context managers for finalization.

Just look at all the evolution steps.

3.1 Past:

    def pytest_funcarg__db(request):
        db = DB()
        request.addfinalizer(db.clear)
        return db

3.2 Present:

    @fixture
    def db(request):
        db = DB()
        request.addfinalizer(db.clear)
        return db

3.3 Future:

    @fixture
    def db():
        db = DB()
        yield db
        db.clear()

Or even better:

    @fixture
    def db():
        with DB() as db:
            yield db

So, do you agree that this is the future we want to be in?

* * *

There are a certain challenges that we need to solve then.

4.1 Backwards-compatibility (which I already mentioned).

4.2 There is a high chance (in my experience) of mistyping
`return` for `yield` like:

    @fixture
    def db():
        db = DB()
        return db
        db.clear()

Even worse, this fixture could still appear to work, until
some later time, when they system runs out of file
descriptors, for example.

Here are the 3 solutions discussed:

5.1 Extra argument to `fixture` to designate yield-style:

    @fixture(yield_context=True)
    def db():
        db = DB()
        yield db
        db.clear()

5.2 Different fixture decorator to designate yield-sytle:

    @yield_fixture
    def db():
        db = DB()
        yield db
        db.clear()

5.3 A configuration option, say, in pytest.ini that will
switch bettween return-style and yield-style.

What I would like to propose is:

6.1 Keep single @fixture decorator.

6.2 Give user a clear error-message in case fixture yields
more than one time (done).

6.3 Using simple static-analysis, warn user when there
are statements after `return`, to get rid of return/yield
typos.

4. Document yield-style as the preferable way of
finalization.

All of that keeps the API simple (no new decorators,
arguments, configuration options), warns user in case of
every known caveat, and finally, provides us the beautiful
yield-style fixtures by default.

—Vladimir Keleshev