[pytest-dev] Re: solving the "too static" fixture scope problem

[Ronny Pfannschmidt]

Hi Holger,

i think that just 'each' is too generic,
i'd rather see something more specific like scope='session,function'

i think its very important to be in control of the actual scopes used in a fixture

an example i have in mind is a semi-generic tmpdir fixture

@pytest.fixture(scope='session,function')
def tmpdir(request, _pytest_basetmp):
  if request.scope == 'session':
    return _basetmp.ensure('session', dir=1)
  elif request.scope == 'function':
    return _basetmp.ensure('tests', dir=1)\
      .make_numbered_dir(request.function__name__)


or a little more detailed idea about databases:

@pytest.fixture(scope='session,function'):
def db_connection(request):
  if db_transactions_nested and request.scope=='session':
    conn = connect(...)
    schema.create_all(bind=conn)
    dbsetup.install_initial_data(bind=conn)
    return conn
  elif db_transactions_nested and request.scope=='function':
    conn = request.get_fixture('db_connection', scope='session')
    transaction = conn.begin_nested()
    request.addfinalizer(transaction.rollback)
    return conn
  elif request.scope=='function':
    conn = connect(...)
    schema.create_all(bind=conn)
    dbsetup.install_initial_data(bind=conn)
    return conn
  else:
    pyyest.fail('unexpected scope state')
    
the key point for me is being explicit about scopes when declaring fixtures and requesting them.

-- Ronny

On Freitag, 11. Oktober 2013 10:40:45 CEST, holger krekel wrote:
> Hi pytest users and developers,
>
> I'd like to discuss and get feedback on 
> extending the fixture mechanism and fix what
> i consider a biggest limitation at the moment.
>
> Problem: fixtures are tied statically to a scope
> =================================================
>
> For example, you cannot use monkeypatch in a higher
> than "function" scoped fixture. Same is true for
> tmpdir and probably also many user-defined fixtures.
> I've certainly had this problem myself many times
> that i had a fixture function that didn't really
> care in what scope it was used.  There are
> ways to get around this problem but they are not
> pretty:
>
>     @pytest.fixture(scope="module")
>     def myfix_module(request
>         return _myfix(request)
>     
>     @pytest.fixture(scope="function")
>     def myfix_function(request
>         return _myfix(request)
>
> where _myfix is the function that doesn't
> care about the actual scope.  Even then, you
> can't use builtin fixtures like "monkeypatch",
> "tmpdir", etc.
>
> Solution Idea: introduce "each" scoped fixtures
> =====================================================
>
> The idea is allow a fixture function to declare it wants
> to be used in the scope of the requesting fixture function
> (or at function-scope if used from a test).
>
> This is how "monkeypatch" would be implemented then:
>
>     @pytest.fixture(scope="each")
>     def monkeypatch(request):
>         ...  # same implementation as in _pytest/monkeypatch.py
>
> The new "each" scope means that each fixture/test requesting
> the "monkeypatch" fixture would receive its own fixture instance.
>
> So a session-scoped fixture could naturally use it like this:
>
>     @pytest.fixture(scope="session")
>     def myfix(monkeypatch):
>         monkeypatchsetattr(...)
>         return some_value
>
> The passed in monkeypatch object here is a specific instance just 
> for the ``myfix`` fixture function: "each" fixture function
> requesting ``monkeypatch`` gets a new instance of it.  
> If e.g. a test uses another module-scoped fixture defined like this:
>
>     @pytest.fixture(scope="module")
>     def myfix2(monkeypatch):
>         mp.setattr(...)
>         return some_value
>
> this would invoke the ``monkeypatch`` fixture function a second time, 
> resulting in a new instance for use by the ``myfix2`` instance.
>
> The same logic could be applied to other fixtures
> like "tmpdir" or user-defined ones.
>
> Do you like this idea? Would you find it helpful for your test suites?
>
> There is one issue i am not sure about yet, however.  Currently,
> when a test requires fixture A and B, and B requires C and C requires A,
> then the two "A" would be exactly the same object, independently of what
> which scopes are declared.  If A=="tmpdir", then the test's tmpdir and 
> C's tmpdir would be the same directory.  I often don't find 
> this desirable.  
> If tmpdir would be an "each" scoped fixture, then C and the test would 
> each receive a clean new tmpdir.  If that is a backward-compat issue, 
> we could introduce another name for the new "each" scoped tmpdir.
> I usually find myself working around the problem of a "tmpdir"
> shared by multiple different fixtures, though.
>
> cheers,
> holger

-- 
mfg,
Ronny Pfannschmidt