[py-dev] RFC: V2 of the new resource setup/parametrization facilities

holger krekel holger at merlinux.eu
Sat Jun 30 10:08:41 CEST 2012

Hi Floris,

some preliminary notes, i'll probably think some more about your feedback ...

On Sat, Jun 30, 2012 at 01:23 +0100, Floris Bruynooghe wrote:
> Hello Holger,
> On Fri, Jun 29, 2012 at 10:55:23AM +0000, holger krekel wrote:
> [...]
> > Direct scoping of funcarg factories
> [...]
> > Direct parametrization of funcarg factories 
> These two seem fine, but personally I would prefer them to use the
> same marker with keyword-only arguments::
>    @pytest.mark.factory(scope='session', parametrize=['mysql', 'pg'])
>    def pytest_funcarg__db(request):
>        ...
> This seems like a more natural API which collects the different
> functions, certainly when using both for one funcarg.

I'll consider it, probably under the name of "factoryattr" or so. 

> However it bothers me that funcargs now have two types of scope: an
> implied scope derived from where it is defined and which defines their
> visibility (e.g. only inside a class, module, directory).  And then
> this new scope which is essentially a caching/teardown scope.  The
> fact that the ScopeMismatch exception is needed is a result of this I
> think.

previously, the scope-mismatch could happen as well and go unnoticed::

    def pytest_funcarg__Y(request):
        return request.function.__name__

    def pytest_funcarg__X(request):
        def setup():
            return request.getfuncargvalue("Y")
        return request.cached_setup(setup, scope="session")

The result will depend on which test function is first requested.
In the future, we might want to try raise a ScopeMismatchError here
as well.

> The previous resource/funcarg split avoided this confusion.

a) What about just naming it "cachescope"?
b) i moved register_factory/getresource to implementation details
   not the least because Carl Meyer as a relatively recent pytest user
   expressed his expectation of a consistent pytest_funcarg__ factory
   story - and if we are going to anyway have to support the existing ones, 
   i'd now like to focus on extending it and only go for a usage-level visible
   paradigm change if it's really needed. Does this make general
   sense to you?
> Lastly, when do scoped funcarg resources get invoked?  Only at the
> time a test function requests it or always at the time when the scope
> is entered?

factories are invoked when a test function or one of its involved setup 
methods needs it.  A scope is only "entered" if there is a test to be executed
within it. Does this clarify?

> > support for setup_session and setup_directory
> > ------------------------------------------------------
> [...]
> >     # content of conftest.py
> >     def setup_session(db):
> >         ... use db resource or do some initial global init stuff
> >         ... before any test is run.
> >         
> >     def setup_directory(db):
> >         # called when the first test in the directory tree is about
> > to execute
> I think the naming of these functions break the py.test convention,
> normally the only functions picked up from conftest.py start with
> pytest_.  I can certainly imagine a conftest.py or plugin already
> having a setup_session function.  These are new functions and do not
> provide a compatibility API with other testing frameworks, so I think
> they would be better named pytest_setup_session and
> pytest_setup_directory.

I think using pytest_* hooks also has consistency problems:

* hooks cannot usually receive arbitrary funcargs
* xUnit-style consistency: consider explaining the new functions
  to someone only knowing setup_module/ class etc.

I am wondering, however, do we even need a "setup_session"? setup_directory
should usually be enough, i guess, and it's more unlikely people used
that name already (and we could warn about setup_session in 2.X to
reserve introducing it in 2.X+1).

And what what about putting setup_directory into an __init__.py file?
I don't really like requiring __init__ files, but am fine to go with it if
you and others prefer that.  I would guess, that using the already 
directory-scoped conftest.py file feels fine to someone coming new to pytest.

> It also feels slightly weird that they do not get their respective
> Node passed in.  This is a little inconsistent with the current
> setup_X method which all take a module, class or method argument.  I
> can't think of an immediate use for it as you can push out pretty much
> everything you need to do to a properly scoped funcarg resource.  

We can certainly add modulenode, classnode etc. to the respective
setup-methods because they participate in the funcarg-protocol 
(which allows accepting less parameters than are available).

> And following that reasoning the setup function would end up having no
> body at all, which also seems weird.

If a setup-function has no body, then tests could just require it themselves
and that'd be enough.  If there is a need, we could introduce a marker for 
requiring funcarg-resources such that tests do not need to require it 
in their signature.

> > Implementation level 
> > ===================================================================
> [...]
> > the "request" object incorporates scope-specific behaviour
> > ------------------------------------------------------------------
> [...]
> > In fact, the request object is likely going to provide a "node" 
> > attribute, denoting the current collection node on which it internally
> > operates.  (Prior to pytest-2.3 there already was an internal
> > _pyfuncitem).
> Does this mean you will revert the currently checked-in behaviour
> where a Node is actually a Request subclass and is the object passed
> to the funcarg resource factories?

Yes, probably.  Request-state basically consists of "(requested_funcargname,
node)" info currently.  There probably would be a request.node attribute 
which allows access to node information if needed.

> Hope this was helpful feedback,

Yes, certainly.  Hope i understood it correctly and my reply made 
sense so far.  



> Floris
> -- 
> Debian GNU/Linux -- The Power of Freedom
> www.debian.org | www.gnu.org | www.kernel.org

More information about the Pytest-dev mailing list