[pytest-dev] Configurable collection roots and reusable testsuites - Brainstorming

Ronny Pfannschmidt opensource at ronnypfannschmidt.de
Mon Jan 25 08:23:27 EST 2016

Im aware of that rudimentary mechanism,
i would like to elevate a "testsuite" to a set of collected tests in
filled with "dependencies" via fixtures

that approach would also provide a more clean mechanism that are
currently handled with session scoped fixtures that are parametrized

one of such examples would be backends for common operations

i think its clear that a more detailed approach is needed,
a more detailed discussion about the model might be a good idea for a sprint

-- Ronny

Am 25.01.2016 um 11:30 schrieb holger krekel:
> Hi Ronny, Ionel, all,
> On Mon, Jan 25, 2016 at 00:46 +0100, Ronny Pfannschmidt wrote:
>> Hi everyone,
>> a topic thats gripping since quite a while, is wanting to have reusable
>> testsuites,
>> so plugins/extensions that implement a behaviout,
>> can import and use integration tests from a installed package,
>> and use them in a different configuration
> That's already possible to a degree.
> If you hg-checkout https://bitbucket.org/hpk42/devpi and then look at
>     devpi/server/test_devpi_server/functional.py
> which is included from
>     devpi/server/test_devpi_server/test_views.py
> and also from the client side (from the installed devpi-server package):
>     devpi/client/testing/test_functional.py
> you'll note that the same tests run against different versions of a
> fixture, i.e. "mapp" which gives high level access to devpi behaviour.
> ASFAIK this pattern is also used by other projects.
> I was sometimes thinking that some way to tell pytest more dynamically
> what to collect, i.e. adding something to the test collection would
> be nice.  You can already do it by hooking into the plugin collection 
> hooks but that's a bit low-level as you need to tie into the collection
> tree carefully.  The hard part of "it would be nice" here is to write
> down a suggestion in forms of documentation on what API/semantics exactly 
> we want to add and then go for implementing it.
> best,
> holger
>> another detail that plays into that is certain plugins needing different
>> collections as well
>> an example is the doctest-module, which should scan for doctests in the
>> installed modules
>> that where imported, and not the ones in the working directory.
>> unfortunately its hard to find a good way to express those configured
>> roots for collection,
>> since the ini format on its own is rather limited, and i would like to
>> avoid introducing sub-languagess
>> -- Ronny
>> _______________________________________________
>> pytest-dev mailing list
>> pytest-dev at python.org
>> https://mail.python.org/mailman/listinfo/pytest-dev

More information about the pytest-dev mailing list