[pytest-dev] Configurable collection roots and reusable testsuites - Brainstorming

[Ronny Pfannschmidt]

Im aware of that rudimentary mechanism,
i would like to elevate a "testsuite" to a set of collected tests in
filled with "dependencies" via fixtures

that approach would also provide a more clean mechanism that are
currently handled with session scoped fixtures that are parametrized

one of such examples would be backends for common operations

i think its clear that a more detailed approach is needed,
a more detailed discussion about the model might be a good idea for a sprint


-- Ronny

Am 25.01.2016 um 11:30 schrieb holger krekel:
> Hi Ronny, Ionel, all,
>
> On Mon, Jan 25, 2016 at 00:46 +0100, Ronny Pfannschmidt wrote:
>> Hi everyone,
>>
>> a topic thats gripping since quite a while, is wanting to have reusable
>> testsuites,
>> so plugins/extensions that implement a behaviout,
>> can import and use integration tests from a installed package,
>> and use them in a different configuration
> That's already possible to a degree.
>
> If you hg-checkout https://bitbucket.org/hpk42/devpi and then look at
>
>     devpi/server/test_devpi_server/functional.py
>
> which is included from
>
>     devpi/server/test_devpi_server/test_views.py
>
> and also from the client side (from the installed devpi-server package):
>
>     devpi/client/testing/test_functional.py
>
> you'll note that the same tests run against different versions of a
> fixture, i.e. "mapp" which gives high level access to devpi behaviour.
> ASFAIK this pattern is also used by other projects.
>
> I was sometimes thinking that some way to tell pytest more dynamically
> what to collect, i.e. adding something to the test collection would
> be nice.  You can already do it by hooking into the plugin collection 
> hooks but that's a bit low-level as you need to tie into the collection
> tree carefully.  The hard part of "it would be nice" here is to write
> down a suggestion in forms of documentation on what API/semantics exactly 
> we want to add and then go for implementing it.
>
> best,
> holger
>
>
>> another detail that plays into that is certain plugins needing different
>> collections as well
>> an example is the doctest-module, which should scan for doctests in the
>> installed modules
>> that where imported, and not the ones in the working directory.
>>
>> unfortunately its hard to find a good way to express those configured
>> roots for collection,
>> since the ini format on its own is rather limited, and i would like to
>> avoid introducing sub-languagess
>>
>> -- Ronny
>> _______________________________________________
>> pytest-dev mailing list
>> pytest-dev at python.org
>> https://mail.python.org/mailman/listinfo/pytest-dev
>>