[py-dev] new resource API documentation comments

[holger krekel]

Hi Brianna,

On Thu, Aug 16, 2012 at 18:58 +1000, Brianna Laugher wrote:
> Hi,
> 
> I just spent some time reading the dev docs so these comments are just
> based on the docs and not actually using the new API. In general it
> looks pretty sensible.

thanks for your time and your feedback, I appreciate it!

> - being able to have funcargs like funcargs directly is really nice, a
> lot more obvious than calling request.getfuncargvalue('foo')

agreed :)

> - why addfinalizer and not teardown?

The old-style pytest_funcarg__NAME(request) api offered a 
request.addfinalizer() function already so i wanted to carry
on with this name.  I also considered addcleanup() similar to
the python unittest package and am open to other naming suggestions 
before the september release.

> -although I don't really know what cached_setup did, the trinity of
> defining the scope, setup and teardown methods made sense to me. Now
> the scope is in a decorator, the setup is implicitly the entire thing
> that is happening and the teardown seems somewhat awkwardly tacked on.

I see that point.  In principle we could have the factory decorator 
take a "teardown" parameter and it would receive the factory-created value.
I'd find it a bit awkward to advertise first naming a teardown before even
stating the setup code, though.  And it wouldn't help the asymetry you are 
observing.

> - none of the "addfinalizer" examples take an argument, how would you
> convert an old-style teardown method to that? e.g. we have a lot of
> funcargs which do things like
>     return request.cached_setup(setup=setup,
>                                 teardown=lambda obj: obj.close(),
>                                 scope='function')

In a factory-function you will create the value to return at some point
and then you can use it from the finalizer, like so:

    ...
    val = createval()
    def fin():
        uncreate(val)
    testcontext.addfinalizer(fin)
    
> - Sometimes things are referred to as "funcargs", sometimes they are
> referred to as "injected resources". Is there any difference here? The
> funcarg is the actual function and the injected resource is the
> instance in a specific test function? I suggest to use the term
> "funcarg" as much as possible as it is specific and a necessary
> concept for using pytest with any depth.

I agree.  In an early draft a radically different approach which used 
the term "resources" was considered.  Due to feedback similar to yours
I eventually went with using "funcargs" again.  A funcarg is what appears
in a test or setup function as an argument, it also _is_ a resource.  I am
open to reduce usages of "resource" if that is confusing.

> Some of the following comments are fairly picky so feel free to ignore them.
> 
> funcargs.txt
> line 118 - I think in this first incarnation of the smtp funcarg
> (factory? what to call it now?), it doesn't actually need to take a
> testcontext, right?

fixed.


> line 527 "Parametrizing test functions" - may be worth having a simple
> example showing a combination using both (test data) parametrization
> and funcarg parametrization, to emphasise how they are differently
> useful. Using a database as an example of funcarg parametrization is
> good, maybe better than values like 1/2. I feel like parametrizing
> tests (test data) is probably the more common use case and it is a
> little buried amongst the heavy duty parametrized funcargs.

I think with test function parametrization the parametrized funcargs
usually relate to the particular test and the parameters are simple
objects.  Objects such as Databases are more complex resources and
parametrization is then best defined at factory level.  Adding a "mixed"
example makes sense.

> line 598 "Basic ``pytest_generate_tests`` example" - I think this is
> not a very basic example! I think it is copied from parametrize.txt
> page, where it might make more sense. Here is what I would consider a
> basic example.
> 
> # code
> def isSquare(n):
>     n = n ** 0.5
>     return int(n) == n
> 
> # test file
> def pytest_generate_tests(metafunc):
>     squares = [1, 4, 9, 16, 25, 36, 49]
>     for n in range(1, 50):
>         expected = n in squares
>         if metafunc.function.__name__ == 'test_isSquare':
>             metafunc.addcall(id=n, funcargs=dict(n=n, expected=expected))
> 
> 
> def test_isSquare(n, expected):
>     assert isSquare(n) == expected
> 
> Well, this is so trivial you might bundle it all into a single test,
> but it can be useful to have each case genuinely be a separate test.
> You could also shoe-horn this data into a parametrize decorator I
> suppose but it is nicer to have more space if your test data is more
> complicated, to explain it.

OK, I consider adding this when i tackle another doc refactoring. thanks
for providing it.

> I am starting to have a feeling that the way my project has been using
> generate_tests is not the way everyone else uses it. In our
> conftest.py one of the enterprising developers on the project (who got
> us all onto py.test initially) put this:
> 
> def pytest_generate_tests(__multicall__, metafunc):
>     """Supports parametrised tests using generate_ fns.
>     Use multicall to call any other pytest_generate_tests hooks first.
>     If the test_ fn has a generate_ fn then call it with the metafunc
> to let it parametrise the test.
>     """
>     __multicall__.execute()
>     name = metafunc.function.__name__.replace('test_', 'generate_')
>     fn = getattr(metafunc.module, name, None)
>     if fn:
>         fn(metafunc)
> 
> this means I can simplify my example above to:
> 
> def generate_isSquare(metafunc):
>     squares = [1, 4, 9, 16, 25, 36, 49]
>     for n in range(1, 50):
>         expected = n in squares
>         metafunc.addcall(id=n, funcargs=dict(n=n, expected=expected))
> 
> maybe this is a useful pattern for others? or maybe we're doing it
> wrong, I dunno :)

It is a good example for implementing a custom scheme and making
the life of test writers (who do not need to care about the helper
in conftest.py) easier.  Personally, i'd prefer the decorator approach
but it all depends a bit on the exact use case.

> in funcarg_compare.txt
> line 53 - (in old-style) "4. there is no way how you can make use of
> funcarg factories in xUnit setup methods."
> 
> Is there an example of this? I would like to see it! This sounds like
> a way for people with existing xUnit style test suites to more easily
> convert to funcarg style.

There are some preliminary docs about it:

    http://pytest.org/dev/setup.html

The examples there use funcargs in setup functions just as you use
them in test function.  Or am i misunderstanding your question?
(you seem to know about the setup docs already, judging from below).
    
> Maybe with all the parametrized test examples it is a good idea to
> show the verbose output, just to be more explicit about what is being
> generated.
> 
> As an aside, there is a comment in example/markers.txt line 144 that
> "You can use the ``-k`` command line option to only run tests with
> names matching
> the given argument". Whether it is an oversight in the functionality
> or the docs, I have always found that -k also matches against markers
> (py.test.mark decorators). (Currently using pytest 2.2.3)

It was intended so the docs are imprecise there.  I take it you
also know about "-m", btw.

> in setup.txt
> line 65 'old = tmpdir.chdir()'
> - it's not really relevant to the example but I don't know what this
> does, and I couldn't find out, so it's just a bit distracting.
> 
> thanks as always for providing this great library :)

you are most welcome and thanks for your feedback!

best,

holger

> cheers
> Brianna
> 
> -- 
> They've just been waiting in a mountain for the right moment:
> http://modernthings.org/
> _______________________________________________
> py-dev mailing list
> py-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/py-dev
>