[py-svn] r65224 - in py/trunk: doc/test py py/execnet py/execnet/testing py/misc/testing py/test py/test/dist/testing py/test/plugin py/test/testing
hpk at codespeak.net
hpk at codespeak.net
Mon May 11 19:31:55 CEST 2009
Author: hpk
Date: Mon May 11 19:31:54 2009
New Revision: 65224
Added:
py/trunk/py/test/funcargs.py
Modified:
py/trunk/doc/test/config.txt
py/trunk/doc/test/funcargs.txt
py/trunk/doc/test/test.txt
py/trunk/py/_com.py
py/trunk/py/execnet/gateway.py
py/trunk/py/execnet/testing/test_gwmanage.py
py/trunk/py/misc/testing/test_com.py
py/trunk/py/test/dist/testing/test_nodemanage.py
py/trunk/py/test/plugin/api.py
py/trunk/py/test/plugin/pytest_recwarn.py
py/trunk/py/test/plugin/pytest_restdoc.py
py/trunk/py/test/plugin/pytest_terminal.py
py/trunk/py/test/plugin/pytest_tmpdir.py
py/trunk/py/test/pycollect.py
py/trunk/py/test/testing/test_funcargs.py
py/trunk/py/test/testing/test_pickling.py
py/trunk/py/test/testing/test_pycollect.py
Log:
merging 9e880f747f29 (trunk) tip
- new funcarg generators
- small fixes and cleanups
Modified: py/trunk/doc/test/config.txt
==============================================================================
--- py/trunk/doc/test/config.txt (original)
+++ py/trunk/doc/test/config.txt Mon May 11 19:31:54 2009
@@ -28,8 +28,10 @@
-------------------------------------------
``py.test`` runs provide means to create per-test session
-temporary (sub) directories. You can create such directories
-like this:
+temporary (sub) directories through the config object.
+You can create directories like this:
+
+.. XXX use a more local example, just with "config"
.. sourcecode: python
Modified: py/trunk/doc/test/funcargs.txt
==============================================================================
--- py/trunk/doc/test/funcargs.txt (original)
+++ py/trunk/doc/test/funcargs.txt Mon May 11 19:31:54 2009
@@ -1,52 +1,76 @@
======================================================
-**funcargs**: powerful and simple test setup
+**funcargs**: powerful test setup and parametrization
======================================================
-In version 1.0 py.test introduces a new mechanism for setting up test
-state for use by Python test functions. It is particularly useful
-for functional and integration testing but also for unit testing.
-Using funcargs you can easily:
-
-* write self-contained, simple to read and debug test functions
-* cleanly encapsulate glue code between your app and your tests
-* setup test state depending on command line options or environment
-
-Using the funcargs mechanism will increase readability
-and allow for easier refactoring of your application
-and its test suites.
+Since version 1.0 it is possible to provide arguments to test functions,
+often called "funcargs". The funcarg mechanisms were developed with
+these goals in mind:
+
+* **no boilerplate**: cleanly encapsulate test setup and fixtures
+* **flexibility**: easily setup test state depending on command line options or environment
+* **readability**: write simple to read and debug test functions
+* **parametrizing tests**: run a test function multiple times with different parameters
+
.. contents:: Contents:
:depth: 2
-The basic funcarg request/provide mechanism
+Basic mechanisms by example
=============================================
-To use funcargs you only need to specify
-a named argument for your test function:
+providing single function arguments as needed
+---------------------------------------------------------
+
+Let's look at a simple example of using funcargs within a test module:
.. sourcecode:: python
- def test_function(myarg):
- # use myarg
+ def pytest_funcarg__myfuncarg(request):
+ return 42
+
+ def test_function(myfuncarg):
+ assert myfuncarg == 42
+
+1. To setup the running of the ``test_function()`` call, py.test
+ looks up a provider for the ``myfuncarg`` argument.
+ The provider method is recognized by its ``pytest_funcarg__`` prefix
+ followed by the requested function argument name.
+ The `request object`_ gives access to test context.
-For each test function that requests this ``myarg``
-argument a matching so called funcarg provider
-will be invoked. A Funcarg provider for ``myarg``
-is written down liks this:
+2. A ``test_function(42)`` call is executed. If the test fails
+ one can see the original provided value.
+
+
+generating test runs with multiple function argument values
+----------------------------------------------------------------------
+
+You can parametrize multiple runs of a test function by
+providing multiple values for function arguments. Here
+is an example for running the same test function three times.
.. sourcecode:: python
- def pytest_funcarg__myarg(self, request):
- # return value for myarg here
+ def pytest_genfuncruns(runspec):
+ if "arg1" in runspec.funcargnames:
+ runspec.addfuncarg("arg1", 10)
+ runspec.addfuncarg("arg1", 20)
+ runspec.addfuncarg("arg1", 30)
+
+ def test_function(arg1):
+ assert myfuncarg in (10, 20, 30)
+
+Here is what happens:
-Such a provider method can live on a test class,
-test module or on a local or global plugin.
-The method is recognized by the ``pytest_funcarg__``
-prefix and is correlated to the argument
-name which follows this prefix. The passed in
-``request`` object allows to interact
-with test configuration, test collection
-and test running aspects.
+1. The ``pytest_genfuncruns()`` hook will be called once for each test
+ function. The if-statement makes sure that we only add function
+ arguments (and runs) for functions that need it. The `runspec object`_
+ provides access to context information.
+
+2. Subsequently the ``test_function()`` will be called three times
+ with three different values for ``arg1``.
+
+Funcarg rules and support objects
+====================================
.. _`request object`:
@@ -65,11 +89,13 @@
``request.function``: python function object requesting the argument
-``request.fspath``: filesystem path of containing module
+``request.cls``: class object where the test function is defined in or None.
+
+``runspec.module``: module object where the test function is defined in.
``request.config``: access to command line opts and general config
-finalizing after test function executed
+cleanup after test function execution
++++++++++++++++++++++++++++++++++++++++
Request objects allow to **register a finalizer method** which is
@@ -86,33 +112,8 @@
request.addfinalizer(lambda: myfile.close())
return myfile
-a unique temporary directory
-++++++++++++++++++++++++++++++++++++++++
-
-request objects allow to create unique temporary
-directories. These directories will be created
-as subdirectories under the `per-testsession
-temporary directory`_. Each request object
-receives its own unique subdirectory whose
-basenames starts with the name of the function
-that triggered the funcarg request. You
-can further work with the provided `py.path.local`_
-object to e.g. create subdirs or config files::
-
- def pytest_funcarg__mysetup(self, request):
- tmpdir = request.maketempdir()
- tmpdir.mkdir("mysubdir")
- tmpdir.join("config.ini").write("[default")
- return tmpdir
-
-Note that you do not need to perform finalization,
-i.e. remove the temporary directory as this is
-part of the global management of the base temporary
-directory.
-.. _`per-testsession temporary directory`: config.html#basetemp
-
-decorating/adding to existing funcargs
+decorating other funcarg providers
++++++++++++++++++++++++++++++++++++++++
If you want to **decorate a function argument** that is
@@ -131,33 +132,73 @@
for a use of this method.
-.. _`funcarg lookup order`:
-
-Order of funcarg provider lookup
-----------------------------------------
+.. _`lookup order`:
-For any funcarg argument request here is the
-lookup order for provider methods:
+Order of provider and test generator lookup
+----------------------------------------------
-1. test class (if we are executing a method)
-2. test module
-3. local plugins
-4. global plugins
+Both test generators as well as funcarg providers
+are looked up in the following three scopes:
+1. test module
+2. local plugins
+3. global plugins
Using multiple funcargs
----------------------------------------
-A test function may receive more than one
-function arguments. For each of the
-function arguments a lookup of a
-matching provider will be performed.
+Test functions can have multiple arguments
+which can either come from a test generator
+or from a provider.
+.. _`runspec object`:
+
+runspec objects
+------------------------
-Funcarg Tutorial Examples
-============================
+Runspecs help to inspect a testfunction and
+to generate tests with combinations of function argument values.
-tutorial example: the "test/app-specific" setup pattern
+generating and combining funcargs
++++++++++++++++++++++++++++++++++++++++++++++++++++
+
+Calling ``runspec.addfuncarg(argname, value)`` will trigger
+tests function calls with the given function
+argument value. For each already existing
+funcarg combination, the added funcarg value will
+
+* be merged to the existing funcarg combination if the
+ new argument name isn't part of the funcarg combination yet.
+
+* otherwise generate a new test call where the existing
+ funcarg combination is copied and updated
+ with the newly added funcarg value.
+
+For simple usage, e.g. test functions with a single
+generated function argument, each call to ``addfuncarg``
+will just trigger a new call.
+
+This scheme allows two sources to generate
+function arguments independently from each other.
+
+Attributes of runspec objects
+++++++++++++++++++++++++++++++++++++++++
+
+``runspec.funcargnames``: set of required function arguments for given function
+
+``runspec.function``: underlying python test function
+
+``runspec.cls``: class object where the test function is defined in or None.
+
+``runspec.module``: the module object where the test function is defined in.
+
+``runspec.config``: access to command line opts and general config
+
+
+Useful Funcarg Tutorial Examples
+=======================================
+
+application specific test setup
---------------------------------------------------------
Here is a basic useful step-wise example for handling application
@@ -202,7 +243,7 @@
return MyApp()
py.test finds the ``pytest_funcarg__mysetup`` method by
-name, see `funcarg lookup order`_ for more on this mechanism.
+name, see also `lookup order`_.
To run the example we put a pseudo MyApp object into ``myapp.py``:
@@ -265,29 +306,8 @@
conn = mysetup.getsshconnection()
# work with conn
-Running this without the command line will yield this run result::
-
- XXX fill in
-
-
-Example: specifying funcargs in test modules or classes
----------------------------------------------------------
-
-.. sourcecode:: python
-
- def pytest_funcarg__mysetup(request):
- result = request.call_next_provider()
- result.extra = "..."
- return result
-
-You can put such a function into a test class like this:
-
-.. sourcecode:: python
-
- class TestClass:
- def pytest_funcarg__mysetup(self, request):
- # ...
- #
+Running this without specifying a command line option will result in a skipped
+test_function.
.. _`accept example`:
@@ -309,16 +329,12 @@
def __init__(self, request):
if not request.config.option.acceptance:
py.test.skip("specify -A to run acceptance tests")
- self.tmpdir = request.config.maketempdir(request.argname)
- self._old = self.tmpdir.chdir()
- request.addfinalizer(self.finalize)
-
- def run(self):
- return py.process.cmdexec("echo hello")
-
- def finalize(self):
- self._old.chdir()
- # cleanup any other resources
+ self.tmpdir = request.config.mktemp(request.function.__name__, numbered=True)
+
+ def run(self, cmd):
+ """ called by test code to execute an acceptance test. """
+ self.tmpdir.chdir()
+ return py.process.cmdexec(cmd)
and the actual test function example:
@@ -327,48 +343,116 @@
def test_some_acceptance_aspect(accept):
accept.tmpdir.mkdir("somesub")
- result = accept.run()
- assert result
+ result = accept.run("ls -la")
+ assert "somesub" in result
-That's it! This test will get automatically skipped with
-an appropriate message if you just run ``py.test``::
-
- ... OUTPUT of py.test on this example ...
-
+If you run this test without specifying a command line option
+the test will get skipped with an appropriate message. Otherwise
+you can start to add convenience and test support methods
+to your AcceptFuncarg and drive running of tools or
+applications and provide ways to do assertions about
+the output.
.. _`decorator example`:
-example: decorating/extending a funcarg in a TestClass
+example: decorating a funcarg in a test module
--------------------------------------------------------------
For larger scale setups it's sometimes useful to decorare
-a funcarg just for a particular test module or even
-a particular test class. We can extend the `accept example`_
-by putting this in our test class:
+a funcarg just for a particular test module. We can
+extend the `accept example`_ by putting this in our test class:
.. sourcecode:: python
- class TestSpecialAcceptance:
- def pytest_funcarg__accept(self, request):
- arg = request.call_next_provider()
- # create a special layout in our tempdir
- arg.tmpdir.mkdir("special")
- return arg
+ def pytest_funcarg__accept(self, request):
+ arg = request.call_next_provider()
+ # create a special layout in our tempdir
+ arg.tmpdir.mkdir("special")
+ return arg
+ class TestSpecialAcceptance:
def test_sometest(self, accept):
assert accept.tmpdir.join("special").check()
-According to the `funcarg lookup order`_ our class-specific provider will
-be invoked first. Here, we just ask our request object to
-call the next provider and decorate its result. This simple
+According to the the `lookup order`_ our module level provider
+will be invoked first and it can ask ask its request object to
+call the next provider and then decorate its result. This
mechanism allows us to stay ignorant of how/where the
function argument is provided.
-Note that we make use here of `py.path.local`_ objects
-that provide uniform access to the local filesystem.
+sidenote: the temporary directory used here are instances of
+the `py.path.local`_ class which provides many of the os.path
+methods in a convenient way.
.. _`py.path.local`: ../path.html#local
+.. _`combine multiple funcarg values`:
+
+
+parametrize test functions by combining generated funcargs
+--------------------------------------------------------------------------
+
+Adding different funcargs will generate test calls with
+all combinations of added funcargs. Consider this example:
+
+.. sourcecode:: python
+
+ def makearg1(runspec):
+ runspec.addfuncarg("arg1", 10)
+ runspec.addfuncarg("arg1", 11)
+
+ def makearg2(runspec):
+ runspec.addfuncarg("arg2", 20)
+ runspec.addfuncarg("arg2", 21)
+
+ def pytest_genfuncruns(runspec):
+ makearg1(runspec)
+ makearg2(runspec)
+
+ # the actual test function
+
+ def test_function(arg1, arg2):
+ assert arg1 in (10, 20)
+ assert arg2 in (20, 30)
+
+Running this test module will result in ``test_function``
+being called four times, in the following order::
+
+ test_function(10, 20)
+ test_function(10, 21)
+ test_function(11, 20)
+ test_function(11, 21)
+
+
+example: test functions with generated and provided funcargs
+-------------------------------------------------------------------
+
+You can mix generated function arguments and normally
+provided ones. Consider this module:
+
+.. sourcecode:: python
+
+ def pytest_genfuncruns(runspec):
+ if "arg1" in runspec.funcargnames: # test_function2 does not have it
+ runspec.addfuncarg("arg1", 10)
+ runspec.addfuncarg("arg1", 20)
+
+ def pytest_funcarg__arg2(request):
+ return [10, 20]
+
+ def test_function(arg1, arg2):
+ assert arg1 in arg2
+
+ def test_function2(arg2):
+ assert args2 == [10, 20]
+
+Running this test module will result in ``test_function``
+being called twice, with these arguments::
+
+ test_function(10, [10, 20])
+ test_function(20, [10, 20])
+
+
Questions and Answers
==================================
@@ -377,14 +461,14 @@
Why ``pytest_funcarg__*`` methods?
------------------------------------
-When experimenting with funcargs we also considered an explicit
-registration mechanism, i.e. calling a register method e.g. on the
-config object. But lacking a good use case for this indirection and
-flexibility we decided to go for `Convention over Configuration`_
-and allow to directly specify the provider. It has the
-positive implication that you should be able to
-"grep" for ``pytest_funcarg__MYARG`` and will find all
-providing sites (usually exactly one).
+When experimenting with funcargs we also
+considered an explicit registration mechanism, i.e. calling a register
+method on the config object. But lacking a good use case for this
+indirection and flexibility we decided to go for `Convention over
+Configuration`_ and allow to directly specify the provider. It has the
+positive implication that you should be able to "grep" for
+``pytest_funcarg__MYARG`` and will find all providing sites (usually
+exactly one).
.. _`Convention over Configuration`: http://en.wikipedia.org/wiki/Convention_over_Configuration
Modified: py/trunk/doc/test/test.txt
==============================================================================
--- py/trunk/doc/test/test.txt (original)
+++ py/trunk/doc/test/test.txt Mon May 11 19:31:54 2009
@@ -11,14 +11,18 @@
features_: a walk through basic features and usage.
+funcargs_: powerful parametrized test function setup
+
+`distributed testing`_: distribute test runs to other machines and platforms.
+
plugins_: using available plugins.
extend_: writing plugins and advanced configuration.
-`distributed testing`_ how to distribute test runs to other machines and platforms.
.. _quickstart: quickstart.html
.. _features: features.html
+.. _funcargs: funcargs.html
.. _plugins: plugins.html
.. _extend: ext.html
.. _`distributed testing`: dist.html
Modified: py/trunk/py/_com.py
==============================================================================
--- py/trunk/py/_com.py (original)
+++ py/trunk/py/_com.py Mon May 11 19:31:54 2009
@@ -123,10 +123,14 @@
return "<Hooks %r %r>" %(self._hookspecs, self._plugins)
class HookCall:
- def __init__(self, registry, name, firstresult):
+ def __init__(self, registry, name, firstresult, extralookup=None):
self.registry = registry
self.name = name
self.firstresult = firstresult
+ self.extralookup = extralookup and [extralookup] or ()
+
+ def clone(self, extralookup):
+ return HookCall(self.registry, self.name, self.firstresult, extralookup)
def __repr__(self):
mode = self.firstresult and "firstresult" or "each"
@@ -136,7 +140,8 @@
if args:
raise TypeError("only keyword arguments allowed "
"for api call to %r" % self.name)
- mc = MultiCall(self.registry.listattr(self.name), **kwargs)
+ attr = self.registry.listattr(self.name, extra=self.extralookup)
+ mc = MultiCall(attr, **kwargs)
return mc.execute(firstresult=self.firstresult)
comregistry = Registry()
Modified: py/trunk/py/execnet/gateway.py
==============================================================================
--- py/trunk/py/execnet/gateway.py (original)
+++ py/trunk/py/execnet/gateway.py Mon May 11 19:31:54 2009
@@ -167,7 +167,8 @@
except IOError:
self._trace('IOError on _stopsend()')
self._channelfactory._finished_receiving()
- self._trace('leaving %r' % threading.currentThread())
+ if threading: # might be None during shutdown/finalization
+ self._trace('leaving %r' % threading.currentThread())
from sys import exc_info
def _send(self, msg):
Modified: py/trunk/py/execnet/testing/test_gwmanage.py
==============================================================================
--- py/trunk/py/execnet/testing/test_gwmanage.py (original)
+++ py/trunk/py/execnet/testing/test_gwmanage.py Mon May 11 19:31:54 2009
@@ -114,7 +114,7 @@
class pytest_funcarg__mysetup:
def __init__(self, request):
- tmp = request.maketempdir()
+ tmp = request.config.mktemp(request.function.__name__, numbered=True)
self.source = tmp.mkdir("source")
self.dest = tmp.mkdir("dest")
Modified: py/trunk/py/misc/testing/test_com.py
==============================================================================
--- py/trunk/py/misc/testing/test_com.py (original)
+++ py/trunk/py/misc/testing/test_com.py Mon May 11 19:31:54 2009
@@ -190,3 +190,23 @@
class Api: pass
mcm = Hooks(hookspecs=Api)
assert mcm.registry == py._com.comregistry
+
+ def test_hooks_extra_plugins(self):
+ registry = Registry()
+ class Api:
+ def hello(self, arg):
+ pass
+ hook_hello = Hooks(hookspecs=Api, registry=registry).hello
+ class Plugin:
+ def hello(self, arg):
+ return arg + 1
+ registry.register(Plugin())
+ class Plugin2:
+ def hello(self, arg):
+ return arg + 2
+ newhook = hook_hello.clone(extralookup=Plugin2())
+ l = newhook(arg=3)
+ assert l == [5, 4]
+ l2 = hook_hello(arg=3)
+ assert l2 == [4]
+
Modified: py/trunk/py/test/dist/testing/test_nodemanage.py
==============================================================================
--- py/trunk/py/test/dist/testing/test_nodemanage.py (original)
+++ py/trunk/py/test/dist/testing/test_nodemanage.py Mon May 11 19:31:54 2009
@@ -3,8 +3,9 @@
class pytest_funcarg__mysetup:
def __init__(self, request):
- basetemp = request.maketempdir()
- basetemp = basetemp.mkdir(request.function.__name__)
+ basetemp = request.config.mktemp(
+ "mysetup:%s" % request.function.__name__,
+ numbered=True)
self.source = basetemp.mkdir("source")
self.dest = basetemp.mkdir("dest")
Added: py/trunk/py/test/funcargs.py
==============================================================================
--- (empty file)
+++ py/trunk/py/test/funcargs.py Mon May 11 19:31:54 2009
@@ -0,0 +1,115 @@
+import py
+
+def getfuncargnames(function):
+ argnames = py.std.inspect.getargs(function.func_code)[0]
+ startindex = hasattr(function, 'im_self') and 1 or 0
+ numdefaults = len(function.func_defaults or ())
+ if numdefaults:
+ return argnames[startindex:-numdefaults]
+ return argnames[startindex:]
+
+def fillfuncargs(function):
+ """ fill missing funcargs. """
+ if function._args:
+ # functions yielded from a generator: we don't want
+ # to support that because we want to go here anyway:
+ # http://bitbucket.org/hpk42/py-trunk/issue/2/next-generation-generative-tests
+ pass
+ else:
+ # standard Python Test function/method case
+ for argname in getfuncargnames(function.obj):
+ if argname not in function.funcargs:
+ request = FuncargRequest(pyfuncitem=function, argname=argname)
+ try:
+ function.funcargs[argname] = request.call_next_provider()
+ except request.Error:
+ request._raiselookupfailed()
+
+class RunSpecs:
+ def __init__(self, function, config=None, cls=None, module=None):
+ self.config = config
+ self.module = module
+ self.function = function
+ self.funcargnames = getfuncargnames(function)
+ self.cls = cls
+ self.module = module
+ self._combinations = []
+
+ def addfuncarg(self, argname, value):
+ if argname not in self.funcargnames:
+ raise ValueError("function %r has no funcarg %r" %(
+ self.function, argname))
+ newcombi = []
+ if not self._combinations:
+ newcombi.append({argname:value})
+ else:
+ for combi in self._combinations:
+ if argname in combi:
+ combi = combi.copy()
+ newcombi.append(combi)
+ combi[argname] = value
+ self._combinations.extend(newcombi)
+
+class FunctionCollector(py.test.collect.Collector):
+ def __init__(self, name, parent, combinations):
+ super(FunctionCollector, self).__init__(name, parent)
+ self.combinations = combinations
+ self.obj = getattr(self.parent.obj, name)
+
+ def collect(self):
+ l = []
+ for i, funcargs in py.builtin.enumerate(self.combinations):
+ function = self.parent.Function(name="%s[%s]" %(self.name, i),
+ parent=self, funcargs=funcargs, callobj=self.obj)
+ l.append(function)
+ return l
+
+class FuncargRequest:
+ _argprefix = "pytest_funcarg__"
+
+ class Error(LookupError):
+ """ error on performing funcarg request. """
+
+ def __init__(self, pyfuncitem, argname):
+ self._pyfuncitem = pyfuncitem
+ self.argname = argname
+ self.function = pyfuncitem.obj
+ self.module = pyfuncitem.getmodulecollector().obj
+ self.cls = getattr(self.function, 'im_class', None)
+ self.config = pyfuncitem.config
+ self.fspath = pyfuncitem.fspath
+ self._plugins = self.config.pluginmanager.getplugins()
+ self._plugins.append(pyfuncitem.getmodulecollector().obj)
+ self._provider = self.config.pluginmanager.listattr(
+ plugins=self._plugins,
+ attrname=self._argprefix + str(argname)
+ )
+
+ def __repr__(self):
+ return "<FuncargRequest %r for %r>" %(self.argname, self._pyfuncitem)
+
+ def call_next_provider(self):
+ if not self._provider:
+ raise self.Error("no provider methods left")
+ next_provider = self._provider.pop()
+ return next_provider(request=self)
+
+ def addfinalizer(self, finalizer):
+ self._pyfuncitem.addfinalizer(finalizer)
+
+ def _raiselookupfailed(self):
+ available = []
+ for plugin in self._plugins:
+ for name in vars(plugin.__class__):
+ if name.startswith(self._argprefix):
+ name = name[len(self._argprefix):]
+ if name not in available:
+ available.append(name)
+ fspath, lineno, msg = self._pyfuncitem.metainfo()
+ line = "%s:%s" %(fspath, lineno)
+ msg = "funcargument %r not found for: %s" %(self.argname, line)
+ msg += "\n available funcargs: %s" %(", ".join(available),)
+ raise LookupError(msg)
+
+
+
Modified: py/trunk/py/test/plugin/api.py
==============================================================================
--- py/trunk/py/test/plugin/api.py (original)
+++ py/trunk/py/test/plugin/api.py Mon May 11 19:31:54 2009
@@ -53,13 +53,15 @@
""" return custom item/collector for a python object in a module, or None. """
pytest_pycollect_obj.firstresult = True
+ def pytest_genfuncruns(self, runspec):
+ """ generate (multiple) parametrized calls to a test function."""
+
def pytest_collectstart(self, collector):
""" collector starts collecting. """
def pytest_collectreport(self, rep):
""" collector finished collecting. """
-
# ------------------------------------------------------------------------------
# runtest related hooks
# ------------------------------------------------------------------------------
Modified: py/trunk/py/test/plugin/pytest_recwarn.py
==============================================================================
--- py/trunk/py/test/plugin/pytest_recwarn.py (original)
+++ py/trunk/py/test/plugin/pytest_recwarn.py Mon May 11 19:31:54 2009
@@ -1,8 +1,7 @@
"""
-"recwarn" funcarg plugin that helps to assert
-that warnings are shown to a user. See the test
-at the bottom for an example.
+provides "recwarn" funcarg for asserting warnings to be shown
+to a user. See the test at the bottom for an example.
"""
import py
Modified: py/trunk/py/test/plugin/pytest_restdoc.py
==============================================================================
--- py/trunk/py/test/plugin/pytest_restdoc.py (original)
+++ py/trunk/py/test/plugin/pytest_restdoc.py Mon May 11 19:31:54 2009
@@ -142,7 +142,7 @@
directives.register_directive('sourcecode', pygments_directive)
def resolve_linkrole(self, name, text, check=True):
- apigen_relpath = self.project.hookgen_relpath
+ apigen_relpath = self.project.apigen_relpath
if name == 'api':
if text == 'py':
Modified: py/trunk/py/test/plugin/pytest_terminal.py
==============================================================================
--- py/trunk/py/test/plugin/pytest_terminal.py (original)
+++ py/trunk/py/test/plugin/pytest_terminal.py Mon May 11 19:31:54 2009
@@ -207,9 +207,10 @@
msg += " -- " + str(sys.executable)
self.write_line(msg)
- rev = py.__pkg__.getrev()
- self.write_line("using py lib: %s <rev %s>" % (
- py.path.local(py.__file__).dirpath(), rev))
+ if self.config.option.debug or self.config.option.traceconfig:
+ rev = py.__pkg__.getrev()
+ self.write_line("using py lib: %s <rev %s>" % (
+ py.path.local(py.__file__).dirpath(), rev))
if self.config.option.traceconfig:
plugins = []
for plugin in self.config.pluginmanager.comregistry:
Modified: py/trunk/py/test/plugin/pytest_tmpdir.py
==============================================================================
--- py/trunk/py/test/plugin/pytest_tmpdir.py (original)
+++ py/trunk/py/test/plugin/pytest_tmpdir.py Mon May 11 19:31:54 2009
@@ -27,9 +27,10 @@
plugintester.hookcheck(TmpdirPlugin)
def test_funcarg(testdir):
+ from py.__.test.funcargs import FuncargRequest
item = testdir.getitem("def test_func(tmpdir): pass")
plugin = TmpdirPlugin()
- p = plugin.pytest_funcarg__tmpdir(item.getrequest("tmpdir"))
+ p = plugin.pytest_funcarg__tmpdir(FuncargRequest(item, "tmpdir"))
assert p.check()
bn = p.basename.strip("0123456789-")
assert bn.endswith("test_func")
Modified: py/trunk/py/test/pycollect.py
==============================================================================
--- py/trunk/py/test/pycollect.py (original)
+++ py/trunk/py/test/pycollect.py Mon May 11 19:31:54 2009
@@ -20,6 +20,7 @@
from py.__.test.collect import configproperty, warnoldcollect
from py.__.code.source import findsource
pydir = py.path.local(py.__file__).dirpath()
+from py.__.test import funcargs
class PyobjMixin(object):
def obj():
@@ -37,6 +38,16 @@
def _getobj(self):
return getattr(self.parent.obj, self.name)
+ def getmodulecollector(self):
+ return self._getparent(Module)
+ def getclasscollector(self):
+ return self._getparent(Class)
+ def _getparent(self, cls):
+ current = self
+ while current and not isinstance(current, cls):
+ current = current.parent
+ return current
+
def getmodpath(self, stopatmodule=True, includemodule=False):
""" return python path relative to the containing module. """
chain = self.listchain()
@@ -150,10 +161,25 @@
if res is not None:
return res
if obj.func_code.co_flags & 32: # generator function
+ # XXX deprecation warning
return self.Generator(name, parent=self)
else:
- return self.Function(name, parent=self)
+ return self._genfunctions(name, obj)
+ def _genfunctions(self, name, funcobj):
+ module = self.getmodulecollector().obj
+ # due to _buildname2items funcobj is the raw function, we need
+ # to work to get at the class
+ clscol = self.getclasscollector()
+ cls = clscol and clscol.obj or None
+ runspec = funcargs.RunSpecs(funcobj, config=self.config, cls=cls, module=module)
+ gentesthook = self.config.hook.pytest_genfuncruns.clone(extralookup=module)
+ gentesthook(runspec=runspec)
+ if not runspec._combinations:
+ return self.Function(name, parent=self)
+ return funcargs.FunctionCollector(name=name,
+ parent=self, combinations=runspec._combinations)
+
class Module(py.test.collect.File, PyCollectorMixin):
def _getobj(self):
return self._memoizedcall('_obj', self._importtestmodule)
@@ -320,11 +346,13 @@
""" a Function Item is responsible for setting up
and executing a Python callable test object.
"""
- def __init__(self, name, parent=None, config=None, args=(), callobj=_dummy):
+ def __init__(self, name, parent=None, config=None, args=(), funcargs=None, callobj=_dummy):
super(Function, self).__init__(name, parent, config=config)
self._finalizers = []
self._args = args
- self.funcargs = {}
+ if funcargs is None:
+ funcargs = {}
+ self.funcargs = funcargs
if callobj is not _dummy:
self._obj = callobj
@@ -350,31 +378,7 @@
def setup(self):
super(Function, self).setup()
- self._setupfuncargs()
-
- def _setupfuncargs(self):
- if self._args:
- # functions yielded from a generator: we don't want
- # to support that because we want to go here anyway:
- # http://bitbucket.org/hpk42/py-trunk/issue/2/next-generation-generative-tests
- pass
- else:
- # standard Python Test function/method case
- funcobj = self.obj
- startindex = getattr(funcobj, 'im_self', None) and 1 or 0
- argnames = py.std.inspect.getargs(self.obj.func_code)[0]
- for i, argname in py.builtin.enumerate(argnames):
- if i < startindex:
- continue
- request = self.getrequest(argname)
- try:
- self.funcargs[argname] = request.call_next_provider()
- except request.Error:
- numdefaults = len(funcobj.func_defaults or ())
- if i + numdefaults >= len(argnames):
- continue # our args have defaults XXX issue warning?
- else:
- request._raiselookupfailed()
+ funcargs.fillfuncargs(self)
def __eq__(self, other):
try:
@@ -385,74 +389,7 @@
except AttributeError:
pass
return False
+
def __ne__(self, other):
return not self == other
- def getrequest(self, argname):
- return FuncargRequest(pyfuncitem=self, argname=argname)
-
-
-class FuncargRequest:
- _argprefix = "pytest_funcarg__"
-
- class Error(LookupError):
- """ error on performing funcarg request. """
-
- def __init__(self, pyfuncitem, argname):
- # XXX make pyfuncitem _pyfuncitem
- self._pyfuncitem = pyfuncitem
- self.argname = argname
- self.function = pyfuncitem.obj
- self.config = pyfuncitem.config
- self.fspath = pyfuncitem.fspath
- self._plugins = self._getplugins()
- self._methods = self.config.pluginmanager.listattr(
- plugins=self._plugins,
- attrname=self._argprefix + str(argname)
- )
-
- def __repr__(self):
- return "<FuncargRequest %r for %r>" %(self.argname, self._pyfuncitem)
-
-
- def _getplugins(self):
- plugins = []
- current = self._pyfuncitem
- while not isinstance(current, Module):
- current = current.parent
- if isinstance(current, (Instance, Module)):
- plugins.insert(0, current.obj)
- return self.config.pluginmanager.getplugins() + plugins
-
- def call_next_provider(self):
- if not self._methods:
- raise self.Error("no provider methods left")
- nextmethod = self._methods.pop()
- return nextmethod(request=self)
-
- def addfinalizer(self, finalizer):
- self._pyfuncitem.addfinalizer(finalizer)
-
- def maketempdir(self):
- basetemp = self.config.getbasetemp()
- tmp = py.path.local.make_numbered_dir(
- prefix=self.function.__name__ + "_",
- keep=0, rootdir=basetemp)
- return tmp
-
- def _raiselookupfailed(self):
- available = []
- for plugin in self._plugins:
- for name in vars(plugin.__class__):
- if name.startswith(self._argprefix):
- name = name[len(self._argprefix):]
- if name not in available:
- available.append(name)
- fspath, lineno, msg = self._pyfuncitem.metainfo()
- line = "%s:%s" %(fspath, lineno)
- msg = "funcargument %r not found for: %s" %(self.argname, line)
- msg += "\n available funcargs: %s" %(", ".join(available),)
- raise LookupError(msg)
-
-
-
Modified: py/trunk/py/test/testing/test_funcargs.py
==============================================================================
--- py/trunk/py/test/testing/test_funcargs.py (original)
+++ py/trunk/py/test/testing/test_funcargs.py Mon May 11 19:31:54 2009
@@ -1,6 +1,22 @@
import py
+from py.__.test import funcargs
-class TestFuncargs:
+def test_getfuncargnames():
+ def f(): pass
+ assert not funcargs.getfuncargnames(f)
+ def g(arg): pass
+ assert funcargs.getfuncargnames(g) == ['arg']
+ def h(arg1, arg2="hello"): pass
+ assert funcargs.getfuncargnames(h) == ['arg1']
+ def h(arg1, arg2, arg3="hello"): pass
+ assert funcargs.getfuncargnames(h) == ['arg1', 'arg2']
+ class A:
+ def f(self, arg1, arg2="hello"):
+ pass
+ assert funcargs.getfuncargnames(A().f) == ['arg1']
+ assert funcargs.getfuncargnames(A.f) == ['arg1']
+
+class TestFillFuncArgs:
def test_funcarg_lookupfails(self, testdir):
testdir.makeconftest("""
class ConftestPlugin:
@@ -8,7 +24,7 @@
return 42
""")
item = testdir.getitem("def test_func(some): pass")
- exc = py.test.raises(LookupError, "item._setupfuncargs()")
+ exc = py.test.raises(LookupError, "funcargs.fillfuncargs(item)")
s = str(exc.value)
assert s.find("xyzsomething") != -1
@@ -18,20 +34,8 @@
def pytest_funcarg__some(self, request):
return request.function.__name__
item.config.pluginmanager.register(Provider())
- item._setupfuncargs()
- assert len(item.funcargs) == 1
-
- def test_funcarg_lookup_default_gets_overriden(self, testdir):
- item = testdir.getitem("def test_func(some=42, other=13): pass")
- class Provider:
- def pytest_funcarg__other(self, request):
- return request.function.__name__
- item.config.pluginmanager.register(Provider())
- item._setupfuncargs()
+ funcargs.fillfuncargs(item)
assert len(item.funcargs) == 1
- name, value = item.funcargs.popitem()
- assert name == "other"
- assert value == item.name
def test_funcarg_basic(self, testdir):
item = testdir.getitem("def test_func(some, other): pass")
@@ -41,7 +45,7 @@
def pytest_funcarg__other(self, request):
return 42
item.config.pluginmanager.register(Provider())
- item._setupfuncargs()
+ funcargs.fillfuncargs(item)
assert len(item.funcargs) == 2
assert item.funcargs['some'] == "test_func"
assert item.funcargs['other'] == 42
@@ -58,9 +62,9 @@
pass
""")
item1, item2 = testdir.genitems([modcol])
- item1._setupfuncargs()
+ funcargs.fillfuncargs(item1)
assert item1.funcargs['something'] == "test_method"
- item2._setupfuncargs()
+ funcargs.fillfuncargs(item2)
assert item2.funcargs['something'] == "test_func"
class TestRequest:
@@ -69,37 +73,44 @@
def pytest_funcarg__something(request): pass
def test_func(something): pass
""")
- req = item.getrequest("other")
+ req = funcargs.FuncargRequest(item, argname="other")
assert req.argname == "other"
assert req.function == item.obj
+ assert hasattr(req.module, 'test_func')
+ assert req.cls is None
assert req.function.__name__ == "test_func"
assert req.config == item.config
assert repr(req).find(req.function.__name__) != -1
+
+ def test_request_attributes_method(self, testdir):
+ item, = testdir.getitems("""
+ class TestB:
+ def test_func(self, something):
+ pass
+ """)
+ req = funcargs.FuncargRequest(item, argname="something")
+ assert req.cls.__name__ == "TestB"
- def test_request_contains_funcargs_methods(self, testdir):
+ def test_request_contains_funcargs_provider(self, testdir):
modcol = testdir.getmodulecol("""
def pytest_funcarg__something(request):
pass
class TestClass:
- def pytest_funcarg__something(self, request):
- pass
def test_method(self, something):
pass
""")
item1, = testdir.genitems([modcol])
assert item1.name == "test_method"
- methods = item1.getrequest("something")._methods
- assert len(methods) == 2
- method1, method2 = methods
- assert not hasattr(method1, 'im_self')
- assert method2.im_self is not None
+ provider = funcargs.FuncargRequest(item1, "something")._provider
+ assert len(provider) == 1
+ assert provider[0].__name__ == "pytest_funcarg__something"
def test_request_call_next_provider(self, testdir):
item = testdir.getitem("""
def pytest_funcarg__something(request): pass
def test_func(something): pass
""")
- req = item.getrequest("something")
+ req = funcargs.FuncargRequest(item, "something")
val = req.call_next_provider()
assert val is None
py.test.raises(req.Error, "req.call_next_provider()")
@@ -109,22 +120,147 @@
def pytest_funcarg__something(request): pass
def test_func(something): pass
""")
- req = item.getrequest("something")
+ req = funcargs.FuncargRequest(item, "something")
l = [1]
req.addfinalizer(l.pop)
item.teardown()
- def test_request_maketemp(self, testdir):
- item = testdir.getitem("def test_func(): pass")
- req = item.getrequest("xxx")
- tmpdir = req.maketempdir()
- tmpdir2 = req.maketempdir()
- assert tmpdir != tmpdir2
- assert tmpdir.basename.startswith("test_func")
- assert tmpdir2.basename.startswith("test_func")
-
def test_request_getmodulepath(self, testdir):
modcol = testdir.getmodulecol("def test_somefunc(): pass")
item, = testdir.genitems([modcol])
- req = item.getrequest("hello")
+ req = funcargs.FuncargRequest(item, "xxx")
assert req.fspath == modcol.fspath
+
+class TestRunSpecs:
+ def test_no_funcargs(self, testdir):
+ def function(): pass
+ runspec = funcargs.RunSpecs(function)
+ assert not runspec.funcargnames
+
+ def test_function_basic(self):
+ def func(arg1, arg2="qwe"): pass
+ runspec = funcargs.RunSpecs(func)
+ assert len(runspec.funcargnames) == 1
+ assert 'arg1' in runspec.funcargnames
+ assert runspec.function is func
+ assert runspec.cls is None
+
+ def test_addfuncarg_basic(self):
+ def func(arg1): pass
+ runspec = funcargs.RunSpecs(func)
+ py.test.raises(ValueError, """
+ runspec.addfuncarg("notexists", 100)
+ """)
+ runspec.addfuncarg("arg1", 100)
+ assert len(runspec._combinations) == 1
+ assert runspec._combinations[0] == {'arg1': 100}
+
+ def test_addfuncarg_two(self):
+ def func(arg1): pass
+ runspec = funcargs.RunSpecs(func)
+ runspec.addfuncarg("arg1", 100)
+ runspec.addfuncarg("arg1", 101)
+ assert len(runspec._combinations) == 2
+ assert runspec._combinations[0] == {'arg1': 100}
+ assert runspec._combinations[1] == {'arg1': 101}
+
+ def test_addfuncarg_combined(self):
+ runspec = funcargs.RunSpecs(lambda arg1, arg2: 0)
+ runspec.addfuncarg('arg1', 1)
+ runspec.addfuncarg('arg1', 2)
+ runspec.addfuncarg('arg2', 100)
+ combinations = runspec._combinations
+ assert len(combinations) == 2
+ assert combinations[0] == {'arg1': 1, 'arg2': 100}
+ assert combinations[1] == {'arg1': 2, 'arg2': 100}
+ runspec.addfuncarg('arg2', 101)
+ assert len(combinations) == 4
+ assert combinations[-1] == {'arg1': 2, 'arg2': 101}
+
+class TestGenfuncFunctional:
+ def test_attributes(self, testdir):
+ p = testdir.makepyfile("""
+ import py
+ def pytest_genfuncruns(runspec):
+ runspec.addfuncarg("runspec", runspec)
+
+ def test_function(runspec):
+ assert runspec.config == py.test.config
+ assert runspec.module.__name__ == __name__
+ assert runspec.function == test_function
+ assert runspec.cls is None
+ class TestClass:
+ def test_method(self, runspec):
+ assert runspec.config == py.test.config
+ assert runspec.module.__name__ == __name__
+ # XXX actually have the unbound test function here?
+ assert runspec.function == TestClass.test_method.im_func
+ assert runspec.cls == TestClass
+ """)
+ result = testdir.runpytest(p, "-v")
+ result.stdout.fnmatch_lines([
+ "*2 passed in*",
+ ])
+
+ def test_arg_twice(self, testdir):
+ testdir.makeconftest("""
+ class ConftestPlugin:
+ def pytest_genfuncruns(self, runspec):
+ assert "arg" in runspec.funcargnames
+ runspec.addfuncarg("arg", 10)
+ runspec.addfuncarg("arg", 20)
+ """)
+ p = testdir.makepyfile("""
+ def test_myfunc(arg):
+ assert arg == 10
+ """)
+ result = testdir.runpytest("-v", p)
+ assert result.stdout.fnmatch_lines([
+ "*test_myfunc*PASS*", # case for 10
+ "*test_myfunc*FAIL*", # case for 20
+ "*1 failed, 1 passed*"
+ ])
+
+ def test_two_functions(self, testdir):
+ p = testdir.makepyfile("""
+ def pytest_genfuncruns(runspec):
+ runspec.addfuncarg("arg1", 10)
+ runspec.addfuncarg("arg1", 20)
+
+ def test_func1(arg1):
+ assert arg1 == 10
+ def test_func2(arg1):
+ assert arg1 in (10, 20)
+ """)
+ result = testdir.runpytest("-v", p)
+ assert result.stdout.fnmatch_lines([
+ "*test_func1*0*PASS*",
+ "*test_func1*1*FAIL*",
+ "*test_func2*PASS*",
+ "*1 failed, 3 passed*"
+ ])
+
+ def test_genfuncarg_inmodule(self, testdir):
+ testdir.makeconftest("""
+ class ConftestPlugin:
+ def pytest_genfuncruns(self, runspec):
+ assert "arg" in runspec.funcargnames
+ runspec.addfuncarg("arg", 10)
+ """)
+ p = testdir.makepyfile("""
+ def pytest_genfuncruns(runspec):
+ runspec.addfuncarg("arg2", 10)
+ runspec.addfuncarg("arg2", 20)
+ runspec.addfuncarg("classarg", 17)
+
+ class TestClass:
+ def test_myfunc(self, arg, arg2, classarg):
+ assert classarg == 17
+ assert arg == arg2
+ """)
+ result = testdir.runpytest("-v", p)
+ assert result.stdout.fnmatch_lines([
+ "*test_myfunc*0*PASS*",
+ "*test_myfunc*1*FAIL*",
+ "*1 failed, 1 passed*"
+ ])
Modified: py/trunk/py/test/testing/test_pickling.py
==============================================================================
--- py/trunk/py/test/testing/test_pickling.py (original)
+++ py/trunk/py/test/testing/test_pickling.py Mon May 11 19:31:54 2009
@@ -34,9 +34,9 @@
p2config._initafterpickle(config.topdir)
return p2config
-class TestImmutablePickling:
- pytest_funcarg__pickletransport = ImmutablePickleTransport
+pytest_funcarg__pickletransport = ImmutablePickleTransport
+class TestImmutablePickling:
def test_pickle_config(self, testdir, pickletransport):
config1 = testdir.parseconfig()
assert config1.topdir == testdir.tmpdir
Modified: py/trunk/py/test/testing/test_pycollect.py
==============================================================================
--- py/trunk/py/test/testing/test_pycollect.py (original)
+++ py/trunk/py/test/testing/test_pycollect.py Mon May 11 19:31:54 2009
@@ -215,6 +215,12 @@
assert not skipped and not failed
class TestFunction:
+ def test_getmodulecollector(self, testdir):
+ item = testdir.getitem("def test_func(): pass")
+ modcol = item.getmodulecollector()
+ assert isinstance(modcol, py.test.collect.Module)
+ assert hasattr(modcol.obj, 'test_func')
+
def test_function_equality(self, tmpdir):
config = py.test.config._reparse([tmpdir])
f1 = py.test.collect.Function(name="name", config=config,
More information about the pytest-commit
mailing list