From morten.brekkevold at uninett.no Mon Nov 1 21:47:30 2010 From: morten.brekkevold at uninett.no (Morten Brekkevold) Date: Mon, 1 Nov 2010 21:47:30 +0100 Subject: [py-dev] Running Django unittests using py.test Message-ID: <20101101204730.GA6418@uninett.no> Hi, We're using nose to run our test suite, but we are considering switching to py.test. Some of our tests inherit Django's django.test.TestCase. This class mucks about with the test invocation machinery inherited from unittest.TestCase to add simple support for loading of database fixtures (no need to use super() calls to parent class' setUp/tearDown). Unfortunately, all these tests will fail when run by py.test, whereas nose runs thems fine. After a quick look at the pytest_unittest plugin, it seems it does its own tinkering and bypasses the normal way that unittest TestCases are invoked. The setUp, test* and tearDown methods are called explicitly. AFAIK, to "manually" run a unittest.TestCase, you instantiate it using the desired test method's name as an argument. The resulting instance is callable; calling it will take care of running setUp, the desired method and the tearDown. Django's TestCase class hooks into this by overriding unittest.TestCase.__call__(), and adds pre_setup and post_teardown hooks as well. Since py.test never uses TestCase instances as callables, Django fixtures are never loaded and these tests fail. I'm not familiar with the inner workings of py.test, so my question is: Is this behavior intentional? If not, can we change it? (this can be reproduced using http://dpaste.com/hold/268698/ - which doesn't require Django, it just rips off Django's TestCase code) -- Morten Brekkevold UNINETT From holger at merlinux.eu Mon Nov 1 23:19:30 2010 From: holger at merlinux.eu (holger krekel) Date: Mon, 1 Nov 2010 23:19:30 +0100 Subject: [py-dev] improving unittest support (was: Re: Running Django unittests using py.test) In-Reply-To: <20101101204730.GA6418@uninett.no> References: <20101101204730.GA6418@uninett.no> Message-ID: <20101101221929.GM11924@trillke.net> Hi Morten, On Mon, Nov 01, 2010 at 21:47 +0100, Morten Brekkevold wrote: > We're using nose to run our test suite, but we are considering switching to > py.test. > > Some of our tests inherit Django's django.test.TestCase. This class mucks > about with the test invocation machinery inherited from unittest.TestCase to > add simple support for loading of database fixtures (no need to use super() > calls to parent class' setUp/tearDown). > > Unfortunately, all these tests will fail when run by py.test, whereas nose > runs thems fine. Indeed, py.test (so far) only covers relatively plain unittest.TestCase usages. > After a quick look at the pytest_unittest plugin, it seems it does its own > tinkering and bypasses the normal way that unittest TestCases are invoked. > The setUp, test* and tearDown methods are called explicitly. Right. > AFAIK, to "manually" run a unittest.TestCase, you instantiate it using the > desired test method's name as an argument. The resulting instance is > callable; calling it will take care of running setUp, the desired method and > the tearDown. Originally the contributor (Guido Wesdorp) and me did this to maintain py.test's own separation of setup/teardown/call phases IIRC. > Django's TestCase class hooks into this by overriding > unittest.TestCase.__call__(), and adds pre_setup and post_teardown hooks as > well. Since py.test never uses TestCase instances as callables, Django > fixtures are never loaded and these tests fail. right. > I'm not familiar with the inner workings of py.test, so my question is: Is > this behavior intentional? If not, can we change it? It is intentional but we can (try to) change it :) > (this can be reproduced using http://dpaste.com/hold/268698/ - which doesn't > require Django, it just rips off Django's TestCase code) Thanks for providing the paste and your precise helpful info. As it happens i am in the process of preparing a pytest-2.0 and just went ahead and changed the unittest-plugin to invoke the test case as you described. This lets your example run. You can install it via:: pip install -i http://pypi.testrun.org pytest best in a virtualenv. Typing "py.test --version" should give you at least 2.0.0.dev18 Does this work better for you? sidenote: the unittest-plugin looks like this now:: http://bitbucket.org/hpk42/pytest/src/tip/pytest/plugin/unittest.py best, holger > > -- > Morten Brekkevold > UNINETT > _______________________________________________ > py-dev mailing list > py-dev at codespeak.net > http://codespeak.net/mailman/listinfo/py-dev > -- From morten.brekkevold at uninett.no Tue Nov 2 10:11:37 2010 From: morten.brekkevold at uninett.no (Morten Brekkevold) Date: Tue, 2 Nov 2010 10:11:37 +0100 Subject: [py-dev] improving unittest support (was: Re: Running Django unittests using py.test) In-Reply-To: <20101101221929.GM11924@trillke.net> References: <20101101204730.GA6418@uninett.no> <20101101221929.GM11924@trillke.net> Message-ID: <20101102091115.GA10005@uninett.no> On Mon, Nov 01, 2010 at 11:19:30PM +0100, holger krekel wrote: > > Django's TestCase class hooks into this by overriding > > unittest.TestCase.__call__(), and adds pre_setup and post_teardown hooks as > > well. Since py.test never uses TestCase instances as callables, Django > > fixtures are never loaded and these tests fail. > > right. > > > I'm not familiar with the inner workings of py.test, so my question is: Is > > this behavior intentional? If not, can we change it? > > It is intentional but we can (try to) change it :) As long as the mentioned internal separation of the phases won't be a problem. I'm sure you know best :) > > (this can be reproduced using http://dpaste.com/hold/268698/ - which doesn't > > require Django, it just rips off Django's TestCase code) > > Thanks for providing the paste and your precise helpful info. No prob, I wrote that just to satisfy my curiosity. Didn't want to start tinkering with py.test code without knowing what I was doing, though. > As it happens i am in the process of preparing a pytest-2.0 and just > went ahead and changed the unittest-plugin to invoke the test case > as you described. This lets your example run. Wow, that was fast :-) > pip install -i http://pypi.testrun.org pytest > Does this work better for you? Definitely. All tests pass with flying colors now, thank you very much. Is there an ETA on a 2.0 release? -- mvh Morten Brekkevold UNINETT From holger at merlinux.eu Thu Nov 4 13:35:49 2010 From: holger at merlinux.eu (holger krekel) Date: Thu, 4 Nov 2010 13:35:49 +0100 Subject: [py-dev] improving unittest support (was: Re: Running Django unittests using py.test) In-Reply-To: <20101102091115.GA10005@uninett.no> References: <20101101204730.GA6418@uninett.no> <20101101221929.GM11924@trillke.net> <20101102091115.GA10005@uninett.no> Message-ID: <20101104123549.GP11924@trillke.net> Hi Morten, On Tue, Nov 02, 2010 at 10:11 +0100, Morten Brekkevold wrote: > On Mon, Nov 01, 2010 at 11:19:30PM +0100, holger krekel wrote: > > > > Django's TestCase class hooks into this by overriding > > > unittest.TestCase.__call__(), and adds pre_setup and post_teardown hooks as > > > well. Since py.test never uses TestCase instances as callables, Django > > > fixtures are never loaded and these tests fail. > > > > right. > > > > > I'm not familiar with the inner workings of py.test, so my question is: Is > > > this behavior intentional? If not, can we change it? > > > > It is intentional but we can (try to) change it :) > > As long as the mentioned internal separation of the phases won't be a > problem. I'm sure you know best :) Let's hope so :) > > > (this can be reproduced using http://dpaste.com/hold/268698/ - which doesn't > > > require Django, it just rips off Django's TestCase code) > > > > Thanks for providing the paste and your precise helpful info. > > No prob, I wrote that just to satisfy my curiosity. Didn't want to start > tinkering with py.test code without knowing what I was doing, though. I am very interested in feedback and co-hacking on unittest support, specifically. I am not using unittest based test suites/cases myself so i depend on support and "registered interest" on that front. > > As it happens i am in the process of preparing a pytest-2.0 and just > > went ahead and changed the unittest-plugin to invoke the test case > > as you described. This lets your example run. > > Wow, that was fast :-) > > > pip install -i http://pypi.testrun.org pytest > > > Does this work better for you? > > Definitely. All tests pass with flying colors now, thank you very much. Is > there an ETA on a 2.0 release? By end next week i plan to have at least a 2.0 beta out. best, holger > -- > mvh > Morten Brekkevold > UNINETT > _______________________________________________ > py-dev mailing list > py-dev at codespeak.net > http://codespeak.net/mailman/listinfo/py-dev > -- From holger at merlinux.eu Tue Nov 23 17:44:54 2010 From: holger at merlinux.eu (holger krekel) Date: Tue, 23 Nov 2010 17:44:54 +0100 Subject: [py-dev] upcoming releases (and distro names) Message-ID: <20101123164454.GU1009@trillke.net> Hi all, here is the new repositories and planned distribution packages: pytest-2.0 bb:pytest (depends on >=py-1.4) pycmd-1.0 bb:pycmd (depends on >=py-1.4) py-1.4 bb:py (no dependencies) The according code was previously contained in py==1.3.4 and the repository was bb:py-trunk which will cease to be active. You can install the most recent development version so e.g. "pytest" via this line:: pip install -i http://pypi.testrun.org -U pytest pip install -i http://pypi.testrun.org -U pycmd ... and i'd be happy to hear about problems as i plan to release things real soon now (tm). note: if you have used the intermediate packages from http://pypi.testrun.org then you may need to uninstall "pylib" via e.g. "pip uninstall pylib" as i dropped this packaging idea. KISS and all that. cheers, holger From holger at merlinux.eu Thu Nov 25 21:39:00 2010 From: holger at merlinux.eu (holger krekel) Date: Thu, 25 Nov 2010 21:39:00 +0100 Subject: [py-dev] pytest-2.0: asserts++, unittest++, reporting++, config++, docs++ Message-ID: <20101125203900.GC1009@trillke.net> py.test 2.0.0: asserts++, unittest++, reporting++, config++, docs++ =========================================================================== Welcome to pytest-2.0.0, a major new release of "py.test", the rapid easy Python testing tool. There are many new features and enhancements, see below for summary and detailed lists. A lot of long-deprecated code has been removed, resulting in a much smaller and cleaner implementation. See the new docs with examples here: http://pytest.org/ I also just released pytest-xdist-1.5, the plugin for distributed testing - which was successfully used to speed up a real-life unittest suite 4-times written for (twisted) trial, thanks to the generic unittest integration improvements. See here for docs and install instructions: http://pypi.python.org/pypi?:action=display&name=pytest-xdist A note on packaging: pytest used to part of the "py" distribution up until version py-1.3.4 but this has changed now: pytest-2.0.0 only contains py.test related code and is expected to be backward-compatible to existing test code. If you want to install pytest, just type one of:: pip install -U pytest easy_install -U pytest Many thanks to all issue reporters and people asking questions or complaining. Particular thanks to Floris Bruynooghe and Ronny Pfannschmidt for their great coding contributions and many others for feedback and help. best, holger krekel New Features ----------------------- - new invocations through Python interpreter and from Python:: python -m pytest # on all pythons >= 2.5 or from a python program:: import pytest ; pytest.main(arglist, pluginlist) see http://pytest.org/2.0.0/usage.html for details. - new and better reporting information in assert expressions if comparing lists, sequences or strings. see http://pytest.org/2.0.0/assert.html#newreport - new configuration through ini-files (setup.cfg or tox.ini recognized), for example:: [pytest] norecursedirs = .hg data* # don't ever recurse in such dirs addopts = -x --pyargs # add these command line options by default see http://pytest.org/2.0.0/customize.html - improved standard unittest support. In general py.test should now better be able to run custom unittest.TestCases like twisted trial or Django based TestCases. Also you can now run the tests of an installed 'unittest' package with py.test:: py.test --pyargs unittest - new "-q" option which decreases verbosity and prints a more nose/unittest-style "dot" output. - many many more detailed improvements details Fixes ----------------------- - fix issue126 - introduce py.test.set_trace() to trace execution via PDB during the running of tests even if capturing is ongoing. - fix issue124 - make reporting more resilient against tests opening files on filedescriptor 1 (stdout). - fix issue109 - sibling conftest.py files will not be loaded. (and Directory collectors cannot be customized anymore from a Directory's conftest.py - this needs to happen at least one level up). - fix issue88 (finding custom test nodes from command line arg) - fix issue93 stdout/stderr is captured while importing conftest.py - fix bug: unittest collected functions now also can have "pytestmark" applied at class/module level Important Notes -------------------- * The usual way in pre-2.0 times to use py.test in python code was to import "py" and then e.g. use "py.test.raises" for the helper. This remains valid and is not planned to be deprecated. However, in most examples and internal code you'll find "import pytest" and "pytest.raises" used as the recommended default way. * pytest now first performs collection of the complete test suite before running any test. This changes for example the semantics of when pytest_collectstart/pytest_collectreport are called. Some plugins may need upgrading. * The pytest package consists of a 400 LOC core.py and about 20 builtin plugins, summing up to roughly 5000 LOCs, including docstrings. To be fair, it also uses generic code from the "pylib", and the new "py" package to help with filesystem and introspection/code manipulation. (Incompatible) Removals ----------------------------- - py.test.config is now only available if you are in a test run. - the following (mostly already deprecated) functionality was removed: - removed support for Module/Class/... collection node definitions in conftest.py files. They will cause nothing special. - removed support for calling the pre-1.0 collection API of "run()" and "join" - removed reading option values from conftest.py files or env variables. This can now be done much much better and easier through the ini-file mechanism and the "addopts" entry in particular. - removed the "disabled" attribute in test classes. Use the skipping and pytestmark mechanism to skip or xfail a test class. - py.test.collect.Directory does not exist anymore and it is not possible to provide an own "Directory" object. If you have used this and don't know what to do, get in contact. We'll figure someting out. Note that pytest_collect_directory() is still called but any return value will be ignored. This allows to keep old code working that performed for example "py.test.skip()" in collect() to prevent recursion into directory trees if a certain dependency or command line option is missing. see :ref:`changelog` for more detailed changes. -- From holger at merlinux.eu Thu Nov 25 21:54:36 2010 From: holger at merlinux.eu (holger krekel) Date: Thu, 25 Nov 2010 21:54:36 +0100 Subject: [py-dev] tox-0.9: better CI bootstrapping / custom PyPI index servers Message-ID: <20101125205436.GD1009@trillke.net> Hi all, i just released tox-0.9, the generic virtualenv-using test integration automation tool (tm). This release brings a new zero-install way to deploy tox and your test suite on Hudson slaves. Apart from one or more Python interpreters you do not need anything on the slave side, see here for details: http://codespeak.net/tox/example/hudson.html#zero-installation-for-slaves Moreover, you can now work with multiple PyPI index servers which is very nice if you (want to) operate your own in-company servers, see here for details: http://codespeak.net/tox/example/basic.html#using-a-different-default-pypi-url Btw, what kind of custom servers are you successfully using? (tell me on twitter @hpk42 if you like). Oh, and for basic info just go to http://codespeak.net/tox if that wasn't clear yet. have fun, holger Changelog for 0.9 ----------------------- - added support for working with different and multiple PYPI indexservers. - fix pip-installation mixups by always unsetting PIP_RESPECT_VIRTUALENV (thanks Armin Ronacher) - issue1: Add a toxbootstrap.py script for tox, thanks to Sridhar Ratnakumar - new option: -r|--recreate to force recreation of virtualenv - show logfile content if there is an error (makes CI output more readable) - depend on py>=1.4.0 which does not contain or install the py.test anymore which is now a separate distribution "pytest". From holger at merlinux.eu Fri Nov 26 09:42:49 2010 From: holger at merlinux.eu (holger krekel) Date: Fri, 26 Nov 2010 09:42:49 +0100 Subject: [py-dev] [TIP] pytest-2.0: asserts++, unittest++, reporting++, config++, docs++ In-Reply-To: References: <20101125203900.GC1009@trillke.net> Message-ID: <20101126084249.GE1009@trillke.net> Hi Phlip, On Thu, Nov 25, 2010 at 17:05 -0800, Phlip wrote: > > I am now converting my brand-new, blue-sky, make-or-break project to > > use your tester. Don't let me down! Heh, i'd like to help, am always looking for real-world feedback and issues :) > Aaand, i would prefer to migrate case-by case. Is there some way to > invoke a single suite, such as from within the Django test runner? > > I'm trying the hard way - throwing everything to pytest via "pip > install django-pytest", but I get this mess: haven't used or tested django-pytest myself ... The below error seems to indicate pytest is not installed. Maybe django-pytest does not require it and you need to install pytest manually? As to your case-by-case strategy: what happens if you simply run "py.test" (2.0) on your code base today? cheers, holger P.S.: I am often at #pylib on irc.freenode.net where we could sort out things a bit more realtime. Otherwise let's stick to one mailing list, i am fine with sticking to just TIP and to take things to private mail if we have more than a couple of to and fros. > [localhost] run: python manage.py test --settings=settings.test --verbosity=0 > Traceback (most recent call last): > File "manage.py", line 11, in > execute_manager(settings) > File "/usr/lib/pymodules/python2.6/django/core/management/__init__.py", > line 438, in execute_manager > utility.execute() > File "/usr/lib/pymodules/python2.6/django/core/management/__init__.py", > line 379, in execute > self.fetch_command(subcommand).run_from_argv(self.argv) > File "/usr/lib/pymodules/python2.6/django/core/management/base.py", > line 191, in run_from_argv > self.execute(*args, **options.__dict__) > File "/usr/lib/pymodules/python2.6/django/core/management/base.py", > line 220, in execute > output = self.handle(*args, **options) > File "/usr/local/lib/python2.6/dist-packages/django_pytest/management/commands/test.py", > line 40, in handle > super(Command, self).handle(*args, **kwargs) > File "/usr/lib/pymodules/python2.6/django/core/management/commands/test.py", > line 34, in handle > failures = TestRunner(test_labels, verbosity=verbosity, > interactive=interactive) > File "/usr/local/lib/python2.6/dist-packages/django_pytest/test_runner.py", > line 7, in run_tests > load_entry_point('py>=1.0.0', 'console_scripts', 'py.test')() > File "/usr/lib/python2.6/dist-packages/pkg_resources.py", line 305, > in load_entry_point > return get_distribution(dist).load_entry_point(group, name) > File "/usr/lib/python2.6/dist-packages/pkg_resources.py", line 2243, > in load_entry_point > raise ImportError("Entry point %r not found" % ((group,name),)) > ImportError: Entry point ('console_scripts', 'py.test') not found > > Uh, I did all the things this said to: > http://pypi.python.org/pypi/django-pytest/0.1.4 > > Except I didn't call from py.test. If I had, I suspect I would not > achieve migrate case by case, which is more important to me because my > fixtures are somewhat elaborate! > > -- > ? Phlip > ? http://c2.com/cgi/wiki?ZeekLand > -- From samuele.pedroni at gmail.com Sun Nov 28 16:35:48 2010 From: samuele.pedroni at gmail.com (Samuele Pedroni) Date: Sun, 28 Nov 2010 16:35:48 +0100 Subject: [py-dev] oejskit 0.8.9, javascript testing support, consolidation release In-Reply-To: References: Message-ID: I have just uploaded to pypi a new release 0.8.9 of oejskit, this is just a consolidation release. The main changes it that it is packaged as source tarball, making it easier to install with Python 2.7, and that it drops support for Python 2.5. This release doesn't support yet py.test 2.0, I have started investigating what is required to support it, it is not a completely straightforward to port the plugin because browser and test session management need to be intertwined, it will take a bit of time to release something working with it. unittest.py support is working and unchanged. About OE jskit: jskit contains infrastructure and in particular a py.test plugin to enable running unit tests for JavaScript code inside browsers. It contains also glue code to run JavaScript tests from unittest.py based test suites. The approach also enables to write integration tests such that the JavaScript code is tested against server-side Python code mocked as necessary. Any server-side framework that can already be exposed through WSGI can play. The plugin requires py.test 1.3.4 or slightly older. More information and downloading at: http://pypi.python.org/pypi/oejskit including a changelog and ?documentation. regards, Samuele Pedroni From holger at merlinux.eu Mon Nov 29 16:20:05 2010 From: holger at merlinux.eu (holger krekel) Date: Mon, 29 Nov 2010 16:20:05 +0100 Subject: [py-dev] py.test api - Running tests in a module object? In-Reply-To: <10C7D726-EFFB-42F4-902C-D023A20FE498@activestate.com> References: <4C5B2316.7050909@activestate.com> <20100805210652.GY1914@trillke.net> <4C5B2BF7.8070107@activestate.com> <20100805212823.GB1914@trillke.net> <10C7D726-EFFB-42F4-902C-D023A20FE498@activestate.com> Message-ID: <20101129152005.GS1009@trillke.net> On Thu, Aug 05, 2010 at 14:39 -0700, Sridhar Ratnakumar wrote: > > On 2010-08-05, at 2:28 PM, holger krekel wrote: > > > On Thu, Aug 05, 2010 at 14:24 -0700, Sridhar Ratnakumar wrote: > >> On 8/5/2010 2:06 PM, holger krekel wrote: > >>> On Thu, Aug 05, 2010 at 13:46 -0700, Sridhar Ratnakumar wrote: > >>>>> I am trying to run py.test on a given module object dynamically. > >>>>> > >>>>> mod = __import__('company.foo.somemod') > >>>>> py.test.run(mod) > >>>>> > >>>>> A quick glance in the py.test source doesn't seem like a trivial thing. > >>>>> IPython introspection did not give me any clue either. > >>>>> > >>>>> Is there a way to do this at all? I did try > >>>>> py.test.cmdline.main(['...']), but that accepts only file path, not the > >>>>> Python module object itself. > >>> there currently is no direct support for running tests in python modules. > >>> py.test basically always starts from the file system. Does it help you > >>> to try to fish the file from somemod.__file__ and pass this to cmdline.main()? > >>> You only expect it to collect tests of a single module, right? > >>> I guess we could add some more direct support for this if > >>> you continue to have the need (please create an issue if so). > >> > >> Hmm, __file__ is a workaround, but it seems to work fine for my use > >> case. I was just wondering if an API was exposed. > >> > >> I am happy with __file__ for now. > > > > What do you think about automating this lookup and allowing > > > > py.test.cmdline.main([module]) > > > > maybe that's enough? > > > That might have helped, but then - just now, I rewrote some routines in my code to simply rely on .py file paths instead of modules. This simplified the code a bit. So now I have come to prefer file paths. :-) If I ever have the need for running tests on module/class/collection-of-funcs objects, I will be sure to open a feature request. Thanks. FYI py.test-2.0 now supports the "--pyargs" option: py.test --pyargs company.foo.somemod so that this will discover the __file__'s directory and collect tests there. You can make this permanent, see here: http://pytest.org/customize.html#adding-default-options cheers, holger From flub at devork.be Mon Nov 29 23:38:29 2010 From: flub at devork.be (Floris Bruynooghe) Date: Mon, 29 Nov 2010 22:38:29 +0000 Subject: [py-dev] Adding a funcarg to pytest_capturelog plugin Message-ID: Hi Meme Your capturelog plugin for py.test is very useful, thanks for writing it! But recently I've had the need to be able to modify the recorded loglevel inside the test (or during test setup) mostly to temporarily silence the logging. For this I added a funcarg named "capturelog" which exposes the loghandler (wrapped so it can be used as a context manager too) which seems like a fairly nice way to work. The only ugly part of this is that I had to store the loghandler instance on the function instance as well instead of just on the item. The second thing I wanted to do was to be able to assert on the contents of log messages, specifically I wanted to ensure there where no critical messages logged. Having the above funcarg already it seemed most logical to just add this to the same loghandler already used by overwriting the .emit() method to store the raw LogRecord instances. I guess you could argue as to weather the list of LogRecord instances or the entire py.io.TextIO version of log should be made available, but I preferred the former. I've cloned your repository at http://bitbucket.org/flub/pytest-capturelog/ to add this functionality, the changes should be pretty simple and obvious. I think it could be useful if you could merge a version of this. Let me know if you have remarks or would like to see things slightly differently. Lastly, while talking on #pylib (IRC) to Holger he mentioned that he wondered if it would be worth to include the capturelog extension as a default plugin in py.test since it is to cooperate with a stdlib module. So if you also think this is interesting you might like to discuss this with him. Regards Floris -- Debian GNU/Linux -- The Power of Freedom www.debian.org | www.gnu.org | www.kernel.org