Re: [Python-Dev] Proposed unittest changes

Ben Finney wrote:
I intend to start working on them in August, after I have finished my current writing commitments. The full list of changes proposed (feel free to start - but ping me or the list) and not shot down was something like: Documenting that the assert method names are to be preferred over the 'FailUnless' names (this stirred up some controversy this weekend so should probably not happen). Adding the following new asserts: assertIn (member, container, msg=None) assertNotIn (member, container, msg=None) assertIs (first, second, msg=None) assertNotIs (first, second, msg=None) assertRaisesWithMessage (exc_class, message, callable, *args, **keywargs) A simple 'RunTests' function that takes a collection of modules, test suites or test cases and runs them with TextTestRunner - passing on keyword arguments to the test runner. This makes running a test suite easier - once you have collected all your test cases together you can just pass them to this function so long as you are happy with the default runner (possibly allowing an alternative runner class to be provided as a keyword argument). Make the error messages for "assertEquals" and "assertNotEquals" more useful - showing the objects that compare incorrectly even if an explicit message is passed in. Additionally, when comparing lists and tuples that are the same length show the members (and indices?) that were different. Possibly even providing a diff in the case of comparing strings (we have an implementation of this at work). assertLessThan assertGreaterThan assertLessThanOrEquals assertGreaterThanOrEquals Guido suggested various variants on assertEquals: assertListEqual(self, list1, list2, msg=None): assertDictEqual(self, d1, d2, msg=None): assertMultiLineEqual(self, first, second, msg=Non In my opinion these can all be provided by improving the messages from assertEquals and don't require new methods. assertSameElements(self, expected_seq, actual_seq, msg=None): I usually achieve this with: assertEquals(set(actual), set(expected)) A convenience method might be nice, but I'm worried about the API growing in an unbounded way. Other suggestions that weren't controversial but I might not get to: assertRaisesWithMessage taking a regex to match the error message expect methods that collect failures and report at the end of the test (allowing an individual test method to raise several errors without stopping) assertIsInstance and assertIsSubclass Michael Foord -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/ http://www.trypython.org/ http://www.ironpython.info/ http://www.theotherdelia.co.uk/ http://www.resolverhacks.net/

Michael Foord <fuzzyman@voidspace.org.uk> writes:
assertSameElements(self, expected_seq, actual_seq, msg=None):
All these are new, so there is no existing expectation of these names from users of the standard library 'unittest' module (i.e. no backward-compatibility concern since these are new methods). If we're planning to deprecate the existing non-PEP-8 names in 2.7 and 3.1, why would we introduce new names that are non-PEP-8? Wouldn't it be better to add these as PEP-8 compatible names in the first instance? -- \ “You've got the brain of a four-year-old boy, and I'll bet he | `\ was glad to get rid of it.” —Groucho Marx | _o__) | Ben Finney

Michael Foord wrote:
Please, let's call this one "assertIsNot". I know it's valid Python to say if a not is b: but it's a much less natural way of expressing the condition, and (for all I know) might even introduce an extra negation operation. "is not" is, I believe, treated as a single operator.
[...]
regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/

Steve Holden <steve@holdenweb.com> writes:
Dang. You're exactly right. The problem is, that makes it quite inconsistent with other "not" uses (such as "assert_not_equal", "assert_not_in", etc.) I would really prefer that all these "not" uses be gramatically consistent for predictability. Is this a case where "assert_is_not" should exist alongside "assert_not_is"? I know that part of the goal here is to have "preferably only one obvious way to do it", but I can see *both* those names as "the obvious way to do it". Is this an instance where the "preferably" clause must be exercised in the negative? -- \ “Every sentence I utter must be understood not as an | `\ affirmation, but as a question.” —Niels Bohr | _o__) | Ben Finney

Ben Finney wrote:
If we can flip the word order in the language syntax, we can sure as heck flip it in a method name :) Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org

Nick Coghlan <ncoghlan@gmail.com> writes:
To be clear, I take it you're in favour of the following names (with no aliases): assert_equal assert_not_equal assert_is assert_is_not assert_in assert_not_in assert_almost_equal assert_not_almost_equal and so on; i.e. that 'assert_is_not' breaks the obvious pattern set by the others, in the interest of matching Python's 'is not' grammar. -- \ “Instead of having ‘answers’ on a math test, they should just | `\ call them ‘impressions’, and if you got a different | _o__) ‘impression’, so what, can't we all be brothers?” —Jack Handey | Ben Finney

Ben Finney wrote:
Well, I'd have said "in the interest of reading correctly in English", though I have to acknowledge this may not be an issue for many Python users whose first language not is English. "assert_not_is" is just dissonant to my ears. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/

Steve Holden wrote:
The two reasons aren't that far apart, given that Python's grammar uses "is not" because it makes more sense in English. One thing to remember is that the word 'is' is actually implied in all of the contracted phrases above other than those already including it explicitly. "x is equal to y" "x is not equal to y" "x is y" "x is not y" "x is in y" "x is not in y" "x is almost equal to y" "x is not almost equal to y" As for which phrasing I personally prefer, unit tests and method names are areas where I'm quite happy to paint the bike shed the same colour as the house :) Cheers, Nick. P.S. Deciphering that somewhat strained metaphor: I don't have a strong preference with regards to the unit test method names. While I tend to go with the assert* variants when left to my own devices, I have no problem sticking to the fail* variants when updating a test that uses them. Camel-case vs underscores in method names isn't something that particularly worries me either. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org

Steve Holden <steve@holdenweb.com> writes:
I'd count this as another (minor) point in favour of making the 'fail*' methods canonical: the names are consistent *and* gramatically sensible: fail_if_equal fail_unless_equal fail_if_is fail_unless_is fail_if_in fail_unless_in fail_if_almost_equal fail_unless_almost_equal -- \ “We are not gonna be great; we are not gonna be amazing; we are | `\ gonna be *amazingly* amazing!” —Zaphod Beeblebrox, _The | _o__) Hitch-Hiker's Guide To The Galaxy_, Douglas Adams | Ben Finney

Ben Finney wrote:
I'd count this as another (minor) point in favour of making the 'fail*' methods canonical: the names are consistent *and* gramatically sensible:
-1 I'm surprised nobody (that I've noticed) has brought up the point yet that test code is a lot easier to read if it makes positive assertions. When reading failure conditions, one has to constantly invert them in order to deduce the behaviour that is tested. failUnless and friends aren't better either IMO since while they do work with positive assertions, the method names themselves are doubly negative. assert* methods are so much more straightforward to comprehend. -- Thomas

Thomas Lotze wrote:
I think this is where I came in. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/

On Tue, 15 Jul 2008 04:55:59 pm Thomas Lotze wrote:
Please don't claim that your subjective opinion is an objective fact.
When reading failure conditions, one has to constantly invert them in order to deduce the behaviour that is tested.
You might have to. Don't assume that everyone else has your difficulty.
Maybe for you. That's not a human universal. Please don't assume that your favourite bike-shed colour must be the favourite colour of everyone else too. -- Steven

On Mon, Jul 14, 2008 at 11:55 PM, somebody wrote:
I'm surprised nobody (that I've noticed) has brought up the point yet that [...]
Not picking on whoever wrote that specifically, but if there's anything that surprises me, it's how many people have voiced opinions already (including many of them that I hadn't heard in this group before). There doesn't seem to be an end to this debate, and it is awfully close to deteriorating to pure bikeshedding and attempted ad-hominem attacks. I really don't have time to participate in detail, since all the time I have for Python I need to spend on trying to help review the 2.6 and 3.0 beta releases. But I want to remind people that radical changes to the unittest infrastructure will inconvenience many large 3rd party projects using Python, and I urge folks to look for ways to improve the unittest APIs in other ways instead. It's not the end of the world if the unittesting API uses assertEqual() instead of assert_equal() until the end of times. It would, however, be a shame if we couldn't agree to *add* a bunch of features, for example better reporting when two lists or long strings differ. -- --Guido van Rossum (home page: http://www.python.org/~guido/)

"Guido van Rossum" <guido@python.org> writes:
I intend to phrase such additions in terms of PEP-8-only names, so this name consolidation seems a natural prerequisite. -- \ “Are you pondering what I'm pondering?” “Well, I think so | `\ Brain, but what if we stick to the seat covers?” —_Pinky and | _o__) The Brain_ | Ben Finney

[Michael Foord]
[Steve Holden]
Please, let's call this one "assertIsNot".
+1
I know it's valid Python to say
if a not is b:
Nope, that's a syntax error.
"is not" and "not in" are both binary infix operators, not to be confused with the distinct use of "not" on its own as a unary prefix operator. "not is" and "in not" are both gibberish.

Tim Peters wrote:
Rats, naturally I was thinking of "if not (a is b):"
regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/

On Sun, 13 Jul 2008 23:51:44 +0100, Michael Foord <fuzzyman@voidspace.org.uk> wrote:
Several of these are implemented in other libraries (Twisted, at least). You might save some time by grabbing them and their unit tests, rather than re-implementing them. Twisted calls `assertIs´ `assertIdentical´, by the way.
Actually, I remember that someone raised an object to this as being not as flexible as some might want - an objection I agree with. Perhaps that was overruled, but I didn't want this to slip by as "not controversial".
The former of these is also in Twisted already, if you want to copy it. Jean-Paul

Michael Foord wrote:
Would it be worth Ben collating your current notes into a draft PEP targeting 2.7/3.1? Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org

Nick Coghlan <ncoghlan@gmail.com> writes:
Would it be worth Ben collating your current notes into a draft PEP targeting 2.7/3.1?
I'll do it and we'll find out. -- \ “A fine is a tax for doing wrong. A tax is a fine for doing | `\ well.” —anonymous | _o__) | Ben Finney

Michael Foord <fuzzyman@voidspace.org.uk> writes:
Thanks. I'm working these into another draft PEP that I hope to have up in a day or two. -- \ “[W]e are still the first generation of users, and for all that | `\ we may have invented the net, we still don't really get it.” | _o__) —Douglas Adams | Ben Finney

Raymond Hettinger wrote:
As proposed the changes don't remove or rename anything - so there will be no code breakage, just additional test methods. However, as we're into the beta phase I don't think these changes can make 2.6 / 3.0 anyway. Michael
-- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/ http://www.trypython.org/ http://www.ironpython.info/ http://www.theotherdelia.co.uk/ http://www.resolverhacks.net/

Michael Foord <fuzzyman@voidspace.org.uk> writes:
As proposed the changes don't remove or rename anything - so there will be no code breakage, just additional test methods.
Right, so I'm putting up a separate PEP just for the renaming. Should be arriving on this list soon.
However, as we're into the beta phase I don't think these changes can make 2.6 / 3.0 anyway.
Definitely agreed. -- \ “You can be a victor without having victims.” —Harriet Woods | `\ | _o__) | Ben Finney

From: "Ben Finney" <ben+python@benfinney.id.au>
Right, so I'm putting up a separate PEP just for the renaming. Should be arriving on this list soon.
I would like to work with you or someone else who is interested on an alternative PEP for a separate, simpler test module using the py.test syntax. That is much simpler to learn and use. Instead of self.assertIsNot and whatnot, you write: assert a is not b No need for tons of word-by-word spellings on things we already have syntax for. Almost anyone who has used py.test can attest its syntax is much more natural, easy to learn, easy to both read and write, and is much lighter weight. I think some variant of py.test could be done that is compatable with unittest and the did not have the "magic" present in earlier versions of py.test. I wrote a recipe (somewhat rough and incomplete) that shows how easily this could be done: http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/572194 Raymond

On Mon, Jul 14, 2008 at 6:26 PM, Raymond Hettinger <python@rcn.com> wrote:.
Bringing the total amount of test modules in the stdlib to 3. OWTDI indeed. Anyway, I don't think something like needs to be (re)written. nose[1] is already an excellent implementation of this that I would like to see in the stdlib. [1] http://www.somethingaboutorange.com/mrl/projects/nose/ -- Cheers, Benjamin Peterson "There's no place like 127.0.0.1."


Raymond Hettinger wrote:
However, to provide readable output for errors in even simple tests (like a == b) py.test does magic with stack frames and code objects - in order to discover the objects being compared. As this relies on what are essentially implementation details of the Python interpreter it means that some implementations (specifically IronPython which doesn't have Python stack frames and only a minimal representation of frame objects) will never be able to run it. I think it would be a bad idea to move *Python testing* itself over to a framework like this. I personally find unittest pretty readable, the feature most lacking is autodiscovery of tests which nose does seem to provide very well although I haven't used it yet. Michael
-- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/ http://www.trypython.org/ http://www.ironpython.info/ http://www.theotherdelia.co.uk/ http://www.resolverhacks.net/

From: "Michael Foord" <fuzzyman@voidspace.org.uk>
Don't have to go that route. Can use plain python assert failures with a stacktrace. Or can trigger pdb, or let the user specify a mode that calls some more advanced test runner or test reporter with introspection. This can be done without making everything hard.
I think it would be a bad idea to move *Python testing* itself over to a framework like this.
Don't want to convert the python testing. Would like to offer a lighter-weight alternative to our users.
It takes about one day of using py.test to realize have much cleaner and more readable its syntax is. Also, writing the tests is *much* more pleasant. It has the same clean, clear joy as writing regular python code. By comparison, the code using unittest.py is javaesque. I've written tons of test with unittest.py and and find it to be joyless. I realize there is a matter of taste involved but if you talk to any regular users of py.test, they will *all* attest to the syntax being much more readable, lightweight, and pleasant to use. It encourages writing tests. That being said, I think there are less magical, much simpler ways to implement it. I think Holger is working on it as we speak. Raymond

On Mon, Jul 14, 2008 at 5:13 PM, Raymond Hettinger <python@rcn.com> wrote:
I, too, have written tons of tests with unittest.py (and Google's extensions, which follow the same style), and reviewed even more. I agree that this is pretty joyless, but I'm not at all sure that the unittest API is the reason. It seems to me that a main problem with writing test code is and will always remain due to the need to use mocks, stubs and other similar techniques (e.g. dependency injection). Typical test code that I've written or reviewed spends more time setting up the input conditions for testing than it spends checking the results. Ten lines of mocking code to one self.assertEqual() call seems typical. -- --Guido van Rossum (home page: http://www.python.org/~guido/)

Guido van Rossum wrote:
Maybe Python needs a good mocking module in the standard library. There are plenty, but we use a particularly nice one at Resolver Systems [1]. :-) It auto-creates attributes as mocks, allowing you to assert calls made to all of its children along with convenience methods like 'assert_called_with' and has a companion decorator that patches class / module level attributes just for the duration of the test. As we're changing more of our tests over to use these we're finding it reduces the volume and complexity of our test code. Michael Foord [1] Based on http://code.google.com/p/mock/ although there is some outstanding code to sync back to the project. -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/ http://www.trypython.org/ http://www.ironpython.info/ http://www.theotherdelia.co.uk/ http://www.resolverhacks.net/

From: "Michael Foord" <fuzzyman@voidspace.org.uk>
Maybe Python needs a good mocking module in the standard library. There are plenty, but we use a particularly nice one at Resolver Systems [1]. :-)
-1 This comes up occassionally and gets shot down. http://bugs.python.org/issue708125 Mock objects mean different things to different people. Some expect more simulated behavior and others want less. It's rare to find agreement about general purpose mock objects and frameworks. Mock libraries create their own complexities and burdens on a programmer's memory. It's often easier to create a small special case mock object than to remember how to configure a general purpose one. And, afaict, there is no fan club for some particular python mock object library -- it seems to only come up in general discussions about possibilities for growing the unittest module, and almost never comes up in the context of solving a real problem that hasn't already be addressed in some other way. Raymond

On Mon, Jul 14, 2008 at 09:37:30PM -0700, Raymond Hettinger wrote: -> From: "Michael Foord" <fuzzyman@voidspace.org.uk> -> >Maybe Python needs a good mocking module in the standard library. There -> >are plenty, but we use a particularly nice one at Resolver Systems [1]. :-) -> -> -1 -> -> This comes up occassionally and gets shot down. -> http://bugs.python.org/issue708125 -> -> Mock objects mean different things to different people. -> Some expect more simulated behavior and others want less. -> It's rare to find agreement about general purpose mock objects and -> frameworks. -> Mock libraries create their own complexities and burdens on a programmer's -> memory. -> It's often easier to create a small special case mock object -> than to remember how to configure a general purpose one. -> And, afaict, there is no fan club for some particular python mock -> object library -- it seems to only come up in general discussions -> about possibilities for growing the unittest module, and almost -> never comes up in the context of solving a real problem that -> hasn't already be addressed in some other way. Also see: http://lists.idyll.org/pipermail/testing-in-python/2007-November/000406.html & associated thread, for those interested in the variety of mock libraries... cheers, --titus -- C. Titus Brown, ctb@msu.edu

And: http://bugs.python.org/issue2156 Raymond

On Mon, Jul 14, 2008 at 6:43 PM, Michael Foord <fuzzyman@voidspace.org.uk> wrote:
Maybe what we need to do then is make the assert statement more powerful. I like the idea of having it call a builtin called __assert__ which is called by the assert statement. The AST for the node could be attached. -- Cheers, Benjamin Peterson "There's no place like 127.0.0.1."

Raymond Hettinger wrote:
Ah, in my haste I skipped over your comment about "magic", my apologies. But in the absence of magic how do you propose to provide a meaningful error message from the failure of: assert a == b To wrap it in a function like "assert equals(a, b)" seems to gain little over unittest. Michael
-- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/ http://www.trypython.org/ http://www.ironpython.info/ http://www.theotherdelia.co.uk/ http://www.resolverhacks.net/

Michael Foord wrote:
Aside from the question of providing nice error messages, two questions that immediately come to mind for me are: - how do I run my tests with -O or -OO? (since the compiler will throw all the assert statements away before any Python code gets a chance to look at them) - how do I test that code raises an expected exception? - how do I explicitly fail a test case? (e.g. I'll often do this when I want to test an operation with a variety of different inputs - I'll test for all of the inputs of interest, collecting the failures in a list, then reporting a single error message at the end detailing all of the cases that failed) And I've also never had any problem whatsoever debugging unit tests with print statements - one of the effects of the -v switch is to display anything which is written to stderr/stdout on the console again. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org

Raymond Hettinger wrote:
+1 for a simpler testing module. Just letting you know there is interest in a lighter weight testing suite. Looking at the unittest discussions, it doesn't look like it is getting easier to use, but more complex. Py.test looks very interesting, especially the test discovery parts. I also agree we don't need special methods for every variation of assertedness. I've been thinking that a few decorators may go a long way to making writing tests easy. It also reduces the level of indentation needed. Ron

Ron Adam <rrr@ronadam.com> writes:
+1 for a simpler testing module.
I've no objection.
Just letting you know there is interest in a lighter weight testing suite.
'doctest' is a very simple testing module, that is a very useful tool.
Looking at the unittest discussions, it doesn't look like it is getting easier to use, but more complex.
How so? One PEP proposed this week specifies to do nothing but conform 'unittest' with the standard library guidelines, and remove redundant names. That surely makes it simpler to use. Another PEP specifies to add helper methods that simplify a number of common cases. What is it you see making unittest "more complex to use"?
My main complaint about 'py.test' is that it's yet-another-framework. We already have 'doctest' and 'unittest', and they play together reasonably well. 'nose' <URL:http://somethingaboutorange.com/mrl/projects/nose/> looks better for consideration, especially since it integrates well with 'unittest'.
I've been thinking that a few decorators may go a long way to making writing tests easy. It also reduces the level of indentation needed.
There are a number of these already in 'nose'. -- \ “I fly Air Bizarre. You buy a combination one-way round-trip | `\ ticket. Leave any Monday, and they bring you back the previous | _o__) Friday. That way you still have the weekend.” —Steven Wright | Ben Finney

Ben Finney wrote:
No complaint here. It's a good place to start.
Another PEP specifies to add helper methods that simplify a number of common cases.
In my opinion adding 19 more methods makes it more complex. I'd rather see more focus on handling test failures in general without the need for so many special helper functions.
What is it you see making unittest "more complex to use"?
More methods and method signatures to learn and remember. assert_true(…) assert_false(…) assert_almost_equal(…) assert_equal(…) assert_not_almost_equal(…) assert_not_equal(…) assert_raises(exc_class, callable_obj, *args, **kwargs) assert_compare_true(op, first, second, msg=None) assert_in(container, member, msg=None) assert_is(first, second, msg=None) assert_less_than(first, second, msg=None) assert_greater_than(first, second, msg=None) assert_less_than_or_equal(first, second, msg=None) assert_greater_than_or_equal(first, second, msg=None) assert_members_equal(first, second, msg=None) assert_sequence_equal(first, second, msg=None) assert_raises_with_message_regex(exc_class, message_regex, callable_obj, *args, **kwargs) assert_compare_false(op, first, second, msg=None) assert_not_in(container, member, msg=None) assert_is_not(first, second, msg=None) assert_not_less_than(first, second, msg=None) assert_not_greater_than(first, second, msg=None) assert_not_less_than_or_equal(first, second, msg=None) assert_not_greater_than_or_equal(first, second, msg=None) assert_members_not_equal(first, second, msg=None) assert_sequence_not_equal(first, second, msg=None) That comes to 26 variations of assert. There are really only a small set of basic conditions to test for. correct values incorrect values unexpected exceptions correct exceptions incorrect exceptions missing exceptions I think the unittest module could better handle testing of exceptions and distinguishing exception produced by test code from the code being tested. That was painfully clear (to me) when we where fixing all the Python 3000 tests.
I love doctest. Because it is very different in how it works, I don't think it competes with more formal testing methods.
Thanks for the link! I'll give 'nose' a try next time I write tests.
Yes, I looked at the decorator and think it is very nice and simple to use. If something like "nose" was a submodule of unittest, unittest may be able to use some of it's parts. Maybe gradually the more java like unittest classes and methods could be depreciated? Cheers, Ron

On Tue, 15 Jul 2008 09:26:45 am Raymond Hettinger wrote:
I am interested in this suggestion. I didn't know about py.test. I admit to dissatisfaction with unittest (too Java-ish and heavyweight for my tastes). I would love a test suite midway in weight between doctests and unittest, so I will check it out. -- Steven

"Steven D'Aprano" <steve@pearwood.info> writes:
I still think 'nose' is a better candidate for this: it appears to offer what people say they want from 'py.test', yet (unlike 'py.test') is integrated well with 'unittest'. -- \ “Pinky, are you pondering what I'm pondering?” “I think so, | `\ Brain, but what kind of rides do they have in Fabioland?” | _o__) —_Pinky and The Brain_ | Ben Finney

Steven D'Aprano <steve <at> pearwood.info> writes:
For what it's worth, I've been using nose for quite a long time and the first reason I did so is, like you, because I wanted to write tests in a light way (without having to declare classes). Then after writing some dozens of tests I switched back to wrapping tests in classes, just because it makes tests more readable and better organized (especially when you come to have setup/teardown functions shared by several tests). (but nose is still very nice)

Antoine Pitrou wrote:
(especially when you come to have setup/teardown functions shared by several tests).
These days, I tend to just write a context manager for common setup/teardown code rather than using the setUp/tearDown hooks (at least for Python's own test suite, where I have the luxury of assuming 2.5+ as the Python version). Where I find unittest.TestCase quite convenient is when I want to run the same set of tests with a few different settings - for those, putting the settings into class attributes and then inheriting from the basic test case and using different values is very convenient. This trick is particularly useful for testing that a class supports inheritance properly. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org

Antoine Pitrou <solipsis@pitrou.net> writes:
It's also entirely compatible with wrapping one's tests in classes. The test discovery and collection in 'nose' is one of the attractions: it discovers them at package, module, class, and plain-function level, whether doctest or not, whether unittest or not, and collects them all to run. -- \ “Well, my brother says Hello. So, hooray for speech therapy.” | `\ —Emo Philips | _o__) | Ben Finney

"Raymond Hettinger" <python@rcn.com> writes:
Agreed. I'm not proposing to have the unittest API change at all in Python 2.6 or 3.0. These changes, even the first deprecations, would not be suitable for anything earlier than 2.7 and 3.1. -- \ “When in doubt tell the truth. It will confound your enemies | `\ and astound your friends.” —Mark Twain, _Following the Equator_ | _o__) | Ben Finney

Michael Foord <fuzzyman@voidspace.org.uk> writes:
"Brett Cannon" <brett@python.org> writes:
Though I'm the champion of this PEP, I'll have to refer to Michael Foord for his reasoning (or reference to others' reasoning) on this. -- \ “The process by which banks create money is so simple that the | `\ mind is repelled.” —John Kenneth Galbraith, _Money: Whence It | _o__) Came, Where It Went_, 1975 | Ben Finney

Ben Finney wrote:
My reasoning goes something like this: self.assertLessThan(abs(self.base.method(parm1, parm2) - self.base.nominal), 2.54, 'Within an inch') vs. distance = self.base.method(parm1, parm2) deviation = self.base.method(parm1, parm2) - self.base.nominal self.assert_(abs(deviation) < 2.54, '%s is %s out of spec (more ' 'than an inch)' % (distance, deviation) It is not so much the assertion on values in variables, as it is assertions on results of calculations. Unfortunately, my tendency would be to forgo the "within an inch" over extracting the values into locals for the purpose of the test; the tests I'm writing currently focus on the manipulation of hardware for the reader, not on the extraction of data for the purpose of testing. Some of the longer sections are already nearly a couple of pages long; that's how much code it takes to do a coherent operation on this particular hardware. I hate that I am using 2 pages now (half a page per is what I'd prefer), and I'm not willing to bloat the operation with more code. As a result, I implemented my own versions of these asserts (Le, Lt, ...) a year or so ago, and still find them so useful that I'll re-implement them where- ever I am working without similar tests available. --Scott David Daniels Scott.Daniels@Acm.Org

Michael Foord <fuzzyman@voidspace.org.uk> writes:
assertSameElements(self, expected_seq, actual_seq, msg=None):
All these are new, so there is no existing expectation of these names from users of the standard library 'unittest' module (i.e. no backward-compatibility concern since these are new methods). If we're planning to deprecate the existing non-PEP-8 names in 2.7 and 3.1, why would we introduce new names that are non-PEP-8? Wouldn't it be better to add these as PEP-8 compatible names in the first instance? -- \ “You've got the brain of a four-year-old boy, and I'll bet he | `\ was glad to get rid of it.” —Groucho Marx | _o__) | Ben Finney

Michael Foord wrote:
Please, let's call this one "assertIsNot". I know it's valid Python to say if a not is b: but it's a much less natural way of expressing the condition, and (for all I know) might even introduce an extra negation operation. "is not" is, I believe, treated as a single operator.
[...]
regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/

Steve Holden <steve@holdenweb.com> writes:
Dang. You're exactly right. The problem is, that makes it quite inconsistent with other "not" uses (such as "assert_not_equal", "assert_not_in", etc.) I would really prefer that all these "not" uses be gramatically consistent for predictability. Is this a case where "assert_is_not" should exist alongside "assert_not_is"? I know that part of the goal here is to have "preferably only one obvious way to do it", but I can see *both* those names as "the obvious way to do it". Is this an instance where the "preferably" clause must be exercised in the negative? -- \ “Every sentence I utter must be understood not as an | `\ affirmation, but as a question.” —Niels Bohr | _o__) | Ben Finney

Ben Finney wrote:
If we can flip the word order in the language syntax, we can sure as heck flip it in a method name :) Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org

Nick Coghlan <ncoghlan@gmail.com> writes:
To be clear, I take it you're in favour of the following names (with no aliases): assert_equal assert_not_equal assert_is assert_is_not assert_in assert_not_in assert_almost_equal assert_not_almost_equal and so on; i.e. that 'assert_is_not' breaks the obvious pattern set by the others, in the interest of matching Python's 'is not' grammar. -- \ “Instead of having ‘answers’ on a math test, they should just | `\ call them ‘impressions’, and if you got a different | _o__) ‘impression’, so what, can't we all be brothers?” —Jack Handey | Ben Finney

Ben Finney wrote:
Well, I'd have said "in the interest of reading correctly in English", though I have to acknowledge this may not be an issue for many Python users whose first language not is English. "assert_not_is" is just dissonant to my ears. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/

Steve Holden wrote:
The two reasons aren't that far apart, given that Python's grammar uses "is not" because it makes more sense in English. One thing to remember is that the word 'is' is actually implied in all of the contracted phrases above other than those already including it explicitly. "x is equal to y" "x is not equal to y" "x is y" "x is not y" "x is in y" "x is not in y" "x is almost equal to y" "x is not almost equal to y" As for which phrasing I personally prefer, unit tests and method names are areas where I'm quite happy to paint the bike shed the same colour as the house :) Cheers, Nick. P.S. Deciphering that somewhat strained metaphor: I don't have a strong preference with regards to the unit test method names. While I tend to go with the assert* variants when left to my own devices, I have no problem sticking to the fail* variants when updating a test that uses them. Camel-case vs underscores in method names isn't something that particularly worries me either. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org

Steve Holden <steve@holdenweb.com> writes:
I'd count this as another (minor) point in favour of making the 'fail*' methods canonical: the names are consistent *and* gramatically sensible: fail_if_equal fail_unless_equal fail_if_is fail_unless_is fail_if_in fail_unless_in fail_if_almost_equal fail_unless_almost_equal -- \ “We are not gonna be great; we are not gonna be amazing; we are | `\ gonna be *amazingly* amazing!” —Zaphod Beeblebrox, _The | _o__) Hitch-Hiker's Guide To The Galaxy_, Douglas Adams | Ben Finney

Ben Finney wrote:
I'd count this as another (minor) point in favour of making the 'fail*' methods canonical: the names are consistent *and* gramatically sensible:
-1 I'm surprised nobody (that I've noticed) has brought up the point yet that test code is a lot easier to read if it makes positive assertions. When reading failure conditions, one has to constantly invert them in order to deduce the behaviour that is tested. failUnless and friends aren't better either IMO since while they do work with positive assertions, the method names themselves are doubly negative. assert* methods are so much more straightforward to comprehend. -- Thomas

Thomas Lotze wrote:
I think this is where I came in. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/

On Tue, 15 Jul 2008 04:55:59 pm Thomas Lotze wrote:
Please don't claim that your subjective opinion is an objective fact.
When reading failure conditions, one has to constantly invert them in order to deduce the behaviour that is tested.
You might have to. Don't assume that everyone else has your difficulty.
Maybe for you. That's not a human universal. Please don't assume that your favourite bike-shed colour must be the favourite colour of everyone else too. -- Steven

On Mon, Jul 14, 2008 at 11:55 PM, somebody wrote:
I'm surprised nobody (that I've noticed) has brought up the point yet that [...]
Not picking on whoever wrote that specifically, but if there's anything that surprises me, it's how many people have voiced opinions already (including many of them that I hadn't heard in this group before). There doesn't seem to be an end to this debate, and it is awfully close to deteriorating to pure bikeshedding and attempted ad-hominem attacks. I really don't have time to participate in detail, since all the time I have for Python I need to spend on trying to help review the 2.6 and 3.0 beta releases. But I want to remind people that radical changes to the unittest infrastructure will inconvenience many large 3rd party projects using Python, and I urge folks to look for ways to improve the unittest APIs in other ways instead. It's not the end of the world if the unittesting API uses assertEqual() instead of assert_equal() until the end of times. It would, however, be a shame if we couldn't agree to *add* a bunch of features, for example better reporting when two lists or long strings differ. -- --Guido van Rossum (home page: http://www.python.org/~guido/)

"Guido van Rossum" <guido@python.org> writes:
I intend to phrase such additions in terms of PEP-8-only names, so this name consolidation seems a natural prerequisite. -- \ “Are you pondering what I'm pondering?” “Well, I think so | `\ Brain, but what if we stick to the seat covers?” —_Pinky and | _o__) The Brain_ | Ben Finney

[Michael Foord]
[Steve Holden]
Please, let's call this one "assertIsNot".
+1
I know it's valid Python to say
if a not is b:
Nope, that's a syntax error.
"is not" and "not in" are both binary infix operators, not to be confused with the distinct use of "not" on its own as a unary prefix operator. "not is" and "in not" are both gibberish.

Tim Peters wrote:
Rats, naturally I was thinking of "if not (a is b):"
regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/

On Sun, 13 Jul 2008 23:51:44 +0100, Michael Foord <fuzzyman@voidspace.org.uk> wrote:
Several of these are implemented in other libraries (Twisted, at least). You might save some time by grabbing them and their unit tests, rather than re-implementing them. Twisted calls `assertIs´ `assertIdentical´, by the way.
Actually, I remember that someone raised an object to this as being not as flexible as some might want - an objection I agree with. Perhaps that was overruled, but I didn't want this to slip by as "not controversial".
The former of these is also in Twisted already, if you want to copy it. Jean-Paul

Michael Foord wrote:
Would it be worth Ben collating your current notes into a draft PEP targeting 2.7/3.1? Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org

Nick Coghlan <ncoghlan@gmail.com> writes:
Would it be worth Ben collating your current notes into a draft PEP targeting 2.7/3.1?
I'll do it and we'll find out. -- \ “A fine is a tax for doing wrong. A tax is a fine for doing | `\ well.” —anonymous | _o__) | Ben Finney

Michael Foord <fuzzyman@voidspace.org.uk> writes:
Thanks. I'm working these into another draft PEP that I hope to have up in a day or two. -- \ “[W]e are still the first generation of users, and for all that | `\ we may have invented the net, we still don't really get it.” | _o__) —Douglas Adams | Ben Finney

Raymond Hettinger wrote:
As proposed the changes don't remove or rename anything - so there will be no code breakage, just additional test methods. However, as we're into the beta phase I don't think these changes can make 2.6 / 3.0 anyway. Michael
-- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/ http://www.trypython.org/ http://www.ironpython.info/ http://www.theotherdelia.co.uk/ http://www.resolverhacks.net/

Michael Foord <fuzzyman@voidspace.org.uk> writes:
As proposed the changes don't remove or rename anything - so there will be no code breakage, just additional test methods.
Right, so I'm putting up a separate PEP just for the renaming. Should be arriving on this list soon.
However, as we're into the beta phase I don't think these changes can make 2.6 / 3.0 anyway.
Definitely agreed. -- \ “You can be a victor without having victims.” —Harriet Woods | `\ | _o__) | Ben Finney

From: "Ben Finney" <ben+python@benfinney.id.au>
Right, so I'm putting up a separate PEP just for the renaming. Should be arriving on this list soon.
I would like to work with you or someone else who is interested on an alternative PEP for a separate, simpler test module using the py.test syntax. That is much simpler to learn and use. Instead of self.assertIsNot and whatnot, you write: assert a is not b No need for tons of word-by-word spellings on things we already have syntax for. Almost anyone who has used py.test can attest its syntax is much more natural, easy to learn, easy to both read and write, and is much lighter weight. I think some variant of py.test could be done that is compatable with unittest and the did not have the "magic" present in earlier versions of py.test. I wrote a recipe (somewhat rough and incomplete) that shows how easily this could be done: http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/572194 Raymond

On Mon, Jul 14, 2008 at 6:26 PM, Raymond Hettinger <python@rcn.com> wrote:.
Bringing the total amount of test modules in the stdlib to 3. OWTDI indeed. Anyway, I don't think something like needs to be (re)written. nose[1] is already an excellent implementation of this that I would like to see in the stdlib. [1] http://www.somethingaboutorange.com/mrl/projects/nose/ -- Cheers, Benjamin Peterson "There's no place like 127.0.0.1."


Raymond Hettinger wrote:
However, to provide readable output for errors in even simple tests (like a == b) py.test does magic with stack frames and code objects - in order to discover the objects being compared. As this relies on what are essentially implementation details of the Python interpreter it means that some implementations (specifically IronPython which doesn't have Python stack frames and only a minimal representation of frame objects) will never be able to run it. I think it would be a bad idea to move *Python testing* itself over to a framework like this. I personally find unittest pretty readable, the feature most lacking is autodiscovery of tests which nose does seem to provide very well although I haven't used it yet. Michael
-- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/ http://www.trypython.org/ http://www.ironpython.info/ http://www.theotherdelia.co.uk/ http://www.resolverhacks.net/

From: "Michael Foord" <fuzzyman@voidspace.org.uk>
Don't have to go that route. Can use plain python assert failures with a stacktrace. Or can trigger pdb, or let the user specify a mode that calls some more advanced test runner or test reporter with introspection. This can be done without making everything hard.
I think it would be a bad idea to move *Python testing* itself over to a framework like this.
Don't want to convert the python testing. Would like to offer a lighter-weight alternative to our users.
It takes about one day of using py.test to realize have much cleaner and more readable its syntax is. Also, writing the tests is *much* more pleasant. It has the same clean, clear joy as writing regular python code. By comparison, the code using unittest.py is javaesque. I've written tons of test with unittest.py and and find it to be joyless. I realize there is a matter of taste involved but if you talk to any regular users of py.test, they will *all* attest to the syntax being much more readable, lightweight, and pleasant to use. It encourages writing tests. That being said, I think there are less magical, much simpler ways to implement it. I think Holger is working on it as we speak. Raymond

On Mon, Jul 14, 2008 at 5:13 PM, Raymond Hettinger <python@rcn.com> wrote:
I, too, have written tons of tests with unittest.py (and Google's extensions, which follow the same style), and reviewed even more. I agree that this is pretty joyless, but I'm not at all sure that the unittest API is the reason. It seems to me that a main problem with writing test code is and will always remain due to the need to use mocks, stubs and other similar techniques (e.g. dependency injection). Typical test code that I've written or reviewed spends more time setting up the input conditions for testing than it spends checking the results. Ten lines of mocking code to one self.assertEqual() call seems typical. -- --Guido van Rossum (home page: http://www.python.org/~guido/)

Guido van Rossum wrote:
Maybe Python needs a good mocking module in the standard library. There are plenty, but we use a particularly nice one at Resolver Systems [1]. :-) It auto-creates attributes as mocks, allowing you to assert calls made to all of its children along with convenience methods like 'assert_called_with' and has a companion decorator that patches class / module level attributes just for the duration of the test. As we're changing more of our tests over to use these we're finding it reduces the volume and complexity of our test code. Michael Foord [1] Based on http://code.google.com/p/mock/ although there is some outstanding code to sync back to the project. -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/ http://www.trypython.org/ http://www.ironpython.info/ http://www.theotherdelia.co.uk/ http://www.resolverhacks.net/

From: "Michael Foord" <fuzzyman@voidspace.org.uk>
Maybe Python needs a good mocking module in the standard library. There are plenty, but we use a particularly nice one at Resolver Systems [1]. :-)
-1 This comes up occassionally and gets shot down. http://bugs.python.org/issue708125 Mock objects mean different things to different people. Some expect more simulated behavior and others want less. It's rare to find agreement about general purpose mock objects and frameworks. Mock libraries create their own complexities and burdens on a programmer's memory. It's often easier to create a small special case mock object than to remember how to configure a general purpose one. And, afaict, there is no fan club for some particular python mock object library -- it seems to only come up in general discussions about possibilities for growing the unittest module, and almost never comes up in the context of solving a real problem that hasn't already be addressed in some other way. Raymond

On Mon, Jul 14, 2008 at 09:37:30PM -0700, Raymond Hettinger wrote: -> From: "Michael Foord" <fuzzyman@voidspace.org.uk> -> >Maybe Python needs a good mocking module in the standard library. There -> >are plenty, but we use a particularly nice one at Resolver Systems [1]. :-) -> -> -1 -> -> This comes up occassionally and gets shot down. -> http://bugs.python.org/issue708125 -> -> Mock objects mean different things to different people. -> Some expect more simulated behavior and others want less. -> It's rare to find agreement about general purpose mock objects and -> frameworks. -> Mock libraries create their own complexities and burdens on a programmer's -> memory. -> It's often easier to create a small special case mock object -> than to remember how to configure a general purpose one. -> And, afaict, there is no fan club for some particular python mock -> object library -- it seems to only come up in general discussions -> about possibilities for growing the unittest module, and almost -> never comes up in the context of solving a real problem that -> hasn't already be addressed in some other way. Also see: http://lists.idyll.org/pipermail/testing-in-python/2007-November/000406.html & associated thread, for those interested in the variety of mock libraries... cheers, --titus -- C. Titus Brown, ctb@msu.edu

And: http://bugs.python.org/issue2156 Raymond

On Mon, Jul 14, 2008 at 6:43 PM, Michael Foord <fuzzyman@voidspace.org.uk> wrote:
Maybe what we need to do then is make the assert statement more powerful. I like the idea of having it call a builtin called __assert__ which is called by the assert statement. The AST for the node could be attached. -- Cheers, Benjamin Peterson "There's no place like 127.0.0.1."

Raymond Hettinger wrote:
Ah, in my haste I skipped over your comment about "magic", my apologies. But in the absence of magic how do you propose to provide a meaningful error message from the failure of: assert a == b To wrap it in a function like "assert equals(a, b)" seems to gain little over unittest. Michael
-- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/ http://www.trypython.org/ http://www.ironpython.info/ http://www.theotherdelia.co.uk/ http://www.resolverhacks.net/

Michael Foord wrote:
Aside from the question of providing nice error messages, two questions that immediately come to mind for me are: - how do I run my tests with -O or -OO? (since the compiler will throw all the assert statements away before any Python code gets a chance to look at them) - how do I test that code raises an expected exception? - how do I explicitly fail a test case? (e.g. I'll often do this when I want to test an operation with a variety of different inputs - I'll test for all of the inputs of interest, collecting the failures in a list, then reporting a single error message at the end detailing all of the cases that failed) And I've also never had any problem whatsoever debugging unit tests with print statements - one of the effects of the -v switch is to display anything which is written to stderr/stdout on the console again. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org

Raymond Hettinger wrote:
+1 for a simpler testing module. Just letting you know there is interest in a lighter weight testing suite. Looking at the unittest discussions, it doesn't look like it is getting easier to use, but more complex. Py.test looks very interesting, especially the test discovery parts. I also agree we don't need special methods for every variation of assertedness. I've been thinking that a few decorators may go a long way to making writing tests easy. It also reduces the level of indentation needed. Ron

Ron Adam <rrr@ronadam.com> writes:
+1 for a simpler testing module.
I've no objection.
Just letting you know there is interest in a lighter weight testing suite.
'doctest' is a very simple testing module, that is a very useful tool.
Looking at the unittest discussions, it doesn't look like it is getting easier to use, but more complex.
How so? One PEP proposed this week specifies to do nothing but conform 'unittest' with the standard library guidelines, and remove redundant names. That surely makes it simpler to use. Another PEP specifies to add helper methods that simplify a number of common cases. What is it you see making unittest "more complex to use"?
My main complaint about 'py.test' is that it's yet-another-framework. We already have 'doctest' and 'unittest', and they play together reasonably well. 'nose' <URL:http://somethingaboutorange.com/mrl/projects/nose/> looks better for consideration, especially since it integrates well with 'unittest'.
I've been thinking that a few decorators may go a long way to making writing tests easy. It also reduces the level of indentation needed.
There are a number of these already in 'nose'. -- \ “I fly Air Bizarre. You buy a combination one-way round-trip | `\ ticket. Leave any Monday, and they bring you back the previous | _o__) Friday. That way you still have the weekend.” —Steven Wright | Ben Finney

Ben Finney wrote:
No complaint here. It's a good place to start.
Another PEP specifies to add helper methods that simplify a number of common cases.
In my opinion adding 19 more methods makes it more complex. I'd rather see more focus on handling test failures in general without the need for so many special helper functions.
What is it you see making unittest "more complex to use"?
More methods and method signatures to learn and remember. assert_true(…) assert_false(…) assert_almost_equal(…) assert_equal(…) assert_not_almost_equal(…) assert_not_equal(…) assert_raises(exc_class, callable_obj, *args, **kwargs) assert_compare_true(op, first, second, msg=None) assert_in(container, member, msg=None) assert_is(first, second, msg=None) assert_less_than(first, second, msg=None) assert_greater_than(first, second, msg=None) assert_less_than_or_equal(first, second, msg=None) assert_greater_than_or_equal(first, second, msg=None) assert_members_equal(first, second, msg=None) assert_sequence_equal(first, second, msg=None) assert_raises_with_message_regex(exc_class, message_regex, callable_obj, *args, **kwargs) assert_compare_false(op, first, second, msg=None) assert_not_in(container, member, msg=None) assert_is_not(first, second, msg=None) assert_not_less_than(first, second, msg=None) assert_not_greater_than(first, second, msg=None) assert_not_less_than_or_equal(first, second, msg=None) assert_not_greater_than_or_equal(first, second, msg=None) assert_members_not_equal(first, second, msg=None) assert_sequence_not_equal(first, second, msg=None) That comes to 26 variations of assert. There are really only a small set of basic conditions to test for. correct values incorrect values unexpected exceptions correct exceptions incorrect exceptions missing exceptions I think the unittest module could better handle testing of exceptions and distinguishing exception produced by test code from the code being tested. That was painfully clear (to me) when we where fixing all the Python 3000 tests.
I love doctest. Because it is very different in how it works, I don't think it competes with more formal testing methods.
Thanks for the link! I'll give 'nose' a try next time I write tests.
Yes, I looked at the decorator and think it is very nice and simple to use. If something like "nose" was a submodule of unittest, unittest may be able to use some of it's parts. Maybe gradually the more java like unittest classes and methods could be depreciated? Cheers, Ron

On Tue, 15 Jul 2008 09:26:45 am Raymond Hettinger wrote:
I am interested in this suggestion. I didn't know about py.test. I admit to dissatisfaction with unittest (too Java-ish and heavyweight for my tastes). I would love a test suite midway in weight between doctests and unittest, so I will check it out. -- Steven

"Steven D'Aprano" <steve@pearwood.info> writes:
I still think 'nose' is a better candidate for this: it appears to offer what people say they want from 'py.test', yet (unlike 'py.test') is integrated well with 'unittest'. -- \ “Pinky, are you pondering what I'm pondering?” “I think so, | `\ Brain, but what kind of rides do they have in Fabioland?” | _o__) —_Pinky and The Brain_ | Ben Finney

Steven D'Aprano <steve <at> pearwood.info> writes:
For what it's worth, I've been using nose for quite a long time and the first reason I did so is, like you, because I wanted to write tests in a light way (without having to declare classes). Then after writing some dozens of tests I switched back to wrapping tests in classes, just because it makes tests more readable and better organized (especially when you come to have setup/teardown functions shared by several tests). (but nose is still very nice)

Antoine Pitrou wrote:
(especially when you come to have setup/teardown functions shared by several tests).
These days, I tend to just write a context manager for common setup/teardown code rather than using the setUp/tearDown hooks (at least for Python's own test suite, where I have the luxury of assuming 2.5+ as the Python version). Where I find unittest.TestCase quite convenient is when I want to run the same set of tests with a few different settings - for those, putting the settings into class attributes and then inheriting from the basic test case and using different values is very convenient. This trick is particularly useful for testing that a class supports inheritance properly. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org

Antoine Pitrou <solipsis@pitrou.net> writes:
It's also entirely compatible with wrapping one's tests in classes. The test discovery and collection in 'nose' is one of the attractions: it discovers them at package, module, class, and plain-function level, whether doctest or not, whether unittest or not, and collects them all to run. -- \ “Well, my brother says Hello. So, hooray for speech therapy.” | `\ —Emo Philips | _o__) | Ben Finney

"Raymond Hettinger" <python@rcn.com> writes:
Agreed. I'm not proposing to have the unittest API change at all in Python 2.6 or 3.0. These changes, even the first deprecations, would not be suitable for anything earlier than 2.7 and 3.1. -- \ “When in doubt tell the truth. It will confound your enemies | `\ and astound your friends.” —Mark Twain, _Following the Equator_ | _o__) | Ben Finney

Michael Foord <fuzzyman@voidspace.org.uk> writes:
"Brett Cannon" <brett@python.org> writes:
Though I'm the champion of this PEP, I'll have to refer to Michael Foord for his reasoning (or reference to others' reasoning) on this. -- \ “The process by which banks create money is so simple that the | `\ mind is repelled.” —John Kenneth Galbraith, _Money: Whence It | _o__) Came, Where It Went_, 1975 | Ben Finney

Ben Finney wrote:
My reasoning goes something like this: self.assertLessThan(abs(self.base.method(parm1, parm2) - self.base.nominal), 2.54, 'Within an inch') vs. distance = self.base.method(parm1, parm2) deviation = self.base.method(parm1, parm2) - self.base.nominal self.assert_(abs(deviation) < 2.54, '%s is %s out of spec (more ' 'than an inch)' % (distance, deviation) It is not so much the assertion on values in variables, as it is assertions on results of calculations. Unfortunately, my tendency would be to forgo the "within an inch" over extracting the values into locals for the purpose of the test; the tests I'm writing currently focus on the manipulation of hardware for the reader, not on the extraction of data for the purpose of testing. Some of the longer sections are already nearly a couple of pages long; that's how much code it takes to do a coherent operation on this particular hardware. I hate that I am using 2 pages now (half a page per is what I'd prefer), and I'm not willing to bloat the operation with more code. As a result, I implemented my own versions of these asserts (Le, Lt, ...) a year or so ago, and still find them so useful that I'll re-implement them where- ever I am working without similar tests available. --Scott David Daniels Scott.Daniels@Acm.Org
participants (18)
-
Antoine Pitrou
-
Ben Finney
-
Ben Finney
-
Benjamin Peterson
-
C. Titus Brown
-
Guido van Rossum
-
Jean-Paul Calderone
-
Jonathan Lange
-
Michael Foord
-
Nick Coghlan
-
Raymond Hettinger
-
Ron Adam
-
Scott David Daniels
-
Steve Holden
-
Steven D'Aprano
-
Terry Reedy
-
Thomas Lotze
-
Tim Peters