Overloading operators for testing
I am using declarative testing a lot and I found out why unit tests are so clunky. The reason why assertEquals(a,b) is used is because if we put `assert a==b` then nose can catch the AssertionError but wont find out what was returned or expected. This could be easily overcome if we allow oveloading == operator from outside. Right now == would have to be changed for every lefhand object that is compared in the tests, builtin types including. We could use a way to change it from above, so to speak. Consider this: def __glob_eq__(a,b): if not a == b: raise FoundInequalityError(a,b) return True assert obj1 == obj2 #<-- using eq above Nose could easily catch FoundInequalityError and print whatever assertEquals would. This goes very handy when you consider declarative unit testing that I use in my project. I have a unitest.TestCase derivative and the actual testcase has a method that yields individual comparisons, like this: class TestBinary(declarativeunittest.TestCase): def alltestsinteractive(self): yield [func(1) == 2] shuffle(alist) yield [sorted(alist) == [1,2,3]] Notice that this allows to put imperative statements in between declarative cases. So shuffled() is no longer necessary in this code. :) pozdrawiam, Arkadiusz Bulski
FYI, pytest already does this:
http://doc.pytest.org/en/latest/
--
Ryan
[ERROR]: Your autotools build scripts are 200 lines longer than your
program. Something’s wrong.
http://kirbyfan64.github.io/
On Sep 17, 2016 7:55 PM, "Arek Bulski"
I am using declarative testing a lot and I found out why unit tests are so clunky. The reason why assertEquals(a,b) is used is because if we put `assert a==b` then nose can catch the AssertionError but wont find out what was returned or expected. This could be easily overcome if we allow oveloading == operator from outside. Right now == would have to be changed for every lefhand object that is compared in the tests, builtin types including. We could use a way to change it from above, so to speak. Consider this:
def __glob_eq__(a,b): if not a == b: raise FoundInequalityError(a,b) return True
assert obj1 == obj2 #<-- using eq above
Nose could easily catch FoundInequalityError and print whatever assertEquals would. This goes very handy when you consider declarative unit testing that I use in my project. I have a unitest.TestCase derivative and the actual testcase has a method that yields individual comparisons, like this:
class TestBinary(declarativeunittest.TestCase): def alltestsinteractive(self):
yield [func(1) == 2] shuffle(alist) yield [sorted(alist) == [1,2,3]]
Notice that this allows to put imperative statements in between declarative cases. So shuffled() is no longer necessary in this code. :)
pozdrawiam, Arkadiusz Bulski
_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Arek Bulski wrote:
def __glob_eq__(a,b): if not a == b: raise FoundInequalityError(a,b) return True
assert obj1 == obj2 #<-- using eq above
How would you ensure that this overriding only applied in the places you want it? You don't want to change the meaning of == in the code under test! Related to that, how would you prevent the use of == in the definition of __glob_eq__ above from triggering infinite recursion? -- Greg
On Sun, Sep 18, 2016 at 02:52:31AM +0200, Arek Bulski wrote:
I am using declarative testing a lot and I found out why unit tests are so clunky.
I don't think unit tests are clunky.
The reason why assertEquals(a,b) is used is because if we put `assert a==b` then nose can catch the AssertionError but wont find out what was returned or expected.
That's not the only reason. assert* methods can take an arbitrary number of arguments. == cannot. It is easy to extend the collection of assert* methods, to use inheritance to modify them, and so on. Not so easy if you have only a single, global function or operator. assert* methods can run even when assertions are disabled.
This could be easily overcome if we allow oveloading == operator from outside. Right now == would have to be changed for every lefhand object that is compared in the tests, builtin types including. We could use a way to change it from above, so to speak.
Having global state that controls the behaviour of == is not a good idea. Look at your proposed code:
Consider this:
def __glob_eq__(a,b): if not a == b: raise FoundInequalityError(a,b) return True
assert obj1 == obj2 #<-- using eq above
That can't work as you have written it, because it will recurse forever. obj1 == obj2 will call the global eq, which calls == which calls the global eq, which calls == which calls the global eq. So let's re-write it: def __glob_eq__(a,b): saved_state = get_global_eq() delete_global_eq() try: if not a == b: # calls the real == raise FoundInequalityError(a,b) return True finally: set_global_eq(saved_state) Okay, that's pretty yucky, and its not thread-safe, but at least its only in one place, right? Except it isn't. Every single call to == (and != ) will have to be re-written to do the same thing. And of course, that just replaces assertEquals. What about all the other assert* methods? But there's another, more fundamental problem with using assert in this way. You cannot run your unit tests with assertions turned off. That is why I think the nose style of using "assert a == b" for unit testing is seriously broken. That is an abuse of the assert statement. It's good enough for a handful of quick and dirty tests, but is not good enough for a full-sized unit test suite. -- Steve
As Ryan points out, pytest does this right. The way I understand it, pytest is actively maintained and nose isn't. You should switch to pytest as soon as possible. Best, Neil On Saturday, September 17, 2016 at 8:55:43 PM UTC-4, Arek Bulski wrote:
I am using declarative testing a lot and I found out why unit tests are so clunky. The reason why assertEquals(a,b) is used is because if we put `assert a==b` then nose can catch the AssertionError but wont find out what was returned or expected. This could be easily overcome if we allow oveloading == operator from outside. Right now == would have to be changed for every lefhand object that is compared in the tests, builtin types including. We could use a way to change it from above, so to speak. Consider this:
def __glob_eq__(a,b): if not a == b: raise FoundInequalityError(a,b) return True
assert obj1 == obj2 #<-- using eq above
Nose could easily catch FoundInequalityError and print whatever assertEquals would. This goes very handy when you consider declarative unit testing that I use in my project. I have a unitest.TestCase derivative and the actual testcase has a method that yields individual comparisons, like this:
class TestBinary(declarativeunittest.TestCase): def alltestsinteractive(self):
yield [func(1) == 2] shuffle(alist) yield [sorted(alist) == [1,2,3]]
Notice that this allows to put imperative statements in between declarative cases. So shuffled() is no longer necessary in this code. :)
pozdrawiam, Arkadiusz Bulski
On 21/09/2016 00:14, Neil Girdhar wrote:
As Ryan points out, pytest does this right. The way I understand it, pytest is actively maintained and nose isn't. You should switch to pytest as soon as possible.
Best,
Neil
Nose is no longer maintained but long live nose2 https://pypi.python.org/pypi/nose2/0.6.5 https://github.com/nose-devs/nose2 -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence
participants (6)
-
Arek Bulski -
Greg Ewing -
Mark Lawrence -
Neil Girdhar -
Ryan Gonzalez -
Steven D'Aprano