where statement in Pyret
Hey I've read about Pyret on hackernews: http://www.pyret.org/ and found the 'where' statement very compeling. Functions can end with a where that contains small unit tests.
From the documentation example:
fun sum(l): cases(List) l: | empty => 0 | link(first, rest) => first + sum(rest) end where: sum([]) is 0 sum([1, 2, 3]) is 6 end It's quite similar to the doctests ideas I guess - but not intended to be documentation like them. I ended up disliking docttests because of this doc+test duality by the way: it often ends up as a not so good documentation and not so good tests. Anyways, having a dedicated keyword to append after a function some tests as part of the language has benefits imho: - the scope is reduced to the function - so it helps making 'real' isolated unit tests. - we do have the unittest conventions, but here it make tests a first class citizen in the language. Cheers Tarek
While it is a nice idea, i don't think this feature deserves its own syntax. Besides doctests, another way of achieving this in Python might be: def sum(l): # implementation of sum def test_sum(): assert sum([]) == 0 assert sum([1, 2, 3]) == 6 which IMO is nice enough. -- Markus "Tarek Ziadé" <tarek@ziade.org> wrote:
Le 11/10/13 11:00 AM, Markus Unterwaditzer a écrit :
And even if we place the test function just besides the tested function, Python will not make any distinction : they are both just functions. Having the ability to distinguish tests and regular code at the language level has benefits like the ability to ignore tests when you run the app in production etc. Cheers Tarek
I agree that such hints about the tests would be nice for performance, but i don't think considerations about performance were ever given a high priority during the design of Python. How about this: def sum(): # implementation @sum.test def test_sum(): assert sum([]) == 0 This potentially could do the same things you expected from the where-statement, without introducing new keywords. Maybe this could also be buried into some stdlib module in order to avoid "polluting" the core language if this way of writing tests doesn't gain any traction. import unittest # or whatever def sum(): # implementation @unittest.test_func(sum): def test_sum(): assert sum([]) == 0 -- Markus On 2013-11-10 11:06, Tarek Ziadé wrote:
On 11/10/2013 5:06 AM, Tarek Ziadé wrote:
To put it another way, we already have syntax support for testing: assert, which is sugar for conditional stateements, but with extra features; and functions, which are executed when called. Like most suggested new keywords, 'where' is certain to by in use already as an identifier. So we need a really good reason, with no better alternative, to make it a keyword.
[test_xxx functions]
As others have noted, one big reason for the convention is that testing class methods usually requires big chunks of code that are better isolated in another file. However, the convention is not a rule, and for pure module-level functions that run in isolation, one is free to include tests in the doc string or just after. See example below.
And even if we place the test function just besides the tested function, Python will not make any distinction : they are both just functions.
Functions are fine. Distinction is easily done by an obvious name convention.
Functions only run when called. For my book, most example code consists of classical functions. For these, I am doing the following, adapted for your sum example. For didactic reasons, I like having the tests immediately follow the function code; the input-output pairs serve as testable documentation. (This code does not run at the moment as I am midstream in changing the test module.) The first and last statements are boilerplate that is part of a _template.py file. ---- from xploro.test import main, ftest def sum_rec(seq): if seq: return seq[0] + sum_rec(seq[1:]) else: return 0 def sum_for(seq): ret = 0 for num in seq: ret += num return ret def test_sum(): ftest((sum_rec, sum_for), (([], 0), ([1], 1), ([1,2,3], 6),) ) if __name__ == '__main__': main() ---- ftest calls each function with each input of the input-output pairs and checks that the function output matches the output given. main scans globals() for functions named 'test_xxx' and calls them. Anyway, I prefer the above to the 'where' suggestion. -- Terry Jan Reedy
Le 11/11/13 2:21 AM, Terry Reedy a écrit :
In other words, a program with tests functions in your code will be different in memory that the same program without tests functions.
That's a good template indeed, I have 3 remarks though: 1/ you are importing your test framework even if you don't run the tests. 2/ those are not pure unit tests, since test_sum() tests 2 separate functions - but I guess this rule can be broken in this case. 3/ main() is an optional artifact from unittest - most developers your a script that takes care of the test discovery (unittest2, nosetests, etc) So I would rather write in plain python: ---- def sum_rec(seq): if seq: return seq[0] + sum_rec(seq[1:]) else: return 0 def sum_for(seq): ret = 0 for num in seq: ret += num return ret def test_sum(): from xploro.test import main, ftest ftest((sum_for, sum_rec), (([], 0), ([1], 1), ([1,2,3], 6),) ) ---- Cheers Tarek
On Sun, Nov 10, 2013 at 10:55:16AM +0100, Tarek Ziadé wrote:
Hey
I've read about Pyret on hackernews: http://www.pyret.org/
Looks very interesting.
Sadly, I *really* dislike that. To me, "where" has absolutely nothing to do with testing. I see that Pyret also includes a "check" keyword which also does testing. That seems like a more sensible keyword. I would prefer to see some variation on Nick Coglan's ideas about a "where" keyword for local scoping of temporary variables. This is an idea that's been floating around for a long time: https://mail.python.org/pipermail/python-list/2005-January/329539.html Quite frankly, although I believe that tests are of vital importance, I remain to be convinced whether they should be "a first-class citizen of the language" as you put it. In my experience, for every line of code you write, you'll probably need anything from 3-10 lines of test code. I don't think that much test code belongs in the same module as the function itself -- that's an invitation to have people stint on their testing. Look at the example given -- do you really feel that this is sufficient testing for the sum() function? I don't object to having a few, simple, fast tests in the main module, but for "real" unit testing and regression testing, they ought to be moved out into a separate file. Tests are often, usually, boring code which just distracts from the code you care about. I would hate to see it become common to have something like this: def some_function(): # 20 lines of code where: # 170 lines of tests become standard. But in any case, I think it is far too early to be thinking about stealing an idea from Pyret. The language isn't even stable yet, let alone at the stage where this idea has proven itself in the real world. Pyret and the where clause is still experimental.
In my experience, people who write poor doctests would write equally poor unit tests, or "where" tests if Python had this feature.
I don't understand this comment. In what way are unit tests from the unittest module, or doctests, not "real" unit tests? -- Steven
Steven D'Aprano wrote:
I would prefer to see some variation on Nick Coglan's ideas about a "where" keyword for local scoping of temporary variables.
I second that. This is the way mathematicians use the word "where", and it would be a much better way of spending a keyword.
In my experience, for every line of code you write, you'll probably need anything from 3-10 lines of test code.
Indeed. Also, the situations where you can meaningfully test each function on its own using a few concisely- expressed test cases are relatively rare, IMO. It looks good for the kind of exercises you find in programming courses, but it doesn't scale up to real-life code that requires complex data structures and test harnesses to be set up. -- Greg
On 10 November 2013 19:55, Tarek Ziadé <tarek@ziade.org> wrote:
It would make more sense to just bake py.test style rich assertions into the language in some way and let people write: def sum(iterable): # implementation of sum assert sum([]) == 0 assert sum([1, 2, 3]) == 6 A mechanism to say "always execute assert statements in this module regardless of optimisation level" could also be useful. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
Le 10/11/2013 15:48, Nick Coghlan a écrit :
This has the same problem as doctests: it works well for trivial tests like the above, but will be difficult to scale towards more complicated testing. unittest-like structuration is really what works best for most testing situations, IMO. Alternative testing schemes for "easier" or "more intuitive" testing have generally failed as general-purpose tools. Regards Antoine.
On 11 Nov 2013 02:12, "Antoine Pitrou" <solipsis@pitrou.net> wrote:
like the above, but will be difficult to scale towards more complicated testing. Yeah, part of my point was actually that module level assertions allow this kind of thing today, and there are good reasons people don't do it in practice.
Agreed, but the spelling of test assertions as methods on test cases isn't an essential part of that structure. For 3.5, it would be nice to offer either testtools style "matcher" objects or py.test style always-on-in-test-modules rich assertions. Either approach has a real world base to draw from, but would still involve a fair bit of work (they're also not mutually exclusive - a matcher based approach could be used as the back end for rich assertions). Cheers, Nick.
On Mon, Nov 11, 2013 at 09:58:06AM +0100, Tarek Ziadé wrote:
I often spell that: if __debug__: assert sum([]) == 0 # more complicated testing here I've been playing around with doctest, and I think it may be useful to have a "check" decorator that you use like this: @check def spam(): """Return spam. >>> spam() 'spam spam spam' """ return ' '.join(['spam']*3) If the doctests pass, the function is returned unchanged, otherwise an exception is raised. Note that this will encourage a specific style of more limited tests focused on just one function at a time. Normally, doctests aren't run until after the entire module is loaded, which lets you do things like this: def spam(): """Return spam. >>> spam() 'spam spam spam' Like ham, but more tasty: >>> taste(spam()) > taste(ham()) True """ return ' '.join(['spam']*3) # definitions of taste and ham follow later in the module. That *won't work* with a check decorator, since at the time the decorator runs, the functions taste and ham don't exist. The same would apply to a "where" clause (or whatever name it is given). This forces the doc tests to be more tightly focused on the function in isolation, which would be both good and bad. The good is that it would discourage overloading the docstring with too many too complex tests. The bad is that it would limit what can be tested this way, and the errors would no doubt be confusing to beginners. "What do you mean NameError? taste is defined right there..." On balance, I think that such a check() decorator would be useful enough that it's worth my time writing it. (Perhaps not useful enough to include in doctest, but I'll certainly stick it in my own personal toolbox.) But I don't think it would be useful enough to make it syntax. -- Steven
On Mon, 11 Nov 2013 07:57:21 +1000 Nick Coghlan <ncoghlan@gmail.com> wrote:
That's true, OTOH I disagree that assertions as methods is any kind of hindrance to easy testing. Actually, that paradigm makes it trivial to define your own assertions in a way that makes them look like the built-in ones. Regards Antoine.
On 11/10/2013 08:48 AM, Nick Coghlan wrote:
A mechanism to say "always execute assert statements in this module regardless of optimisation level" could also be useful.
+ 1 Currently assert statements are removed if the -O flag is used, but somehow that never seemed quite right to me. It makes more sense to have a -A option to turn them on, rather than using -O to turn them off. That change *along with* a way to "always execute asserts in this scope" would make asserts more useful. Cheers, Ron
On Sun, Nov 10, 2013 at 10:37 AM, Ron Adam <ron3200@gmail.com> wrote:
That change *along with* a way to "always execute asserts in this scope" would make asserts more useful.
assert not test_mode or sum([]) == 0 assert not test_mode or sum([1, 2, 3]) == 6 -- Keeping medicines from the bloodstreams of the sick; food from the bellies of the hungry; books from the hands of the uneducated; technology from the underdeveloped; and putting advocates of freedom in prisons. Intellectual property is to the 21st century what the slave trade was to the 16th.
While it is a nice idea, i don't think this feature deserves its own syntax. Besides doctests, another way of achieving this in Python might be: def sum(l): # implementation of sum def test_sum(): assert sum([]) == 0 assert sum([1, 2, 3]) == 6 which IMO is nice enough. -- Markus "Tarek Ziadé" <tarek@ziade.org> wrote:
Le 11/10/13 11:00 AM, Markus Unterwaditzer a écrit :
And even if we place the test function just besides the tested function, Python will not make any distinction : they are both just functions. Having the ability to distinguish tests and regular code at the language level has benefits like the ability to ignore tests when you run the app in production etc. Cheers Tarek
I agree that such hints about the tests would be nice for performance, but i don't think considerations about performance were ever given a high priority during the design of Python. How about this: def sum(): # implementation @sum.test def test_sum(): assert sum([]) == 0 This potentially could do the same things you expected from the where-statement, without introducing new keywords. Maybe this could also be buried into some stdlib module in order to avoid "polluting" the core language if this way of writing tests doesn't gain any traction. import unittest # or whatever def sum(): # implementation @unittest.test_func(sum): def test_sum(): assert sum([]) == 0 -- Markus On 2013-11-10 11:06, Tarek Ziadé wrote:
On 11/10/2013 5:06 AM, Tarek Ziadé wrote:
To put it another way, we already have syntax support for testing: assert, which is sugar for conditional stateements, but with extra features; and functions, which are executed when called. Like most suggested new keywords, 'where' is certain to by in use already as an identifier. So we need a really good reason, with no better alternative, to make it a keyword.
[test_xxx functions]
As others have noted, one big reason for the convention is that testing class methods usually requires big chunks of code that are better isolated in another file. However, the convention is not a rule, and for pure module-level functions that run in isolation, one is free to include tests in the doc string or just after. See example below.
And even if we place the test function just besides the tested function, Python will not make any distinction : they are both just functions.
Functions are fine. Distinction is easily done by an obvious name convention.
Functions only run when called. For my book, most example code consists of classical functions. For these, I am doing the following, adapted for your sum example. For didactic reasons, I like having the tests immediately follow the function code; the input-output pairs serve as testable documentation. (This code does not run at the moment as I am midstream in changing the test module.) The first and last statements are boilerplate that is part of a _template.py file. ---- from xploro.test import main, ftest def sum_rec(seq): if seq: return seq[0] + sum_rec(seq[1:]) else: return 0 def sum_for(seq): ret = 0 for num in seq: ret += num return ret def test_sum(): ftest((sum_rec, sum_for), (([], 0), ([1], 1), ([1,2,3], 6),) ) if __name__ == '__main__': main() ---- ftest calls each function with each input of the input-output pairs and checks that the function output matches the output given. main scans globals() for functions named 'test_xxx' and calls them. Anyway, I prefer the above to the 'where' suggestion. -- Terry Jan Reedy
Le 11/11/13 2:21 AM, Terry Reedy a écrit :
In other words, a program with tests functions in your code will be different in memory that the same program without tests functions.
That's a good template indeed, I have 3 remarks though: 1/ you are importing your test framework even if you don't run the tests. 2/ those are not pure unit tests, since test_sum() tests 2 separate functions - but I guess this rule can be broken in this case. 3/ main() is an optional artifact from unittest - most developers your a script that takes care of the test discovery (unittest2, nosetests, etc) So I would rather write in plain python: ---- def sum_rec(seq): if seq: return seq[0] + sum_rec(seq[1:]) else: return 0 def sum_for(seq): ret = 0 for num in seq: ret += num return ret def test_sum(): from xploro.test import main, ftest ftest((sum_for, sum_rec), (([], 0), ([1], 1), ([1,2,3], 6),) ) ---- Cheers Tarek
On Sun, Nov 10, 2013 at 10:55:16AM +0100, Tarek Ziadé wrote:
Hey
I've read about Pyret on hackernews: http://www.pyret.org/
Looks very interesting.
Sadly, I *really* dislike that. To me, "where" has absolutely nothing to do with testing. I see that Pyret also includes a "check" keyword which also does testing. That seems like a more sensible keyword. I would prefer to see some variation on Nick Coglan's ideas about a "where" keyword for local scoping of temporary variables. This is an idea that's been floating around for a long time: https://mail.python.org/pipermail/python-list/2005-January/329539.html Quite frankly, although I believe that tests are of vital importance, I remain to be convinced whether they should be "a first-class citizen of the language" as you put it. In my experience, for every line of code you write, you'll probably need anything from 3-10 lines of test code. I don't think that much test code belongs in the same module as the function itself -- that's an invitation to have people stint on their testing. Look at the example given -- do you really feel that this is sufficient testing for the sum() function? I don't object to having a few, simple, fast tests in the main module, but for "real" unit testing and regression testing, they ought to be moved out into a separate file. Tests are often, usually, boring code which just distracts from the code you care about. I would hate to see it become common to have something like this: def some_function(): # 20 lines of code where: # 170 lines of tests become standard. But in any case, I think it is far too early to be thinking about stealing an idea from Pyret. The language isn't even stable yet, let alone at the stage where this idea has proven itself in the real world. Pyret and the where clause is still experimental.
In my experience, people who write poor doctests would write equally poor unit tests, or "where" tests if Python had this feature.
I don't understand this comment. In what way are unit tests from the unittest module, or doctests, not "real" unit tests? -- Steven
Steven D'Aprano wrote:
I would prefer to see some variation on Nick Coglan's ideas about a "where" keyword for local scoping of temporary variables.
I second that. This is the way mathematicians use the word "where", and it would be a much better way of spending a keyword.
In my experience, for every line of code you write, you'll probably need anything from 3-10 lines of test code.
Indeed. Also, the situations where you can meaningfully test each function on its own using a few concisely- expressed test cases are relatively rare, IMO. It looks good for the kind of exercises you find in programming courses, but it doesn't scale up to real-life code that requires complex data structures and test harnesses to be set up. -- Greg
On 10 November 2013 19:55, Tarek Ziadé <tarek@ziade.org> wrote:
It would make more sense to just bake py.test style rich assertions into the language in some way and let people write: def sum(iterable): # implementation of sum assert sum([]) == 0 assert sum([1, 2, 3]) == 6 A mechanism to say "always execute assert statements in this module regardless of optimisation level" could also be useful. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
Le 10/11/2013 15:48, Nick Coghlan a écrit :
This has the same problem as doctests: it works well for trivial tests like the above, but will be difficult to scale towards more complicated testing. unittest-like structuration is really what works best for most testing situations, IMO. Alternative testing schemes for "easier" or "more intuitive" testing have generally failed as general-purpose tools. Regards Antoine.
On 11 Nov 2013 02:12, "Antoine Pitrou" <solipsis@pitrou.net> wrote:
like the above, but will be difficult to scale towards more complicated testing. Yeah, part of my point was actually that module level assertions allow this kind of thing today, and there are good reasons people don't do it in practice.
Agreed, but the spelling of test assertions as methods on test cases isn't an essential part of that structure. For 3.5, it would be nice to offer either testtools style "matcher" objects or py.test style always-on-in-test-modules rich assertions. Either approach has a real world base to draw from, but would still involve a fair bit of work (they're also not mutually exclusive - a matcher based approach could be used as the back end for rich assertions). Cheers, Nick.
On Mon, Nov 11, 2013 at 09:58:06AM +0100, Tarek Ziadé wrote:
I often spell that: if __debug__: assert sum([]) == 0 # more complicated testing here I've been playing around with doctest, and I think it may be useful to have a "check" decorator that you use like this: @check def spam(): """Return spam. >>> spam() 'spam spam spam' """ return ' '.join(['spam']*3) If the doctests pass, the function is returned unchanged, otherwise an exception is raised. Note that this will encourage a specific style of more limited tests focused on just one function at a time. Normally, doctests aren't run until after the entire module is loaded, which lets you do things like this: def spam(): """Return spam. >>> spam() 'spam spam spam' Like ham, but more tasty: >>> taste(spam()) > taste(ham()) True """ return ' '.join(['spam']*3) # definitions of taste and ham follow later in the module. That *won't work* with a check decorator, since at the time the decorator runs, the functions taste and ham don't exist. The same would apply to a "where" clause (or whatever name it is given). This forces the doc tests to be more tightly focused on the function in isolation, which would be both good and bad. The good is that it would discourage overloading the docstring with too many too complex tests. The bad is that it would limit what can be tested this way, and the errors would no doubt be confusing to beginners. "What do you mean NameError? taste is defined right there..." On balance, I think that such a check() decorator would be useful enough that it's worth my time writing it. (Perhaps not useful enough to include in doctest, but I'll certainly stick it in my own personal toolbox.) But I don't think it would be useful enough to make it syntax. -- Steven
On Mon, 11 Nov 2013 07:57:21 +1000 Nick Coghlan <ncoghlan@gmail.com> wrote:
That's true, OTOH I disagree that assertions as methods is any kind of hindrance to easy testing. Actually, that paradigm makes it trivial to define your own assertions in a way that makes them look like the built-in ones. Regards Antoine.
On 11/10/2013 08:48 AM, Nick Coghlan wrote:
A mechanism to say "always execute assert statements in this module regardless of optimisation level" could also be useful.
+ 1 Currently assert statements are removed if the -O flag is used, but somehow that never seemed quite right to me. It makes more sense to have a -A option to turn them on, rather than using -O to turn them off. That change *along with* a way to "always execute asserts in this scope" would make asserts more useful. Cheers, Ron
On Sun, Nov 10, 2013 at 10:37 AM, Ron Adam <ron3200@gmail.com> wrote:
That change *along with* a way to "always execute asserts in this scope" would make asserts more useful.
assert not test_mode or sum([]) == 0 assert not test_mode or sum([1, 2, 3]) == 6 -- Keeping medicines from the bloodstreams of the sick; food from the bellies of the hungry; books from the hands of the uneducated; technology from the underdeveloped; and putting advocates of freedom in prisons. Intellectual property is to the 21st century what the slave trade was to the 16th.
participants (9)
-
Antoine Pitrou -
David Mertz -
Greg Ewing -
Markus Unterwaditzer -
Nick Coghlan -
Ron Adam -
Steven D'Aprano -
Tarek Ziadé -
Terry Reedy