unittest's redundant assertions: asserts vs. failIf/Unlesses
Hi all,
This gem from unittest.py is pretty much the opposite of "one obvious way":
# Synonyms for assertion methods
assertEqual = assertEquals = failUnlessEqual
assertNotEqual = assertNotEquals = failIfEqual
assertAlmostEqual = assertAlmostEquals = failUnlessAlmostEqual
assertNotAlmostEqual = assertNotAlmostEquals = failIfAlmostEqual
assertRaises = failUnlessRaises
assert_ = assertTrue = failUnless
assertFalse = failIf
Could these be removed for 3k?
There was a short discussion about this among some of those those present in the Python Core sprint room at PyCon today and most preferred the "assertEqual" form for [Not][Almost]Equal and Raises.
With assertFalse vs. failIf (and assertTrue vs. failUnless) there was far less agreement. JUnit uses assertTrue exclusively, and most people said they feel that using assertTrue would be more consistent, but many (myself included) still think failUnless and failIf are much more natural. Another issue with assertTrue is that it doesn't actually test for 'True', strictly speaking, since it is based on equality, not identity.
Its also interesting to note the original commit message:
r34209 | purcell | 2003-09-22 06:08:12 -0500 (Mon, 22 Sep 2003) [...]
- New assertTrue and assertFalse aliases for comfort of JUnit users
[...]
assertEqual (and its cousins) were already present at that point.
In any case, if the decision is made to not use failUnless, something still needs to be done with assert_ vs. assertTrue. assert_ seems somewhat better to me, in that it has fewer characters, but I think that a case could certainly be made to keep both of these.
I certainly don't have the authority to make a call on any of this, but if someone else decides what colour to paint this bike shed, I can try to get it done (hopefully with 2.6 warnings) tomorrow.
Cheers,
-Gabriel
P.S. If you were in the sprint room and feel terribly misrepresented, please feel free to give me a swift kick both on-list and in person tomorrow morning.
+1 to assert* from me. the fail* variants always feel like double-negatives. I also always use assertTrue instead of assert_. But I don't care enough to argue about it. :)
On Wed, Mar 19, 2008 at 2:24 AM, Gabriel Grant grantgm@mcmaster.ca wrote:
Hi all,
This gem from unittest.py is pretty much the opposite of "one obvious way":
# Synonyms for assertion methods assertEqual = assertEquals = failUnlessEqual assertNotEqual = assertNotEquals = failIfEqual assertAlmostEqual = assertAlmostEquals = failUnlessAlmostEqual assertNotAlmostEqual = assertNotAlmostEquals = failIfAlmostEqual assertRaises = failUnlessRaises assert_ = assertTrue = failUnless assertFalse = failIfCould these be removed for 3k?
There was a short discussion about this among some of those those present in the Python Core sprint room at PyCon today and most preferred the "assertEqual" form for [Not][Almost]Equal and Raises.
With assertFalse vs. failIf (and assertTrue vs. failUnless) there was far less agreement. JUnit uses assertTrue exclusively, and most people said they feel that using assertTrue would be more consistent, but many (myself included) still think failUnless and failIf are much more natural. Another issue with assertTrue is that it doesn't actually test for 'True', strictly speaking, since it is based on equality, not identity.
Its also interesting to note the original commit message:
r34209 | purcell | 2003-09-22 06:08:12 -0500 (Mon, 22 Sep 2003) [...]
- New assertTrue and assertFalse aliases for comfort of JUnit users
[...]
assertEqual (and its cousins) were already present at that point.
In any case, if the decision is made to not use failUnless, something still needs to be done with assert_ vs. assertTrue. assert_ seems somewhat better to me, in that it has fewer characters, but I think that a case could certainly be made to keep both of these.
I certainly don't have the authority to make a call on any of this, but if someone else decides what colour to paint this bike shed, I can try to get it done (hopefully with 2.6 warnings) tomorrow.
Cheers,
-Gabriel
P.S. If you were in the sprint room and feel terribly misrepresented, please feel free to give me a swift kick both on-list and in person tomorrow morning. _______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/jyasskin%40gmail.com
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On Mar 19, 2008, at 3:20 AM, Jeffrey Yasskin wrote:
+1 to assert* from me. the fail* variants always feel like double-negatives. I also always use assertTrue instead of assert_. But I don't care enough to argue about it. :)
I'm in the camp that Gabriel describes. I prefer assertEqual/ assertRaises and failIf/failUnless.
I like the latter because it reads nicely: fail unless [this thing is true], fail if [this thing is true].
OTOH, I'd rather there be OOWTDI so whatever the consensus is is fine with me.
- -Barry
OTOH, I'd rather there be OOWTDI so whatever the consensus is is fine with me.
This strikes me as a gratuitous API change of the kind Guido was warning about in his recent post: "Don't change your APIs incompatibly when porting to Py3k"
Yes it removes redundancy, but it really doesn't change the cognitive load (at least for native speakers). If the blessed set were restricted to assert*, what would users of fail* do when trying to test their packages on py3k? Search and replace, or monkey patch unittest? I'm guessing monkey patch unittest, which means the change saves nothing, and costs plenty.
Note the acronym is OOWTDI, not OONTDI - using a different name does not necessarily make it a different way.
Michael Urman writes:
Yes it removes redundancy, but it really doesn't change the cognitive load (at least for native speakers).
Actually, OONTDI is important to me, cognitively. Multiple names implies the possibility of multiple semantics, often unintentionally. Here that can't be the case by construction, but I wouldn't have known that if I weren't reading this thread. And I still won't be sure for any given one of them, since there are so many remembering the whole list is "hard."
If the blessed set were restricted to assert*, what would users of fail* do when trying to test their packages on py3k? Search and replace, or monkey patch unittest?
So we should add this to 2to3, no? They're going to run that anyway.
I'm guessing monkey patch unittest, which means
I can flag them as people in too much of a hurry to do things right.<wink>
the change saves nothing, and costs plenty.
That's a bit short-sighted, no? It saves nothing for old working code. But 90% of the Python code in use 5 years from now will be written between now and then.
On Wed, Mar 19, 2008 at 10:44 AM, Stephen J. Turnbull stephen@xemacs.org wrote:
So we should add this to 2to3, no? They're going to run that anyway.
If 2to3 can handle this, that removes the larger half of my objection. I was under the impression that this kind of semantic inferencing was beyond its capabilities. But even if so, maybe it's safe to assume that those names aren't used in other contexts.
My remaining smaller half of the objection is that these aliases appear to have been added to reduce the friction when moving from another unit test system. Since the exact names are as much a matter of muscle memory as anything else being changed by py3k, that's not very important in this context.
I still don't see the benefit paying for the cost. Are people genuinely confused by the plethora of names for the operations (instead of by their occasional "misuse")? But I'm not the one offering a patch here, so I'll pipe down now. :)
On Wed, Mar 19, 2008 at 10:12 AM, Michael Urman murman@gmail.com wrote:
On Wed, Mar 19, 2008 at 10:44 AM, Stephen J. Turnbull stephen@xemacs.org wrote:
So we should add this to 2to3, no? They're going to run that anyway.
If 2to3 can handle this, that removes the larger half of my objection. I was under the impression that this kind of semantic inferencing was beyond its capabilities. But even if so, maybe it's safe to assume that those names aren't used in other contexts.
2to3 can indeed handle this, but I'm not sure I would want it run automatically (rather have it be opt-in, the way several other fixers are). Solid test suites are critical to the transition process, and changing method names around may upset that. It's unlikely, sure, but it may add to general unease.
The way I'd see such a fixer working is that people would run it over their 2.x codebase, commit that change, then transition the rest of their code at release-time, without having to worry about gratuitous code changes in their test suite.
Collin Winter
On 02:21 pm, murman@gmail.com wrote:
OTOH, I'd rather there be OOWTDI so whatever the consensus is is fine with me.
This strikes me as a gratuitous API change of the kind Guido was warning about in his recent post: "Don't change your APIs incompatibly when porting to Py3k"
I agree emphatically. Actually I think this is the most extreme case. The unit test stuff should be as stable as humanly possible between 2 and 3, moreso than any other library. It's one thing to have a test that fails because the functionality is broken. It's much worse to have a test that fails because the meaning of the test has changed - and this is going to happen anyway with e.g. assertEquals(foo.keys(), ['some', 'list']) which is a common enough assertion. Mixin in changes to the test framework creates even more confusion and pain. Granted, this change is apparently trivial and easy to understand, but each new traceback requires more thinking and there is going to be quite enough thinking going on in 2->3.
I say this from experience. Twisted's "trial" has been burned repeatedly both by introducing apparently trivial changes of our own to our testing tool and by apparently trivial changes to the Python stdlib unittest module, upon which we depend. Nothing we haven't been able to handle, but the 2->3 transition exacerbates this as it does all potential compatibility problems.
Whatever the stdlib does in this regard we'll definitely continue to provide an insulating layer in trial and keep both types of assertions for compatibility. So maybe the answer is to use Twisted for your unit tests rather than worry about your own compatibility lib ;-).
On Wed, Mar 19, 2008 at 2:15 PM, glyph@divmod.com wrote:
On 02:21 pm, murman@gmail.com wrote:
OTOH, I'd rather there be OOWTDI so whatever the consensus is is fine with me.
This strikes me as a gratuitous API change of the kind Guido was warning about in his recent post: "Don't change your APIs incompatibly when porting to Py3k"
I agree emphatically. Actually I think this is the most extreme case. The unit test stuff should be as stable as humanly possible between 2 and 3, moreso than any other library.
This is convincing for me. Move my +1 back to 3.1.
On Wed, Mar 19, 2008 at 5:16 PM, Jeffrey Yasskin jyasskin@gmail.com wrote:
On Wed, Mar 19, 2008 at 2:15 PM, glyph@divmod.com wrote:
On 02:21 pm, murman@gmail.com wrote:
OTOH, I'd rather there be OOWTDI so whatever the consensus is is fine with me.
This strikes me as a gratuitous API change of the kind Guido was warning about in his recent post: "Don't change your APIs incompatibly when porting to Py3k"
I agree emphatically. Actually I think this is the most extreme case. The unit test stuff should be as stable as humanly possible between 2 and 3, moreso than any other library.
This is convincing for me. Move my +1 back to 3.1.
Same here; let's tread carefully here and not change this with 3.0. Starting to deprecate in 3.1 and killing in 3.3 would be soon enough. I like using only the assertKeyword variants, removing assert_, fail*, and assertEquals. However I don't like changing assertTrue and assertFalse to insist that the value is exactly True or False -- if you really care that much, let's add assertIs(x, y) which asserts that x and y are the same object. I also think that all tests should use the operator their name implies, e.g. assertEqual(x, y) should do something like
if x == y: pass else: raise AssertionError(...)
rather than
if x != y: raise AssertionError(...)
Someone please open a bug for this task.
On Mon, Apr 7, 2008 at 7:37 PM, Guido van Rossum guido@python.org wrote:
On Wed, Mar 19, 2008 at 5:16 PM, Jeffrey Yasskin jyasskin@gmail.com wrote:
On Wed, Mar 19, 2008 at 2:15 PM, glyph@divmod.com wrote:
On 02:21 pm, murman@gmail.com wrote:
OTOH, I'd rather there be OOWTDI so whatever the consensus is is fine with me.
This strikes me as a gratuitous API change of the kind Guido was warning about in his recent post: "Don't change your APIs incompatibly when porting to Py3k"
I agree emphatically. Actually I think this is the most extreme case. The unit test stuff should be as stable as humanly possible between 2 and 3, moreso than any other library.
This is convincing for me. Move my +1 back to 3.1.
Same here; let's tread carefully here and not change this with 3.0. Starting to deprecate in 3.1 and killing in 3.3 would be soon enough. I like using only the assertKeyword variants, removing assert_, fail*, and assertEquals. However I don't like changing assertTrue and assertFalse to insist that the value is exactly True or False -- if you really care that much, let's add assertIs(x, y) which asserts that x and y are the same object. I also think that all tests should use the operator their name implies, e.g. assertEqual(x, y) should do something like
if x == y: pass else: raise AssertionError(...)
rather than
if x != y: raise AssertionError(...)
Someone please open a bug for this task.
See 2578.
-- --Guido van Rossum (home page: http://www.python.org/~guido/)
Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/musiccomposition%40gmail.c...
Guido van Rossum wrote:
On Wed, Mar 19, 2008 at 5:16 PM, Jeffrey Yasskin jyasskin@gmail.com wrote:
On Wed, Mar 19, 2008 at 2:15 PM, glyph@divmod.com wrote:
On 02:21 pm, murman@gmail.com wrote:
OTOH, I'd rather there be OOWTDI so whatever the consensus is is fine with me.
This strikes me as a gratuitous API change of the kind Guido was warning about in his recent post: "Don't change your APIs incompatibly when porting to Py3k"
I agree emphatically. Actually I think this is the most extreme case. The unit test stuff should be as stable as humanly possible between 2 and 3, moreso than any other library.
This is convincing for me. Move my +1 back to 3.1.
Same here; let's tread carefully here and not change this with 3.0. Starting to deprecate in 3.1 and killing in 3.3 would be soon enough. I like using only the assertKeyword variants, removing assert_, fail*, and assertEquals. However I don't like changing assertTrue and assertFalse to insist that the value is exactly True or False -- if you really care that much, let's add assertIs(x, y) which asserts that x and y are the same object. I also think that all tests should use the operator their name implies, e.g. assertEqual(x, y) should do something like
if x == y: pass else: raise AssertionError(...)
rather than
if x != y: raise AssertionError(...)
Someone please open a bug for this task.
This sounds like a good compromise and I'm happy to take on the cleanup - unless someone else beats me to it. I guess it should wait until 3.0 final is out of the door before adding the DeprecationWarnings.
Michael Foord
"Michael Foord" fuzzyman@voidspace.org.uk wrote in message news:47FB33C6.4080008@voidspace.org.uk... | > Someone please open a bug for this task. | > | > | This sounds like a good compromise and I'm happy to take on the cleanup | - unless someone else beats me to it. I guess it should wait until 3.0 | final is out of the door before adding the DeprecationWarnings.
I think, however, that the docs should be revised now, for 2.6/3.0. In the list of methods under TestCase Objects, the preferred method of each pair (or triplit) should be given first and the others marked as future deprecations, not recommended for new test code. The explanations of the methods should use the preferred names. So failIf/assertFalse should be reversed in order (assuming that failIf is the one to go) and the text "The inverse of the failUnless() method" should be changed to "The inverse of the assertTrue() method" (if that is the one to be kept).
I would also (lesser priority) revise examples to only use the preferred names. The would make things easiest for new unittest users that are not Java refugees.
tjr
On Apr 8, 2008, at 11:47 PM, Terry Reedy wrote:
I think, however, that the docs should be revised now, for 2.6/3.0. In the list of methods under TestCase Objects, the preferred method of each pair (or triplit) should be given first and the others marked as future deprecations, not recommended for new test code.
+1 !
This kind of information, once known, should be included in all maintained versions of the documentation so that users can build good coding habits, or have more time to adjust existing code.
-Fred
On Tue, Apr 8, 2008 at 9:56 PM, Fred Drake fdrake@acm.org wrote:
On Apr 8, 2008, at 11:47 PM, Terry Reedy wrote:
I think, however, that the docs should be revised now, for 2.6/3.0. In the list of methods under TestCase Objects, the preferred method of each pair (or triplit) should be given first and the others marked as future deprecations, not recommended for new test code.
+1 !
This kind of information, once known, should be included in all maintained versions of the documentation so that users can build good coding habits, or have more time to adjust existing code.
Great point, Fred! This sounds like a general strategy that we might employ more often -- when deprecating something, the first step ought to be to document the best practice.
"Guido van Rossum" guido@python.org writes:
Same here; let's tread carefully here and not change this with 3.0. Starting to deprecate in 3.1 and killing in 3.3 would be soon enough.
I'm very glad this is on the table. Even though I'd really like to see the names become PEP-8-conformant in the 2.x series, the arguments against such haste are compelling.
So I'm convinced to be +1 on post-3.0 for changing the unittest API.
I like using only the assertKeyword variants, removing assert_, fail*, and assertEquals.
I'm the opposite. I prefer the 'fail*' variants over the 'assert*' variants, because "fail" tells me exactly what the function will *do*, while "assert" leaves it implicit what will actually happen if the assertion is false.
For this reason, I'd prefer the "fail*" names to be recommended, and the "assert*" names to be, if not deprecated, then at least second-class citizens.
Ben Finney wrote:
"Guido van Rossum" guido@python.org writes:
[...]
I like using only the assertKeyword variants, removing assert_, fail*, and assertEquals.
I'm the opposite. I prefer the 'fail*' variants over the 'assert*' variants, because "fail" tells me exactly what the function will *do*, while "assert" leaves it implicit what will actually happen if the assertion is false.
For this reason, I'd prefer the "fail*" names to be recommended, and the "assert*" names to be, if not deprecated, then at least second-class citizens.
On the other hand, “assert*” names are positive statements of what the behaviour of the system-under-test is. And conversely, “fail*” names are a bit like implementation details of how the test is written. So I personally have a mild preference for the assert* names.
My suspicion is that it will be very hard to get consensus on the colour of this particular bike shed :)
-Andrew.
Andrew Bennetts <andrew-pythondev <at> puzzling.org> writes:
On the other hand, “assert*” names are positive statements of what the behaviour of the system-under-test is. And conversely, “fail*” names are a bit like implementation details of how the test is written. So I personally have a mild preference for the assert* names.
The problem with "fail*" is that you get names like "failIfNotEqual" (or perhaps even "failUnlessNotEqual") which are double negatives and therefore much more annoying to read and decipher. I always had the same problem when reading Perl's "unless" statements. They are, IMO, useless complication.
"assert*" names are, well, assertive :-)
(not to mention "assert" is a widely established name in various languages - including Python - for checking that things went as expected)
Antoine Pitrou solipsis@pitrou.net writes:
The problem with "fail*" is that you get names like "failIfNotEqual"
That would better be written (preferring PEP 8 names) "fail_unless_equal".
(or perhaps even "failUnlessNotEqual")
idem, "fail_if_equal".
which are double negatives
Exactly. With "if" and "unless" at our disposal, we can avoid such double negatives.
(not to mention "assert" is a widely established name in various languages - including Python - for checking that things went as expected)
That's another reason to avoid "assert" in the name: these methods *don't* necessarily use the 'assert' statement. Avoiding the implication that they do use that is a good thing.
Ben Finney <bignose+hates-spam <at> benfinney.id.au> writes:
That would better be written (preferring PEP 8 names) "fail_unless_equal".
Which is still a double negative ("fail" and "unless" are both negative words).
That's another reason to avoid "assert" in the name: these methods *don't* necessarily use the 'assert' statement.
But all those constructs (assert, assertEqual, etc.) raise the same exception type named AssertionError which, if you care for naming consistency, is more important than whether or not they are implemented in terms of each other. :-)
Antoine Pitrou wrote:
Ben Finney <bignose+hates-spam <at> benfinney.id.au> writes:
That would better be written (preferring PEP 8 names) "fail_unless_equal".
Which is still a double negative ("fail" and "unless" are both negative words).
"Fail" isn't a negative. As Guido said, it's a description of the test behavior under particular circumstances. "fail_unless_equal" says quite clearly that the test requires equality of the values.
That's another reason to avoid "assert" in the name: these methods *don't* necessarily use the 'assert' statement.
But all those constructs (assert, assertEqual, etc.) raise the same exception type named AssertionError which, if you care for naming consistency, is more important than whether or not they are implemented in terms of each other. :-)
But the important behavior is the failure of the test, not the specific exception that is raised to fail it. Or would you prefer tests that raise other exceptions to succeed? The exception type is an implementation detail, and a fairly unimportant one as far as the purpose of testing goes.
regards Steve
Let's split hairs a little...
Steve Holden <steve <at> holdenweb.com> writes:
"Fail" isn't a negative. As Guido said, it's a description of the test behavior under particular circumstances.
In most circumstances, "fail" is a negative word defined as the contrary of something else (that is, as the "failure to pass/succeed/perform/achieve/..."), while the reverse is not true (few people would define "success" or "passing a test" as the negative of "failure", except in desperate circumstances). Although I'm not a native English speaker, I don't think our respective languages and cultures differ on this point.
"fail_unless_equal" says quite clearly that the test requires equality of the values.
Actually, saying "that the test requires equality of the values" translates directly into an "assert equals" (or "enforce equals" if you want a stronger word) rather than a "fail if not equal". It is a grammatical fact...
In other words, if you express a requirement, you intent to say how the implementation under test is supposed to behave for it to be considered successful, not the conditions under which its behaviour constitutes a failure.
As you said, if an exception is thrown which isn't part of the testing protocol (e.g. something other than an AssertionError), the test is still said to fail... if the intent of testing were to test for failure conditions, on the contrary, the test would be said to be passed (because it wouldn't have met the failure conditions).
Regards
Antoine.
Antoine Pitrou wrote:
Let's split hairs a little...
Steve Holden <steve <at> holdenweb.com> writes:
"Fail" isn't a negative. As Guido said, it's a description of the test behavior under particular circumstances.
In most circumstances, "fail" is a negative word defined as the contrary of something else (that is, as the "failure to pass/succeed/perform/achieve/..."), while the reverse is not true (few people would define "success" or "passing a test" as the negative of "failure", except in desperate circumstances). Although I'm not a native English speaker, I don't think our respective languages and cultures differ on this point.
"fail_unless_equal" says quite clearly that the test requires equality of the values.
Actually, saying "that the test requires equality of the values" translates directly into an "assert equals" (or "enforce equals" if you want a stronger word) rather than a "fail if not equal". It is a grammatical fact...
In other words, if you express a requirement, you intent to say how the implementation under test is supposed to behave for it to be considered successful, not the conditions under which its behaviour constitutes a failure.
Agreed. I tend to think of testing as action followed by assertion - I do this and this should have happened. Your tests usually define 'expected behaviour' rather than defining how your code won't fail...
Michael
As you said, if an exception is thrown which isn't part of the testing protocol (e.g. something other than an AssertionError), the test is still said to fail... if the intent of testing were to test for failure conditions, on the contrary, the test would be said to be passed (because it wouldn't have met the failure conditions).
Regards
Antoine.
Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.u...
On Mon, 14 Jul 2008 12:45:58 am Michael Foord wrote:
I tend to think of testing as action followed by assertion - I do this and this should have happened. Your tests usually define 'expected behaviour' rather than defining how your code won't fail...
Who is the "your" that you are speaking for there? It isn't me.
I tend to think of tests as *both* expected behaviour and unexpected behaviour: sometimes a test is most naturally written as "this should happen" and sometimes as "this shouldn't happen". It depends on what I'm testing.
When it comes to test-driven development, I will often start by thinking "What test can I write to make this code fail?" -- not "what test can I write to make this code pass?". Having come up with a failure mode, the test is often most naturally written as "fail if" or "fail unless", and I resent having to turn the condition around into a "assert if not" test just to satisfy others who are never going to read my code. I wish people would go find another bike shed to interfere with.
On Mon, 14 Jul 2008 12:45:58 am Michael Foord wrote:
I tend to think of testing as action followed by assertion - I do this and this should have happened. Your tests usually define 'expected behaviour' rather than defining how your code won't fail...
Who is the "your" that you are speaking for there? It isn't me.
Although Michael unfortunately chose to use "Your", it is clear to me that the context implies he actually meant "My" (as in, his)
I resent having to turn the condition around into a "assert if not" test just to satisfy others who are never going to read my code. I wish people would go find another bike shed to interfere with.
Oh please, just get over it, and try to stick to the issue at hand rather than trying to score points or acting under the assumption that everyone is out to give you cause to "resent" things...
As Steve said, this is getting silly...
Mark
On Mon, 14 Jul 2008 04:19:57 am Mark Hammond wrote:
try to stick to the issue at hand
I'm sorry, did I reply to the wrong message? I thought this was part of a thread titled "unittest's redundant assertions: asserts vs. failIf/Unlesses". What *is* the issue at hand if not the question of positive assertions versus fail if/unless?
As Steve said, this is getting silly...
It certainly is.
Python may be Guido's language, and if he wants to use his dictatorial powers to say that tests must be written as positive assertions because that's the way he likes it, that's his prerogative. But let's not pretend that this particular bike shed colour has any objectively rational reason, or that the change won't force some people to have to change the way they think about tests.
Steven D'Aprano wrote:
On Mon, 14 Jul 2008 04:19:57 am Mark Hammond wrote:
try to stick to the issue at hand
I'm sorry, did I reply to the wrong message? I thought this was part of a thread titled "unittest's redundant assertions: asserts vs. failIf/Unlesses". What *is* the issue at hand if not the question of positive assertions versus fail if/unless?
As Steve said, this is getting silly...
It certainly is.
Python may be Guido's language, and if he wants to use his dictatorial powers to say that tests must be written as positive assertions because that's the way he likes it, that's his prerogative. But let's not pretend that this particular bike shed colour has any objectively rational reason, or that the change won't force some people to have to change the way they think about tests.
But sometimes even the wrong decision is better than no decision, and I suspect most if us can agree that it's better if everyone thinks about tests the same way. Otherwise test code is difficult to understand for some of its user population.
regards Steve
On Mon, 14 Jul 2008 09:54:16 am Steve Holden wrote:
Python may be Guido's language, and if he wants to use his dictatorial powers to say that tests must be written as positive assertions because that's the way he likes it, that's his prerogative. But let's not pretend that this particular bike shed colour has any objectively rational reason, or that the change won't force some people to have to change the way they think about tests.
But sometimes even the wrong decision is better than no decision, and I suspect most if us can agree that it's better if everyone thinks about tests the same way.
There's a term for what happens when everybody in a community or group thinks about a subject the same way.
http://en.wikipedia.org/wiki/Groupthink
Antoine Pitrou wrote:
Let's split hairs a little...
Steve Holden <steve <at> holdenweb.com> writes:
"Fail" isn't a negative. As Guido said, it's a description of the test behavior under particular circumstances.
In most circumstances, "fail" is a negative word defined as the contrary of something else (that is, as the "failure to pass/succeed/perform/achieve/..."), while the reverse is not true (few people would define "success" or "passing a test" as the negative of "failure", except in desperate circumstances). Although I'm not a native English speaker, I don't think our respective languages and cultures differ on this point.
"The test will fail" is an assertion (in English, not in Python :). It is not a negative. "The test will not fail" is an assertion containing a negative. "The test will not not fail" is an assertion containing a double negative.
"fail_unless_equal" says quite clearly that the test requires equality of the values.
Actually, saying "that the test requires equality of the values" translates directly into an "assert equals" (or "enforce equals" if you want a stronger word) rather than a "fail if not equal". It is a grammatical fact...
Right. For extra points, discuss the differences between "fail_unless_equal", "fail_if_not_equal", assert_equals" and "assert_unequal".
In other words, if you express a requirement, you intent to say how the implementation under test is supposed to behave for it to be considered successful, not the conditions under which its behaviour constitutes a failure.
As you said, if an exception is thrown which isn't part of the testing protocol (e.g. something other than an AssertionError), the test is still said to fail... if the intent of testing were to test for failure conditions, on the contrary, the test would be said to be passed (because it wouldn't have met the failure conditions).
But Guido said:
I like using only the assertKeyword variants, removing assert_, fail*, and assertEquals.
So we are now no longer discussing what color to paint the bike shed, we are discussing how to describe the color. Let's stop. This is getting silly.
regards Steve
Steve Holden <steve <at> holdenweb.com> writes:
But Guido said:
I like using only the assertKeyword variants, removing assert_, fail*, and assertEquals.
So we are now no longer discussing what color to paint the bike shed, we are discussing how to describe the color. Let's stop. This is getting silly.
It's certainly getting silly - the language is named Python after all! However, the whole thread started by someone opposing Guido's statement above, which at least explains (if not justifies) the origins of this particular instance of silliness...
(and I've probably made my point explicitly enough, so I won't insist, even if I sometimes enjoy arguing on my spare time ;-))
cheers
Antoine.
On Mon, 14 Jul 2008 12:39:48 am Antoine Pitrou wrote:
Let's split hairs a little...
Steve Holden <steve <at> holdenweb.com> writes:
"Fail" isn't a negative. As Guido said, it's a description of the test behavior under particular circumstances.
In most circumstances, "fail" is a negative word defined as the contrary of something else (that is, as the "failure to pass/succeed/perform/achieve/..."), while the reverse is not true (few people would define "success" or "passing a test" as the negative of "failure", except in desperate circumstances).
"Few people"? Do you have studies to support your claim, or are you just projecting your own opinion as if it were an objective fact?
I often consider "success" as the negative of failure: my code didn't fail. The bridge didn't fall down. The invasion didn't get bogged down in a long and pointless guerrilla war. The medicine didn't have massive side-effects. These are all successes.
assert_bridge_not_collapsed() assert_no_guerrilla_war() assert_if_no_side-effects()
fail_if_bridge_collapsed() fail_if_guerrilla_war() fail_if_side_effects()
Speaking for myself, the last three are far more natural to me.
I'll also point out that in science, success is often considered the opposite of failure. You design your experiments to disprove the theory, to fail, and if they don't fail, the theory is considered successful. A theory is considered "correct" when it has failed to fail. Some philosophers of science, like the late Karl Popper, consider this falsification to be the defining characteristic of science.
[...]
In other words, if you express a requirement, you intent to say how the implementation under test is supposed to behave for it to be considered successful, not the conditions under which its behaviour constitutes a failure.
Please don't tell me what my intent is. Unless you are a mind-reader, you have no way of telling what my intent is.
When I write tests, my intent is often to specify the conditions that constitute a failure. I don't want to negate it to specify a success just to satisfy you.
Steve Holden writes:
"Fail" isn't a negative. As Guido said, it's a description of the test behavior under particular circumstances.
This is not true, however. "Fail" is a description of a potentailly very large class of behaviors. Knowing that the test failed does not tell you which of the failure behaviors occurred, only that the success behavior did not occur.
The analogy to the fact that True != not not 10 is telling, I think.
* Stephen J. Turnbull stephen@xemacs.org [2008-07-14 08:51:27 +0900]:
The analogy to the fact that True != not not 10 is telling, I think.
What?
True == (not not 10)
True
Tristan Seligmann writes:
- Stephen J. Turnbull stephen@xemacs.org [2008-07-14 08:51:27 +0900]:
The analogy to the fact that True != not not 10 is telling, I think.
What?
True == (not not 10)
True
FWIW, I meant 10 != not not 10.
On Mon, 14 Jul 2008 04:27:40 pm Stephen J. Turnbull wrote:
FWIW, I meant 10 != not not 10.
10 != not not 10
File "<stdin>", line 1 10 != not not 10 ^ SyntaxError: invalid syntax
With respect, I think that the fact that you made an analogy with Python code that you hadn't tested, got it wrong, then corrected it, and *still* got it wrong, is telling. Its part of the pattern of this thread. People have repeatedly and consistently asserted that their particular favourite bike-shed colour is not just more attractive than any other colour, but supposedly objectively and logically better than any other colours. It's that second part that I object to.
When it comes to assert* versus fail* tests, this is one case where I don't believe "one obvious way to do it" should apply. A similar, and I hope uncontroversial, case is the greater-than and less-than operators. It would be frankly silly to declare that Python code should always use x < y and never y > x on the basis that there should be "one obvious way". Sometimes a particular test is most naturally written as g-t, and sometimes as l-t, and sometimes the choice between them is just arbitrary.
I believe that assert* and fail* tests are the same: while the choice is often arbitrary, sometimes a test is naturally written as an assert and sometimes as a fail. My own tests often look like this:
fail_if_spam() fail_unless_ham() assert_parrot() fail_if_spanish_inquisition()
because the specific parrot test is best written as an assertion rather than a fail. And frankly, I simply don't believe that this imposes an undue mental cost on people who might read my code.
It's certainly true that I could invert the nature of the tests:
assert_not_spam() assert_ham() assert_parrot() assert_not_spanish_inquisition()
just for the sake of consistency (and that would be a good thing *why*?), but only at the cost of inverting the code inside the test, which may or may not be a simple thing to do.
Steven D'Aprano writes:
On Mon, 14 Jul 2008 04:27:40 pm Stephen J. Turnbull wrote:
FWIW, I meant 10 != not not 10.
10 != not not 10
File "<stdin>", line 1 10 != not not 10 ^ SyntaxError: invalid syntax
With respect, I think that the fact that you made an analogy with Python code that you hadn't tested, got it wrong, then corrected it, and *still* got it wrong, is telling.
It's telling only if you ignore the fact that it's the ad hominem fallacy.
When it comes to assert* versus fail* tests, this is one case where I don't believe "one obvious way to do it" should apply.
In the context you were talking about, where you don't need to show anybody your tests, I don't see any reason why TOOWTDI should apply. In a debugging context, I don't see any reason why it should, either.
But here we're talking about the standard unit-testing framework, which is used by Python itself. So it *is* about communication with other developers, and it *does* make sense to avoid proliferation of redundant vocabulary. Some of us have enough trouble remembering when parentheses are redundant, as you noticed.
Regression test suites aspire to full coverage. If you mix assert* and fail* clauses, you are going to need to invert one or the other to determine whether there are cases that are missed.
IMO those are strong indications that only one of the pair should be used in a test suite.
I believe that assert* and fail* tests are the same:
In logic, they are. I just think that assert* is much more natural in several simple cases, such as the pure regression test where you use Python X.Y to compute foo(), and add "assertEqual(foo(), pyXYresult)" to the test suite. I'd be a lot more sympathetic if you'd describe a similarly plausible and generic application for fail*.
On Tue, 15 Jul 2008 06:32:53 pm Stephen J. Turnbull wrote:
Steven D'Aprano writes:
On Mon, 14 Jul 2008 04:27:40 pm Stephen J. Turnbull wrote:
FWIW, I meant 10 != not not 10.
10 != not not 10
File "<stdin>", line 1 10 != not not 10 ^ SyntaxError: invalid syntax
With respect, I think that the fact that you made an analogy with Python code that you hadn't tested, got it wrong, then corrected it, and *still* got it wrong, is telling.
It's telling only if you ignore the fact that it's the ad hominem fallacy.
The ad hominem fallacy does not mean "somebody pointed out I made a silly mistake". It is not a fallacy to reject a person's argument because it is hastily or carelessly made.
However, fair is fair -- I too have to accept a slice of humble pie. In *my* haste to make my point, I neglected to actually make the point I intended to. I went on to write "It's part of the pattern of this thread" but didn't explain why. Namely that with so many people trying to push their personal preference, and signs of frustration on both sides, people are making unsupported statements of opinion as if they were facts, and generally getting sloppy. Ironic, yes?
Having communicated poorly, and by doing so giving you offence, I will accept an appropriate fish-slapping.
When it comes to assert* versus fail* tests, this is one case where I don't believe "one obvious way to do it" should apply.
In the context you were talking about, where you don't need to show anybody your tests, I don't see any reason why TOOWTDI should apply. In a debugging context, I don't see any reason why it should, either.
But here we're talking about the standard unit-testing framework, which is used by Python itself.
But I'm using that same framework, so decisions made regarding that framework are going to impact me. Terry Reedy says that "stdlib modules are suggestions and examples, not commands", which is strictly true but I think he discounts just how strong the stdlib's influence is. Perhaps large projects, and tiny ones, are free to go their own way, but in my experience (such that it is), small projects tend to follow the stdlib.
So it *is* about communication with other developers, and it *does* make sense to avoid proliferation of redundant vocabulary.
Certainly redundant vocabulary has a cost, but it also has a benefit. At least for those people (including myself) who frequently think of a unit test as a sequence of potential failures rather than expected successes. Only if the code "fails to fail" does it pass the test, and so I often find fail* tests more suitable. YMMV.
Steve Holden wrote:
"Fail" isn't a negative.
That depends on what you're trying to find out by reading the code. If you're trying to find out under what conditions the test succeeds, then it succeeds if it doesn't fail, so you have a negative.
Whichever convention is chosen, there will be situations in which you want to mentally negate it. If you start with a positive, the mental negation produces a single negative. If you start with a negative, the mental negation produces a double negative. Therefore it is less confusing overall to start with as few negatives as possible.
Antoine Pitrou solipsis@pitrou.net writes:
Ben Finney <bignose+hates-spam <at> benfinney.id.au> writes:
That would better be written (preferring PEP 8 names) "fail_unless_equal".
Which is still a double negative ("fail" and "unless" are both negative words).
Hmm, not to this native-English-speaker's ear. "fail" is a verb stating what will be done, while "unless" and "if" are the conditions under which it will be done.
That's another reason to avoid "assert" in the name: these methods *don't* necessarily use the 'assert' statement.
But all those constructs (assert, assertEqual, etc.) raise the same exception type named AssertionError
Only by default. They can be overridden to raise any exception type.
The only thing they have in common at that point (when the exception is raised) is that they have failed the test.
Antoine Pitrou wrote:
Ben Finney <bignose+hates-spam <at> benfinney.id.au> writes:
That would better be written (preferring PEP 8 names) "fail_unless_equal".
Which is still a double negative ("fail" and "unless" are both negative words).
That's another reason to avoid "assert" in the name: these methods *don't* necessarily use the 'assert' statement.
But all those constructs (assert, assertEqual, etc.) raise the same exception type named AssertionError which, if you care for naming consistency, is more important than whether or not they are implemented in terms of each other. :-)
Regarding: "all those constructs ... raise the same exception type named AssertionError.."
As an experiment a while back (out of frustration), I created some tests that used more specific (local unittest module only) exceptions. The clarity of the failed errors and the mental effort to debug the problems was much improved for me.
I sub-classed unittest.TestCase, and added two new methods and a few local and unique test-only exceptions.
* test -> a test function to call * ref -> addition helpful debug info
assertTestReturns(test, expect, ref="")
raises on failure:
Wrong_Result_Returned Unexpected_Exception_Raised
assertTestRaises(test, expect, ref="")
raises on failure:
No_Exception_Raised Wrong_Exception_Raised
These more specific exceptions (over plain AssertionError), allowed for more specific error reports.
A testcase with an error would produce a standard python exception so you know instantly that you need to fix the test rather than the code being tested.
Also the more specific exception code can better handle some cases where a wrong traceback would be returned. ie.. the test code traceback rather than the code being tested traceback.
Is there any interest in going in this direction with unittests?
Ron
Ben Finney wrote:
That's another reason to avoid "assert" in the name: these methods *don't* necessarily use the 'assert' statement. Avoiding the implication that they do use that is a good thing.
Perhaps some positive alternative such as "verify" could be used instead.
Andrew Bennetts andrew-pythondev@puzzling.org writes:
On the other hand, “assert*” names are positive statements of what the behaviour of the system-under-test is.
That statement is better made by the name of the test case. The method used for implementing the test shouldn't need to be part of that statement.
And conversely, “fail*” names are a bit like implementation details of how the test is written.
Entirely appropriate, since those names are used exactly at the point of implementing the test, and the names are not visible outside that implementation. No?
My suspicion is that it will be very hard to get consensus on the colour of this particular bike shed :)
Perhaps so :-)
"Guido van Rossum" guido@python.org writes:
Same here; let's tread carefully here and not change this with 3.0. Starting to deprecate in 3.1 and killing in 3.3 would be soon enough. I like using only the assertKeyword variants, removing assert_, fail*, and assertEquals.
Are we to interpret the above ("… using only the assertKeyword variants, removing assert_, …") as "we should actively remove names that are PEP 8 compatible from 'unittest', instead preferring names that go against PEP 8?
I really hope I'm misinterpreting this and that there's another explanation. Preferably one that includes "we have a plan to make 'unittest' conform with the existing PEP 8".
Ben Finney wrote:
"Guido van Rossum" guido@python.org writes:
Same here; let's tread carefully here and not change this with 3.0. Starting to deprecate in 3.1 and killing in 3.3 would be soon enough. I like using only the assertKeyword variants, removing assert_, fail*, and assertEquals.
Are we to interpret the above ("… using only the assertKeyword variants, removing assert_, …") as "we should actively remove names that are PEP 8 compatible from 'unittest', instead preferring names that go against PEP 8?
I really hope I'm misinterpreting this and that there's another explanation. Preferably one that includes "we have a plan to make 'unittest' conform with the existing PEP 8".
"we have a plan to make 'unittest' conform with the existing PEP 8" - that was the outcome of the discussion. PEP 8'ification to begin in 2.7 / 3.1 with the deprecation of non-PEP8 compliant method names.
There was some added functionality discussed, but it is too late to get this into 2.6 / 3.0.
I volunteered to take it on, and have a record of the whole discussion. Unfortunately my writing commitment is going to keep me occupied until August - after which it will be one of my highest priorities.
Michael Foord
Michael Foord fuzzyman@voidspace.org.uk writes:
"we have a plan to make 'unittest' conform with the existing PEP 8"
- that was the outcome of the discussion. PEP 8'ification to begin
in 2.7 / 3.1 with the deprecation of non-PEP8 compliant method names.
Thanks, that's exactly the reassurance I was hoping for :-)
I volunteered to take it on, and have a record of the whole discussion. Unfortunately my writing commitment is going to keep me occupied until August - after which it will be one of my highest priorities.
I've contacted you yesterday regarding this, but to reiterate: This issue is of interest to me, and I'd like to help with it if I can. Please contact me privately if I can be of assistance, especially if I can help during your busy period.
On 19/03/2008, Michael Urman murman@gmail.com wrote:
OTOH, I'd rather there be OOWTDI so whatever the consensus is is fine with me.
This strikes me as a gratuitous API change of the kind Guido was warning about in his recent post: "Don't change your APIs incompatibly when porting to Py3k"
This seems compelling to me. And as Glyph mentioned, the testing APIs are the most critical ones to keep working when moving from 2 to 3.
-1 on this change as part of 3.0. Either do it in 2.6 (which wouldn't satisfy backward compatibility requirements) or defer it to 3.1 or later. OK, starting 3.0 with deprecated methods is a nuisance, but the testing framework seems to me to be a valid special case.
Or just don't bother at all. It's not *that* bad.
Paul.
On Wed, Mar 19, 2008 at 7:05 PM, Paul Moore p.f.moore@gmail.com wrote:
This strikes me as a gratuitous API change of the kind Guido was warning about in his recent post: "Don't change your APIs incompatibly when porting to Py3k"
This seems compelling to me. And as Glyph mentioned, the testing APIs are the most critical ones to keep working when moving from 2 to 3.
It seems as though this declaration, in this case, is in direct conflict with the overarching theme of "one obvious way to do things". That statement, of course, is the reason so many things are being changed in the move to 3k already, and I don't see why this shouldn't be one of them (particularly, because it's so easy to account for this in 2to3). As one is a global statement, and the other is fairly local, I vote for the change.
Leif Walsh wrote:
On Wed, Mar 19, 2008 at 7:05 PM, Paul Moore p.f.moore@gmail.com wrote:
This strikes me as a gratuitous API change of the kind Guido was warning about in his recent post: "Don't change your APIs incompatibly when porting to Py3k"
This seems compelling to me. And as Glyph mentioned, the testing APIs are the most critical ones to keep working when moving from 2 to 3.
It seems as though this declaration, in this case, is in direct conflict with the overarching theme of "one obvious way to do things". That statement, of course, is the reason so many things are being changed in the move to 3k already, and I don't see why this shouldn't be one of them (particularly, because it's so easy to account for this in 2to3). As one is a global statement, and the other is fairly local, I vote for the change.
As Guido(?) pointed out, this would be acceptable because it's simply different spellings of the same way.
regards Steve
2008/3/19, Barry Warsaw barry@python.org:
+1 to assert* from me. the fail* variants always feel like double-negatives. I also always use assertTrue instead of assert_. But I don't care enough to argue about it. :)
+1 to the plain affirmative propositions (assert*) instead of the kind-of-double-negative stuff. It helps a lot specially if you're not a native english speaker.
+1 to remove them in Py3k.
Questions:
- 2to3 should "fix" them?
- should we add a warning in 2.6 and remove them in 2.7? Or 2.7/2.8?
Regards,
Gabriel Grant wrote:
Hi all,
This gem from unittest.py is pretty much the opposite of "one obvious way":
# Synonyms for assertion methods assertEqual = assertEquals = failUnlessEqual assertNotEqual = assertNotEquals = failIfEqual assertAlmostEqual = assertAlmostEquals = failUnlessAlmostEqual assertNotAlmostEqual = assertNotAlmostEquals = failIfAlmostEqual assertRaises = failUnlessRaises assert_ = assertTrue = failUnless assertFalse = failIfCould these be removed for 3k?
There was a short discussion about this among some of those those present in the Python Core sprint room at PyCon today and most preferred the "assertEqual" form for [Not][Almost]Equal and Raises.
With assertFalse vs. failIf (and assertTrue vs. failUnless) there was far less agreement. JUnit uses assertTrue exclusively, and most people said they feel that using assertTrue would be more consistent, but many (myself included) still think failUnless and failIf are much more natural. Another issue with assertTrue is that it doesn't actually test for 'True', strictly speaking, since it is based on equality, not identity.
+1 on standardising on 'assert*' and removing 'fail*'.
+1 on making 'assertTrue' test for True rather than any non-false object (and vice versa for assertFalse)
For migration a simple subclass of TestCase that provides the old methods/semantics is trivial to write. No need for monkey-patching.
Michael Foord
Its also interesting to note the original commit message:
r34209 | purcell | 2003-09-22 06:08:12 -0500 (Mon, 22 Sep 2003) [...]
- New assertTrue and assertFalse aliases for comfort of JUnit users
[...]
assertEqual (and its cousins) were already present at that point.
In any case, if the decision is made to not use failUnless, something still needs to be done with assert_ vs. assertTrue. assert_ seems somewhat better to me, in that it has fewer characters, but I think that a case could certainly be made to keep both of these.
I certainly don't have the authority to make a call on any of this, but if someone else decides what colour to paint this bike shed, I can try to get it done (hopefully with 2.6 warnings) tomorrow.
Cheers,
-Gabriel
P.S. If you were in the sprint room and feel terribly misrepresented, please feel free to give me a swift kick both on-list and in person tomorrow morning. _______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.u...
On Wed, Mar 19, 2008 at 6:24 PM, Gabriel Grant grantgm@mcmaster.ca wrote:
Hi all,
This gem from unittest.py is pretty much the opposite of "one obvious way":
# Synonyms for assertion methods
[snip]
Could these be removed for 3k?
I agree with others who say that we shouldn't do this for Python 3k. If we want to get rid of them, we should deprecate them, wait a release or so, *then* remove them.
That said, can we axe one of (assertEqual, assertEquals) too?
jml
Jonathan Lange wrote:
On Wed, Mar 19, 2008 at 6:24 PM, Gabriel Grant grantgm@mcmaster.ca wrote:
Hi all,
This gem from unittest.py is pretty much the opposite of "one obvious way":
# Synonyms for assertion methods[snip]
Could these be removed for 3k?
I agree with others who say that we shouldn't do this for Python 3k. If we want to get rid of them, we should deprecate them, wait a release or so, *then* remove them.
That said, can we axe one of (assertEqual, assertEquals) too?
We should keep 'assertEquals'.
Michael
jml _______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.u...
On Wed, Mar 19, 2008 at 5:02 PM, Jonathan Lange jml@mumak.net wrote:
On Wed, Mar 19, 2008 at 6:24 PM, Gabriel Grant grantgm@mcmaster.ca wrote:
Hi all,
This gem from unittest.py is pretty much the opposite of "one obvious way":
# Synonyms for assertion methods[snip]
Could these be removed for 3k?
I agree with others who say that we shouldn't do this for Python 3k. If we want to get rid of them, we should deprecate them, wait a release or so, *then* remove them.
It seems to me if we want to remove this at some point, the time is now. I'm not aware of anything starting off deprecated in 3k - my impression is the whole point of 3k is to clean house.
Deprecation warnings can be put into 2.6 if people think thats necessary, but the more important thing will be including it in 2to3. I'm working on that right now, so if/when the actual wording is finalized, it should just be a matter of changing around the order of the function names in a dict.
-Gabriel
participants (25)
-
Andrew Bennetts -
Antoine Pitrou -
Barry Warsaw -
Ben Finney -
Benjamin Peterson -
Collin Winter -
Facundo Batista -
Fred Drake -
Gabriel Grant -
glyph＠divmod.com -
Greg Ewing -
Guido van Rossum -
Jeffrey Yasskin -
Jonathan Lange -
Leif Walsh -
Mark Hammond -
Michael Foord -
Michael Urman -
Paul Moore -
Ron Adam -
Stephen J. Turnbull -
Steve Holden -
Steven D'Aprano -
Terry Reedy -
Tristan Seligmann