<html><head></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><br><div><div>On Feb 15, 2010, at 3:50 PM, Michael Foord wrote:</div><br class="Apple-interchange-newline"><blockquote type="cite">
<div bgcolor="#ffffff" text="#000000">
On 15/02/2010 20:27, Glyph Lefkowitz wrote:
<blockquote cite="mid:5A4642E0-C933-47A6-B420-14EAD5BE6048@twistedmatrix.com" type="cite"><br>
<div>
<div>On Feb 13, 2010, at 12:46 PM, Guido van Rossum wrote:</div>
<br class="Apple-interchange-newline">
<blockquote type="cite">
<div>On Fri, Feb 12, 2010 at 8:01 PM, Glyph Lefkowitz<br>
<<a moz-do-not-send="true" href="mailto:glyph@twistedmatrix.com">glyph@twistedmatrix.com</a>>
wrote:</div></blockquote></div></blockquote></div></blockquote><blockquote type="cite"><div bgcolor="#ffffff" text="#000000"><blockquote cite="mid:5A4642E0-C933-47A6-B420-14EAD5BE6048@twistedmatrix.com" type="cite"><div><blockquote type="cite">
<div>
<blockquote type="cite">I find setUpClass
more hostile to *other* kinds of testing, because this convenience for
simple integration tests makes more involved, performance-intensive
integration tests harder to write and manage.</blockquote>
<br>
That sounds odd, as if the presence of this convenience would prohibit<br>
you from also implement other features.<br>
</div>
</blockquote>
<div></div><div><font class="Apple-style-span" color="#000000"><font class="Apple-style-span" color="#006312"><br></font></font></div></div></blockquote></div></blockquote><blockquote type="cite"><div bgcolor="#ffffff" text="#000000"><blockquote cite="mid:5A4642E0-C933-47A6-B420-14EAD5BE6048@twistedmatrix.com" type="cite"><div><div>
</div>
<div>And I am pretty sure this is not just my over-reaction; Michael
still appears to be wrestling with the problems I'm describing.</div>
</div>
</blockquote>
And I appreciate your input.<br></div></blockquote><div><br></div><div>Thanks :).</div><br><blockquote type="cite"><div bgcolor="#ffffff" text="#000000">
<blockquote cite="mid:5A4642E0-C933-47A6-B420-14EAD5BE6048@twistedmatrix.com" type="cite">
<div>
<div> In a recent message he was talking about either breaking
compatibility with TestSuite implementations that override run(), or
test-reordering - both of which I consider important, core features of
the unittest module.</div>
</div>
</blockquote>
<br>
Well, by "breaking compatibility with custom TestSuite implementations
that override run" I mean that is one possible place to put the
functionality. Code that does override it will *not* stop working, it
just won't support the new features.<br></div></blockquote><div><br></div><div>Ah, I see. This doesn't sound *too* bad, but I'd personally prefer it if the distinction were a bit more clearly drawn. I'd like frameworks to be able to implement extension functionality without having to first stub out functionality. In other words, if I want a test suite without setUpClass, I'd prefer to avoid having an abstraction inversion.</div><div><br></div><div>Practically speaking this could be implemented by having a very spare, basic TestSuite base class and ClassSuite/ModuleSuite subclasses which implement the setUpXXX functionality.</div><div><br></div><blockquote type="cite"><div bgcolor="#ffffff" text="#000000">
If we chose this implementation strategy there would be no
compatibility issues for existing tests / frameworks that don't use the
new features.</div></blockquote><div><br></div>That's very good to hear.<br><br><blockquote type="cite"><div bgcolor="#ffffff" text="#000000">If tests do want to use the new features then the
framework authors will need to ensure they are compatible with them.
This seems like a reasonable trade-off to me. We can ensure that it is
easy to write custom TestSuite objects that work with earlier versions
of unittest but are also compatible with setUpClass in 2.7 (and
document the recipe - although I expect it will just mean that
TestSuite.run should call a single method if it exists).<br></div></blockquote><div><br></div><div>This is something that I hope Jonathan Lange or Robert Collins will chime in to comment on: expanding the protocol between suite and test is an area which is fraught with peril, but it seems like it's something that test framework authors always want to do. (Personally, *I* really want to do it because I want to be able to run things asynchronously, so the semantics of 'run()' need to change pretty dramatically to support that...) It might be good to eventually develop a general mechanism for this, rather than building up an ad-hoc list of test-feature compatibility recipes which involve a list of if hasattr(...): foo(); checks in every suite implementation.</div><br><blockquote type="cite"><div bgcolor="#ffffff" text="#000000">
Perhaps a better idea might be to also add startTest and stopTest
methods to TestSuite so that frameworks can build in features like
timing tests (etc) without having to override run itself. This is
already possible in the TestResult of course, which is a more common
extensibility point in *my* experience.<br></div></blockquote></div><br><div>I think timing and monitoring tests can mostly be done in the TestResult class; those were bad examples. There's stuff like synthesizing arguments for test methods, or deciding to repeat a potentially flaky test method before reporting a failure, which are not possible to do from the result. I'm not sure that startTest and stopTest hooks help with those features, the ones which really need suites; it would seem it mostly gives you a hook to do stuff that could already be done in TestResult anyway.</div><div><br></div></body></html>