replacing __dict__ with an OrderedDict

[Terry Reedy]

On 1/9/2012 8:05 PM, Roy Smith wrote:
> In article<11jrt8-l32.ln1 at satorlaser.homedns.org>,
>   Ulrich Eckhardt<ulrich.eckhardt at dominolaser.com>  wrote:
>
>>> Some people advocate that the test framework should
>>> intentionally randomize the order, to flush out inter-test dependencies
>>> that the author didn't realize existed (or intend).
>>
>> If you now
>> happen to influence one test with another and the next run randomizes
>> the tests differently, you will never see the fault again. Without this
>> reproducability, you don't gain anything but the bad stomach feeling
>> that something is wrong.
>
> The standard solution to that is to print out the PRNG initialization
> state and provide a way in your test harness to re-initialize it to that
> state.  I've done things like that in test scenarios where it is
> difficult or impossible to cover the problem space deterministically.
>
>> Your unfortunate case is where test X creates persistent state that must
>> be present in order for test X+1 to produce meaningful results. This
>> kind of dependency obviously blows, as it means you can't debug test X+1
>> separately. I'd call this operational dependency.
>>
>> This kind of dependency is IMHO a bug in the tests themselves.
>
> For the most part, I'm inclined to agree.  However, there are scenarios
> where having each test build the required state from scratch is
> prohibitively expensive.  Imagine if you worked at NASA wanted to run
> test_booster_ignition(), test_booster_cutoff(),
> test_second_stage_ignition(), and test_self_destruct().  I suppose you
> could run them in random order, but you'd use up a lot of rockets that
> way.
>
> Somewhat more seriously, let's say you wanted to do test queries against
> a database with 100 million records in it.  You could rebuild the
> database from scratch for each test, but doing so might take hours per
> test.  Sometimes, real life is just *so* inconvenient.
>
>> There is another dependency and that I'd call a logical dependency. This
>> occurs when e.g. test X tests for an API presence and test Y tests the
>> API behaviour. In other words, Y has no chance to succeed if X already
>> failed.
>
> Sure.  I run into that all the time.  A trivial example would be the
> project I'm working on now.  I've come to realize that a long unbroken
> string of E's means, "Dummy, you forgot to bring the application server
> up before you ran the tests".  It would be nicer if the test suite could
> have run a single test which proved it could create a TCP connection and
> when that failed, just stop.

Many test cases in the Python test suite have multiple asserts. I 
believe both resource sharing and sequential dependencies are reasons. I 
consider 'one assert (test) per testcase' to be on a par with 'one class 
per file'.

-- 
Terry Jan Reedy