replacing __dict__ with an OrderedDict

[Steven D'Aprano]

On Fri, 06 Jan 2012 10:20:28 -0700, Ian Kelly wrote:

> On Fri, Jan 6, 2012 at 10:01 AM, Lie Ryan <lie.1296 at gmail.com> wrote:
>> That unittest executes its tests in alphabetical order is
>> implementation detail for a very good reason, and good unittest
>> practice dictates that execution order should never be defined (some
>> even argued that the execution order should be randomized). If the test
>> runner turns out to execute tests concurrently, that should not cause
>> problems for a well-designed test. Displaying the test results in a
>> more convenient order for viewing is what you really wanted in 99.99%
>> of the cases.
> 
> Randomizing the order is not a bad idea, but you also need to be able to
> run the tests in a consistent order, from a specific random seed. In the
> real world, test conflicts and dependencies do happen,

In the real world, test conflicts and dependencies are bugs in your test 
suite that should be fixed, like any other bug in code. The fact that it 
is test code that is failing is irrelevant.

Also in the real world, sometimes it is too hard to fix a bug and you 
just live with it.


> and if we observe
> a failure, make a change, rerun the tests and observe success, we need
> to be able to be sure that we actually fixed the bug, and that it didn't
> pass only because it was run in a different order.

Every test should stand alone. You should be able to isolate each and 
every test and run it without the others. If you can't, your test suite 
is buggy. (Chances are good that most test suites are buggy. After all, 
who writes tests for their tests? That's just the start of an infinite 
regress with rapidly diminishing benefit.) 

You may not be able to run tests *simultaneously*, due to clashes 
involving external resources, but you should be able to run them in 
random order.



-- 
Steven