Probabilistic unit tests?

Steven D'Aprano steve+comp.lang.python at
Fri Jan 11 18:46:31 CET 2013

On Fri, 11 Jan 2013 16:26:20 +0000, Alister wrote:

> On Thu, 10 Jan 2013 17:59:05 -0800, Nick Mellor wrote:
>> Hi,
>> I've got a unit test that will usually succeed but sometimes fails. An
>> occasional failure is expected and fine. It's failing all the time I
>> want to test for.
>> What I want to test is "on average, there are the same number of males
>> and females in a sample, give or take 2%."

> unit test are for testing your code, not checking if input data is in
> the correct range so unless you are writing a program intended to
> generate test data I don't see why unit test are appropriate in this
> case.

I don't believe Nick is using unittest to check input data. As I 
understand it, Nick has a program which generates random values. If his 
program works correctly, it should generate approximately equal numbers 
of "male" and "female" values. So he writes a unit test to check that the 
numbers are roughly equal.

This is an appropriate test, although as I already suggested earlier, 
unit tests are not well suited for non-deterministic testing.


More information about the Python-list mailing list