Probabilistic unit tests?
thebalancepro at gmail.com
Fri Jan 11 02:59:05 CET 2013
I've got a unit test that will usually succeed but sometimes fails. An occasional failure is expected and fine. It's failing all the time I want to test for.
What I want to test is "on average, there are the same number of males and females in a sample, give or take 2%."
Here's the unit test code:
from collections import counter
sex_count = Counter()
for contact in range(self.binary_check_sample_size):
p = get_record_as_dict()
sex_count[p['Sex']] += 1
delta=sample_size * 2.0 / 100.0)
My question is: how would you run an identical test 5 times and pass the group *as a whole* if only one or two iterations passed the test? Something like:
for n in range(5):
# if test passed: break
(except that would create 5+1 tests as written!)
Thanks for any thoughts,
More information about the Python-list