On Sun, Sep 30, 2018 at 11:34 AM Steven D'Aprano <steve@pearwood.info> wrote:
On Sun, Sep 30, 2018 at 10:29:50AM -0400, David Mertz wrote:
But given that in general unit tests tend to only exercise a handful of
values (have a look at the tests in the Python stdlib) I think it is
fair to say that in practice unit tests typically do not have anywhere
near the coverage of live data used during alpha and beta testing.

I still think it's a mixture.  I write tests to also address "this really shouldn't happen" cases as well (often in a loop, or using a Nose class for scaffolding).

There's some saying/joke about software testing along the lines of:

For an argument that should be in range 1-100: 
  try 50; try 1; try 100
  try 101; try 0; 
  try -1, 
  try 3.14
  try 1+1j;
  try the string "fish", 
  try a null pointer;

Many of those oddball cases can easily be in a list of values to test in unit tests, but may be impossible or unlikely to make it to the function call in the normal/possible flow of the program.
I'm curious. When you write a function or method, do you include input
checks? Here's an example from the Python stdlib (docstring removed for

# bisect.py
def insort_right(a, x, lo=0, hi=None):
    if lo < 0:
        raise ValueError('lo must be non-negative')
    if hi is None:
        hi = len(a)
    while lo < hi:
        mid = (lo+hi)//2
        if x < a[mid]: hi = mid
        else: lo = mid+1
    a.insert(lo, x)

Do you consider that check for lo < 0 to be disruptive? How would you
put that in a unit test?

I definitely put in checks like that.  However, I might well write a test like:

    assert lo >= 0, "lo must be non-negative"

That would allow disabling the check for production.  However, I presume the stdlib does this for a reason; it presumably wants to allow callers to catch a specific exception.  I haven't seen anything in the contract discussion that allows raising a particular exception when a contract is violated, only a general one for all contracts.

The case of 'if hi is None' is different.  It's remediation of a missing value where it's perfectly fine to impute an unspecified value.  So that would be a poor contract.

Keeping medicines from the bloodstreams of the sick; food
from the bellies of the hungry; books from the hands of the
uneducated; technology from the underdeveloped; and putting
advocates of freedom in prisons.  Intellectual property is
to the 21st century what the slave trade was to the 16th.