[Numpy-discussion] Proposal for matrix_rank function in numpy

[Robert Kern]

On Fri, Dec 18, 2009 at 21:21, Fernando Perez <fperez.net at gmail.com> wrote:
> On Wed, Dec 16, 2009 at 10:56 AM, Skipper Seabold <jsseabold at gmail.com> wrote:
>> Presumably the doctests should be turned into
>> actual tests (noting Robert's comment) to make it more likely that it
>> gets in
>
> Just curious: is there a policy against pure doctests in numpy? I've
> always found that doctests and 'real tests' serve complementary
> purposes and are both useful:
>
> - small, clear tests that make for *illustrative* examples for the end
> user should be left in the docstring, and picked up by the test suite
> as normal doctests.
>
> - tests with a lot more logic that get cumbersome to write as doctests
> can go into 'normal' tests into the test suite.
>
> - There's also a valid third category: for cases where it's convenient
> to write the test interactively but one doesn't want the example in
> the main docstring, putting a function in the test suite that simply
> has the doctest as a docstring works (it may require a little
> decorator, I don't recall).
>
> I'm just wondering if there's a policy of requiring that all tests
> become non-doctests...

My policy and rationale, which I believe is reflected in the docstring
standard, is that examples in the docstrings should put pedagogical
concerns above all others. In my experience, a properly robust doctest
sacrifices the readability, clarity, and terseness of a good example.
Thus, not all examples run as doctests, so docstrings are not added to
numpy's test suite. For example, for floating point functions, one
should use allclose(), etc. to test the results against the gold
standard. However, using that in the example makes it less clear what
is going on. Compare:

In [2]: np.sin(np.linspace(0, 2*np.pi, 10))
Out[2]:
array([  0.00000000e+00,   6.42787610e-01,   9.84807753e-01,
         8.66025404e-01,   3.42020143e-01,  -3.42020143e-01,
        -8.66025404e-01,  -9.84807753e-01,  -6.42787610e-01,
        -2.44929360e-16])

In [4]: np.allclose(np.sin(np.linspace(0, 2*np.pi, 10)), array([0.0,
.642787610, .984807753, .866025404, .342020143, -.342020143,
-.866025404, -.984807753, -.642787610, 0.0]))
Out[4]: True

I certainly don't mind properly written doctests outside of the real
docstrings (e.g. in particular files under test/ that just contain
doctests); they're a handy way to write certain tests although they
have well known and annoying limits for numerical work. I just don't
want documentation examples to be doctests.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth."
  -- Umberto Eco