[Numpy-discussion] Many failing doctests - release blocker? Enable for default test runs?

[Matthew Brett]

Hi,

I noticed that enabling the doctests on the 1.7.x maintenance branch
caused lots and lots of doctest failures.

(np-devel)[mb312 at blair ~/dev_trees]$ python -c 'import numpy as np;
np.test(doctests=True)'
1.7.0rc1.dev-1e8fcdf
Running unit tests and doctests for numpy
NumPy version 1.7.0rc1.dev-1e8fcdf
NumPy is installed in
/Users/mb312/.virtualenvs/np-devel/lib/python2.6/site-packages/numpy
Python version 2.6.6 (r266:84374, Aug 31 2010, 11:00:51) [GCC 4.0.1
(Apple Inc. build 5493)]
nose version 1.1.2
...
Ran 3839 tests in 59.928s

FAILED (KNOWNFAIL=4, SKIP=4, errors=23, failures=175)

The doctests also throw up somewhere round 10 matplotlib plots, so
presumably those would fail as well on a machine without a display
without forcing the import of an 'Agg' backend or similar.

I have never checked the doctests on Python 3.  Has anyone run those recently?

For the projects I work on most, we enable doctests for the default
test run - as in 'doctests=True' by default in the numpy testing
machinery.   Do ya'll see any disadvantage in doing that for numpy?

In case someone gets to this before I do, we've also got some logic
for doing conditional skips of doctests when optional packages are not
available such as matplotlib, inspired by something similar in
IPython:

https://github.com/nipy/nipy/blob/master/nipy/testing/doctester.py#L193

If Christmas allows I'll send a pull request with something like that
in the next few days.

Cheers,

Matthew