[Short version: It doesn't look like my proposal or any
simple alternative is tenable.]
Yes, it does.
On Aug 10, 2013, at 10:28 AM, Ralf Gommers wrote:
> It does break backwards compatibility though, because now you can do:
>
> import numpy as np
> np.testing.assert_equal(x, y)
I realize that a design goal in numpy was that (most?) submodules are
available without any additional imports. This is the main reason for
the "import numpy" overhead. The tension between ease-of-use for some
and overhead for others is well known. For example, Sage tickets 3577,
6494, and 11714 relate to deferring numpy import during startup.
The three relevant questions are:
1) is numpy.testing part of that promise? This can be split
into multiple ways.
o The design goal could be that only the numerics that people use
for interactive/REPL computing are accessible without
additional explicit imports, which implies that the import of
numpy.testing is an implementation side-effect of providing
submodule-level "test()" and "bench()" APIs
o all NumPy modules with user-facing APIs should be accessible
from numpy without additional imports
While I would like to believe that the import of numpy.testing
is an implementation side-effect of providing test() and bench(),
I believe that I'm unlikely to convince the majority.
For justifiable reasons, the numpy project is loath to break
backwards compatibility, and I don't think there's an existing
bright-line policy which would say that "import numpy; numpy.testing"
should be avoided.
2) If it isn't a promise that "numpy.testing" is usable after an
"import numpy" then how many people will be affected by an
implementation change, and at what level of severity?
I looked to see which packages might fail. A Debian code
search of "numpy.testing" showed no problems, and no one
uses "np.testing".
I did a code search at http://code.ohloh.net . Of the first
200 or so hits for "numpy.testing", nearly all of them fell
into uses like:
from numpy.testing import Tester
from numpy.testing import assert_equal, TestCase
from numpy.testing.utils import *
from numpy.testing import *
There were, however, several packages which would fail:
test_io.py and test_image.py and test_array_bridge.py in MediPy
(Interestingly, test_circle.py has a "import numpy.testing",
so it's not universal practice in that package)
calculators_test.py in OpenQuake Engine
ForcePlatformsExtractorTest.py in b-tk
Note that these failures are in the test programs, and not
in the main body code, so are unlikely to break end-user
programs.
HOWEVER!
The real test is for people who do "import numpy as np" then
refer to "np.testing". There are "about 454" such matches in
Ohloh.
One example is 'test_polygon.py' from scikit-image. Others are:
test_anova.py in statsmodel
test_graph.py in scikit-learn
test_rmagic.py in IPython
test_mlab.py in matplotlib
Nearly all the cases I looked at were in files starting "test",
or a handful which ended in "test.py" or "Test.py". Others use
np.test only as part of a unit test, such as:
affine_grid.py and others in pyMor (as part of in-file unit tests)
technical_indicators.py in QuantPy (as part of unittest.TestCase)
coord_tools.py in NiPy-OLD (as part of in-file unit tests)
predstd.py and others in statsmodels (as a main-line unit test)
galsim_test_helpers.py in GalSim
These would likely not break end-user code.
Sadly, not all are that safe. For examples:
simple_contrast.py example program for nippy
try_signal_lti.py in joePython
run.py in python-seminar
verify.py in bell_d_project (a final project for a CS class)
ex_shrink_pickle.py in statsmodels (as an example?)
parametric_design.py in nippy (uses assert_almost_equal to verify an example)
model.py in pymc-devs's pymc
model.py in duffy
zipline in olmar
utils.py in MNE
.... and I gave up at result 320 of 454.
Based on this, about 1% of the programs which use numpy.testing
would break. This tells me that there are enough user programs
which would fail that I don't think numpy will decide to make
this change.
And the third question is
3) Are there other alternatives?
I do have more detailed timings. "nose" is not imported
Or as Ralf Gommers wrote:
> Do you have more detailed timings? I'm guessing the bottleneck is importing nose.
during an "import numpy". (For one, "import nose" takes
a full 0.11 seconds on my laptop and adds 199 modules
to sys.modules!)
The hit is the "import unittest" in numpy.testing, which
exists only to place "TestCase" in the numpy.testing namespace.
"numpy.testing.TestCase" is only used by the unit tests,
and not by any direct end-user code.
Here's the full hierarchical timing breakdown showing
- module name
- cumulative time to load
- parent module
testing: 0.0065 (numpy.core.numeric)
unittest: 0.0055 (testing)
result: 0.0011 (unittest)
traceback: 0.0004 (result)
linecache: 0.0000 (traceback)
StringIO: 0.0004 (result)
errno: 0.0000 (StringIO)
case: 0.0021 (unittest)
difflib: 0.0011 (case)
pprint: 0.0004 (case)
util: 0.0000 (case)
suite: 0.0002 (unittest)
loader: 0.0006 (unittest)
fnmatch: 0.0002 (loader)
main: 0.0010 (unittest)
time: 0.0000 (main)
signals: 0.0006 (main)
signal: 0.0000 (signals)
weakref: 0.0005 (signals)
UserDict: 0.0000 (weakref)
_weakref: 0.0000 (weakref)
_weakrefset: 0.0000 (weakref)
exceptions: 0.0000 (weakref)
runner: 0.0000 (unittest)
utils: 0.0005 (testing)
nosetester: 0.0002 (utils)
numpy.testing.utils: 0.0000 (nosetester)
numpytest: 0.0001 (testing)
As you can see, "unittest" imports a large number of modules.
I see no good way to get rid of this unittest import.
Even if all of the tests were rewritten to use unittest.TestCase,
numpy.testing.TestCase would still need to be present so
third-party packages could derive from it, and there's no (easy?)
way to make that TestCase some sort of deferred object which
gets the correct TestCase when needed.
In conclusion, it looks like my proposal is not tenable and
there's no easy way to squeeze out that ~5% of startup overhead.