[Numpy-discussion] Is numpy.test() supposed to be multithreaded?

[Sebastian Berg]

On Mi, 2016-06-29 at 02:03 -0700, Nathaniel Smith wrote:
> As a general rule I wouldn't worry too much about test speed. Speed
> is
> extremely dependent on exact workloads. And this is doubly so for
> test
> suites -- production workloads tend to do a small number of normal
> things over and over, while a good test suite never does the same
> thing twice and spends most of its time exercising weird edge
> conditions. So unless your actual workload is running the numpy test
> suite :-), it's probably not worth trying to track down.
> 

Agreed, the test suit, and likely also the few tests which might take
most time in the end, could be arbitrarily weird and skewed. I could
for example imagine IO speed being a big factor. Also depending on
system configuration (or numpy version) a different number of tests may
be run sometimes.

What might make somewhat more sense would be to compare some of the
benchmarks `python runtests.py --bench` if you have airspeed velocity
installed. While not extensive, a lot of those things at least do test
more typical use cases. Though in any case I think the user should
probably just test some other thing.

- Sebastian


> And yeah, numpy does not in general do automatic multithreading --
> the
> only automatic multithreading you should see is when using linear
> algebra functions (matrix multiply, eigenvalue calculations, etc.)
> that dispatch to the BLAS.
> 
> -n
> 
> On Wed, Jun 29, 2016 at 12:07 AM, Ralf Gommers <ralf.gommers at gmail.co
> m> wrote:
> > 
> > 
> > 
> > On Wed, Jun 29, 2016 at 3:27 AM, Chris Barker - NOAA Federal
> > <chris.barker at noaa.gov> wrote:
> > > 
> > > 
> > > 
> > > > 
> > > > 
> > > > Now the user is writing back to say, "my test code is fast now,
> > > > but
> > > > numpy.test() is still about three times slower than <some other
> > > > server we
> > > > don't manage>".  When I watch htop as numpy.test() executes,
> > > > sure enough,
> > > > it's using one core
> > > 
> > > > 
> > > > * if numpy.test() is supposed to be using multiple cores, why
> > > > isn't it,
> > > > when we've established with other test code that it's now using
> > > > multiple
> > > > cores?
> > > 
> > > Some numpy.linalg functions (like np.dot) will be using multiple
> > > cores,
> > > but np.linalg.test() takes only ~1% of the time of the full test
> > > suite.
> > > Everything else will be running single core. So your observations
> > > are not
> > > surprising.
> > > 
> > > 
> > > Though why it would run slower on one box than another comparable
> > > box is a
> > > mystery...
> > 
> > Maybe just hardware config? I see a similar difference between how
> > long the
> > test suite runs on TravisCI vs my linux desktop (the latter is
> > slower,
> > surprisingly).
> > 
> > Ralf
> > 
> > 
> > _______________________________________________
> > NumPy-Discussion mailing list
> > NumPy-Discussion at scipy.org
> > https://mail.scipy.org/mailman/listinfo/numpy-discussion
> > 
> 
> 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: This is a digitally signed message part
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20160629/dad99bef/attachment.sig>