[IPython-dev] parallel testing of sympy with ipython

[Ondrej Certik]

Hi,

Brian helped me fix my problems with ipython and it then was super
easy to create parallel testing for sympy. Currently in my branch
here:

http://github.com/certik/sympy/tree/test2

e.g. start the cluster:

$ ipcluster local -n 8

Install sympy:

$ git clone git://github.com/certik/sympy.git
$ cd sympy
$ git checkout -b test2 origin/test2
$ python setup.py install --home=~/lib

(make sure ~/lib/lib/python is in your PYTHONPATH)

test in parallel:

$ python t.py
number of tests: 1376
distributing jobs
collecting results
processor: 0
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . F . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . exceptions
________________________________________________________________________________
  File "/home/ondrej/lib/lib/python/sympy/mpmath/tests/test_basic_ops.py",
line 115, in test_complex_misc
    assert mpc(2+1e-15j).ae(2)
  File "/home/ondrej/lib/lib/python/sympy/mpmath/tests/test_basic_ops.py",
line 115, in test_complex_misc
    assert mpc(2+1e-15j).ae(2)


processor: 1
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . exceptions
processor: 2
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . exceptions
processor: 3
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . F . . . . . exceptions
________________________________________________________________________________
  File "/home/ondrej/lib/lib/python/sympy/utilities/tests/test_lambdify.py",
line 97, in test_mpmath_lambda
    assert -prec < f(mpmath.mpf("0.2")) - sin02 < prec
  File "/home/ondrej/lib/lib/python/sympy/utilities/tests/test_lambdify.py",
line 97, in test_mpmath_lambda
    assert -prec < f(mpmath.mpf("0.2")) - sin02 < prec


processor: 4
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . exceptions
processor: 5
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . exceptions
processor: 6
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . exceptions
processor: 7
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . exceptions



All tests pass, except 4 mpmath tests, but I suspect it could be some
bug in mpmath.

Compare the timing:

Sequential:

$ time bin/test sympy
[...]
real	1m4.295s
user	1m3.872s
sys	0m0.332s

Parallel:

$ time python t.py
[..]
real	0m10.206s
user	0m0.308s
sys	0m0.064s

And this doesn't do any load balancing, so in the late phase (last 2s)
it is basically just computing on one processor only. I am looking
forward when I run this on our UNR cluster, I guess all sympy tests
could be done in just couple seconds.



Now some thoughts about load balancing:

It starts here:

http://github.com/certik/sympy/blob/55de47b6f0a7bee01249fc24c03e5567695c4569/t.py#L50

I create a tasks dictionary, which contains processors ids as keys and
sympy tests numbers as values (those tests should be executed on that
processor), where one test is just one testfunction in a file (e.g.
there is almost 1400 tests).

Then I distribute the tests on the processors here using nonblocking
mec.execute:

http://github.com/certik/sympy/blob/55de47b6f0a7bee01249fc24c03e5567695c4569/t.py#L64

And finally I collect the results:

http://github.com/certik/sympy/blob/55de47b6f0a7bee01249fc24c03e5567695c4569/t.py#L69

and report the results to the user. This should of course be
integrated into our testing framework, so that it looks exactly the
same. The goal should be that one would just use

bin/test

or

bin/test -j8

but otherwise the output doesn't change, only it will be 8x faster.

Ondrej