[pypy-dev] speed of sympy tests
Ondrej Certik
ondrej at certik.cz
Tue May 3 09:44:05 CEST 2011
On Tue, May 3, 2011 at 12:31 AM, Maciej Fijalkowski <fijall at gmail.com> wrote:
> On Tue, May 3, 2011 at 9:22 AM, Ondrej Certik <ondrej at certik.cz> wrote:
>> Hi,
>>
>> I have downloaded the pypy 1.5 binary (with jit) and run tests (only
>> sympy/core, so that it's fast) on Ubuntu Natty, 64bit:
>>
>>
>> ondrej at eagle:~/repos/sympy(master)$ bin/test sympy/core/
>> ============================= test process starts ==============================
>> executable: /usr/bin/python (2.7.1-final-0)
>> ground types: python
>>
>> sympy/core/tests/test_arit.py[48] ...f..........................................
>> .. [OK]
>> sympy/core/tests/test_assumptions.py[28] ....f....................... [OK]
>> sympy/core/tests/test_basic.py[10] .......... [OK]
>> sympy/core/tests/test_cache.py[1] . [OK]
>> sympy/core/tests/test_complex.py[13] ............. [OK]
>> sympy/core/tests/test_containers.py[5] ..... [OK]
>> sympy/core/tests/test_count_ops.py[2] .. [OK]
>> sympy/core/tests/test_diff.py[6] ...... [OK]
>> sympy/core/tests/test_equal.py[5] ..... [OK]
>> sympy/core/tests/test_eval.py[8] .....f.. [OK]
>> sympy/core/tests/test_eval_power.py[12] ............ [OK]
>> sympy/core/tests/test_evalf.py[23] ....................... [OK]
>> sympy/core/tests/test_expand.py[6] ...... [OK]
>> sympy/core/tests/test_expr.py[60] ..............................................
>> .............. [OK]
>> sympy/core/tests/test_exprtools.py[4] .... [OK]
>> sympy/core/tests/test_facts.py[11] ........... [OK]
>> sympy/core/tests/test_functions.py[23] .....f................. [OK]
>> sympy/core/tests/test_logic.py[11] ........... [OK]
>> sympy/core/tests/test_match.py[26] ...f...................... [OK]
>> sympy/core/tests/test_numbers.py[47] ...........................................
>> .... [OK]
>> sympy/core/tests/test_operations.py[4] .... [OK]
>> sympy/core/tests/test_priority.py[5] ..... [OK]
>> sympy/core/tests/test_relational.py[7] ....... [OK]
>> sympy/core/tests/test_sets.py[18] .................. [OK]
>> sympy/core/tests/test_subs.py[30] .............................. [OK]
>> sympy/core/tests/test_symbol.py[9] ......... [OK]
>> sympy/core/tests/test_sympify.py[23] ....................... [OK]
>> sympy/core/tests/test_truediv.py[3] ... [OK]
>> sympy/core/tests/test_var.py[5] ..... [OK]
>>
>> ======= tests finished: 448 passed, 5 expected to fail, in 2.90 seconds ========
>> ondrej at eagle:~/repos/sympy(master)$
>> ~/Downloads/pypy-c-jit-43780-b590cf6de419-linux64/bin/pypy bin/test
>> sympy/core/
>> ============================= test process starts ==============================
>> executable: /home/ondrej/Downloads/pypy-c-jit-43780-b590cf6de419-linux64/bin/pypy
>> (2.7.1-final-42)
>> ground types: python
>>
>> sympy/core/tests/test_arit.py[48] ...f..........................................
>> .. [OK]
>> sympy/core/tests/test_assumptions.py[28] ....f....................... [OK]
>> sympy/core/tests/test_basic.py[10] .......... [OK]
>> sympy/core/tests/test_cache.py[1] . [OK]
>> sympy/core/tests/test_complex.py[13] ............. [OK]
>> sympy/core/tests/test_containers.py[5] ..... [OK]
>> sympy/core/tests/test_count_ops.py[2] .. [OK]
>> sympy/core/tests/test_diff.py[6] ...... [OK]
>> sympy/core/tests/test_equal.py[5] ..... [OK]
>> sympy/core/tests/test_eval.py[8] .....f.. [OK]
>> sympy/core/tests/test_eval_power.py[12] ............ [OK]
>> sympy/core/tests/test_evalf.py[23] ....................... [OK]
>> sympy/core/tests/test_expand.py[6] ...... [OK]
>> sympy/core/tests/test_expr.py[60] ..........F...................................
>> .............. [FAIL]
>> sympy/core/tests/test_exprtools.py[4] .... [OK]
>> sympy/core/tests/test_facts.py[11] ........... [OK]
>> sympy/core/tests/test_functions.py[23] .....f................. [OK]
>> sympy/core/tests/test_logic.py[11] ........... [OK]
>> sympy/core/tests/test_match.py[26] ...f...................... [OK]
>> sympy/core/tests/test_numbers.py[47] ...........................................
>> .... [OK]
>> sympy/core/tests/test_operations.py[4] .... [OK]
>> sympy/core/tests/test_priority.py[5] ..... [OK]
>> sympy/core/tests/test_relational.py[7] ....... [OK]
>> sympy/core/tests/test_sets.py[18] .................. [OK]
>> sympy/core/tests/test_subs.py[30] .............................. [OK]
>> sympy/core/tests/test_symbol.py[9] ......... [OK]
>> sympy/core/tests/test_sympify.py[23] ....................... [OK]
>> sympy/core/tests/test_truediv.py[3] ... [OK]
>> sympy/core/tests/test_var.py[5] ..... [OK]
>>
>> ________________________________________________________________________________
>> _____________ sympy/core/tests/test_expr.py:test_as_leading_term3 ______________
>> File "/home/ondrej/repos/sympy/sympy/core/tests/test_expr.py", line
>> 212, in test_as_leading_term3
>> assert (2*x+pi*x+x**2).as_leading_term(x) == (2+pi)*x
>> AssertionError
>>
>> == tests finished: 447 passed, 1 failed, 5 expected to fail, in 13.87 seconds ==
>> DO *NOT* COMMIT!
>>
>>
>>
>>
>> The test failure is probably some minor difference between CPython and
>> PyPy.. However, as you can see, the test execution is 4x slower in
>> PyPy, compared to CPython.
>> Is this the supposed slow down, or is there something specific to sympy?
>>
>> I have also done this test in bin/isympy, CPython:
>>
>>>>> from timeit import default_timer as clock
>>>>> t = clock(); a = expand((1+x+y+z)**20); clock()-t
>> 1.4584941864
>>
>> PyPy:
>>
>>>>> from timeit import default_timer as clock
>>>>> t = clock(); a = expand((1+x+y+z)**20); clock()-t
>> 4.48898005486
>>
>> so PyPy is roughly 3x slower here.
>>
>> So 3x or 4x slower, I think that's already very usable. Especially the
>> sandbox pypy is an awesome idea, for serving web apps. The only
>> problem is that I use Cython a lot, which I think is incompatible with
>> PyPy...
>>
>> Ondrej
>> _______________________________________________
>> pypy-dev at codespeak.net
>> http://codespeak.net/mailman/listinfo/pypy-dev
>>
>
> Hey.
>
> I think the test run is somehow within acceptable numbers (in theory
> tests should run each code once, hence making JIT not really fit
> well), however I think the other slowdown is not really acceptable. We
Well, it is also only run once, so JIT probably doesn't help.
> were looking for a while to include some sympy benchmarks to our
> benchmark suite, maybe the good time is now :)
SymPy uses caching, so maybe that's the problem. It is possible to
turn it down though. Ping me if you have any questions, or run into
any problems with sympy.
Ondrej
More information about the pypy-dev
mailing list