[pypy-dev] speed of sympy tests
Maciej Fijalkowski
fijall at gmail.com
Tue May 3 15:20:49 CEST 2011
On Tue, May 3, 2011 at 3:18 PM, Maciej Fijalkowski <fijall at gmail.com> wrote:
> On Tue, May 3, 2011 at 9:44 AM, Ondrej Certik <ondrej at certik.cz> wrote:
>> On Tue, May 3, 2011 at 12:31 AM, Maciej Fijalkowski <fijall at gmail.com> wrote:
>>> On Tue, May 3, 2011 at 9:22 AM, Ondrej Certik <ondrej at certik.cz> wrote:
>>>> Hi,
>>>>
>>>> I have downloaded the pypy 1.5 binary (with jit) and run tests (only
>>>> sympy/core, so that it's fast) on Ubuntu Natty, 64bit:
>>>>
>>>>
>>>> ondrej at eagle:~/repos/sympy(master)$ bin/test sympy/core/
>>>> ============================= test process starts ==============================
>>>> executable: /usr/bin/python (2.7.1-final-0)
>>>> ground types: python
>>>>
>>>> sympy/core/tests/test_arit.py[48] ...f..........................................
>>>> .. [OK]
>>>> sympy/core/tests/test_assumptions.py[28] ....f....................... [OK]
>>>> sympy/core/tests/test_basic.py[10] .......... [OK]
>>>> sympy/core/tests/test_cache.py[1] . [OK]
>>>> sympy/core/tests/test_complex.py[13] ............. [OK]
>>>> sympy/core/tests/test_containers.py[5] ..... [OK]
>>>> sympy/core/tests/test_count_ops.py[2] .. [OK]
>>>> sympy/core/tests/test_diff.py[6] ...... [OK]
>>>> sympy/core/tests/test_equal.py[5] ..... [OK]
>>>> sympy/core/tests/test_eval.py[8] .....f.. [OK]
>>>> sympy/core/tests/test_eval_power.py[12] ............ [OK]
>>>> sympy/core/tests/test_evalf.py[23] ....................... [OK]
>>>> sympy/core/tests/test_expand.py[6] ...... [OK]
>>>> sympy/core/tests/test_expr.py[60] ..............................................
>>>> .............. [OK]
>>>> sympy/core/tests/test_exprtools.py[4] .... [OK]
>>>> sympy/core/tests/test_facts.py[11] ........... [OK]
>>>> sympy/core/tests/test_functions.py[23] .....f................. [OK]
>>>> sympy/core/tests/test_logic.py[11] ........... [OK]
>>>> sympy/core/tests/test_match.py[26] ...f...................... [OK]
>>>> sympy/core/tests/test_numbers.py[47] ...........................................
>>>> .... [OK]
>>>> sympy/core/tests/test_operations.py[4] .... [OK]
>>>> sympy/core/tests/test_priority.py[5] ..... [OK]
>>>> sympy/core/tests/test_relational.py[7] ....... [OK]
>>>> sympy/core/tests/test_sets.py[18] .................. [OK]
>>>> sympy/core/tests/test_subs.py[30] .............................. [OK]
>>>> sympy/core/tests/test_symbol.py[9] ......... [OK]
>>>> sympy/core/tests/test_sympify.py[23] ....................... [OK]
>>>> sympy/core/tests/test_truediv.py[3] ... [OK]
>>>> sympy/core/tests/test_var.py[5] ..... [OK]
>>>>
>>>> ======= tests finished: 448 passed, 5 expected to fail, in 2.90 seconds ========
>>>> ondrej at eagle:~/repos/sympy(master)$
>>>> ~/Downloads/pypy-c-jit-43780-b590cf6de419-linux64/bin/pypy bin/test
>>>> sympy/core/
>>>> ============================= test process starts ==============================
>>>> executable: /home/ondrej/Downloads/pypy-c-jit-43780-b590cf6de419-linux64/bin/pypy
>>>> (2.7.1-final-42)
>>>> ground types: python
>>>>
>>>> sympy/core/tests/test_arit.py[48] ...f..........................................
>>>> .. [OK]
>>>> sympy/core/tests/test_assumptions.py[28] ....f....................... [OK]
>>>> sympy/core/tests/test_basic.py[10] .......... [OK]
>>>> sympy/core/tests/test_cache.py[1] . [OK]
>>>> sympy/core/tests/test_complex.py[13] ............. [OK]
>>>> sympy/core/tests/test_containers.py[5] ..... [OK]
>>>> sympy/core/tests/test_count_ops.py[2] .. [OK]
>>>> sympy/core/tests/test_diff.py[6] ...... [OK]
>>>> sympy/core/tests/test_equal.py[5] ..... [OK]
>>>> sympy/core/tests/test_eval.py[8] .....f.. [OK]
>>>> sympy/core/tests/test_eval_power.py[12] ............ [OK]
>>>> sympy/core/tests/test_evalf.py[23] ....................... [OK]
>>>> sympy/core/tests/test_expand.py[6] ...... [OK]
>>>> sympy/core/tests/test_expr.py[60] ..........F...................................
>>>> .............. [FAIL]
>>>> sympy/core/tests/test_exprtools.py[4] .... [OK]
>>>> sympy/core/tests/test_facts.py[11] ........... [OK]
>>>> sympy/core/tests/test_functions.py[23] .....f................. [OK]
>>>> sympy/core/tests/test_logic.py[11] ........... [OK]
>>>> sympy/core/tests/test_match.py[26] ...f...................... [OK]
>>>> sympy/core/tests/test_numbers.py[47] ...........................................
>>>> .... [OK]
>>>> sympy/core/tests/test_operations.py[4] .... [OK]
>>>> sympy/core/tests/test_priority.py[5] ..... [OK]
>>>> sympy/core/tests/test_relational.py[7] ....... [OK]
>>>> sympy/core/tests/test_sets.py[18] .................. [OK]
>>>> sympy/core/tests/test_subs.py[30] .............................. [OK]
>>>> sympy/core/tests/test_symbol.py[9] ......... [OK]
>>>> sympy/core/tests/test_sympify.py[23] ....................... [OK]
>>>> sympy/core/tests/test_truediv.py[3] ... [OK]
>>>> sympy/core/tests/test_var.py[5] ..... [OK]
>>>>
>>>> ________________________________________________________________________________
>>>> _____________ sympy/core/tests/test_expr.py:test_as_leading_term3 ______________
>>>> File "/home/ondrej/repos/sympy/sympy/core/tests/test_expr.py", line
>>>> 212, in test_as_leading_term3
>>>> assert (2*x+pi*x+x**2).as_leading_term(x) == (2+pi)*x
>>>> AssertionError
>>>>
>>>> == tests finished: 447 passed, 1 failed, 5 expected to fail, in 13.87 seconds ==
>>>> DO *NOT* COMMIT!
>>>>
>>>>
>>>>
>>>>
>>>> The test failure is probably some minor difference between CPython and
>>>> PyPy.. However, as you can see, the test execution is 4x slower in
>>>> PyPy, compared to CPython.
>>>> Is this the supposed slow down, or is there something specific to sympy?
>>>>
>>>> I have also done this test in bin/isympy, CPython:
>>>>
>>>>>>> from timeit import default_timer as clock
>>>>>>> t = clock(); a = expand((1+x+y+z)**20); clock()-t
>>>> 1.4584941864
>>>>
>>>> PyPy:
>>>>
>>>>>>> from timeit import default_timer as clock
>>>>>>> t = clock(); a = expand((1+x+y+z)**20); clock()-t
>>>> 4.48898005486
>>>>
>>>> so PyPy is roughly 3x slower here.
>>>>
>>>> So 3x or 4x slower, I think that's already very usable. Especially the
>>>> sandbox pypy is an awesome idea, for serving web apps. The only
>>>> problem is that I use Cython a lot, which I think is incompatible with
>>>> PyPy...
>>>>
>>>> Ondrej
>>>> _______________________________________________
>>>> pypy-dev at codespeak.net
>>>> http://codespeak.net/mailman/listinfo/pypy-dev
>>>>
>>>
>>> Hey.
>>>
>>> I think the test run is somehow within acceptable numbers (in theory
>>> tests should run each code once, hence making JIT not really fit
>>> well), however I think the other slowdown is not really acceptable. We
>>
>> Well, it is also only run once, so JIT probably doesn't help.
>>
>>> were looking for a while to include some sympy benchmarks to our
>>> benchmark suite, maybe the good time is now :)
>>
>>
>> SymPy uses caching, so maybe that's the problem. It is possible to
>> turn it down though. Ping me if you have any questions, or run into
>> any problems with sympy.
>>
>> Ondrej
>>
>
> That might sound strange but for me, both pypy and cpython runs within
> 4s. Which python you use?
>
CPython 2.7 runs 2x faster than 2.6. Can we get some more real-time
communication set up, so we can discuss a bit
More information about the pypy-dev
mailing list