[issue1328] cProfile: slow on built-in function calls
New submission from Armin Rigo <armin.rigo@gmail.com>: The cProfile recording overhead with the JIT is low when counting function calls, but very high when counting calls to built-in functions. Jitviewer shows that it takes a lot of residual calls around a single call to a built-in function. It seems to me that we did not put enough efforts in optimizing this. A trivial example shows an overhead of about 500x: def f(n): lst = [] for i in xrange(n): lst.append(i) # built-in call lst.pop() # built-in call ---------- messages: 5010 nosy: arigo, pypy-issue priority: performance bug status: unread title: cProfile: slow on built-in function calls ________________________________________ PyPy bug tracker <tracker@bugs.pypy.org> <https://bugs.pypy.org/issue1328> ________________________________________
Daniel Roberts <Ademan555@gmail.com> added the comment: This looks fixed as of https://bitbucket.org/pypy/pypy/src/7c881e2648065a2b25155d9cbda18534ecd eb2ec/pypy/module/_lsprof/interp_lsprof.py?at=default Performance hit for me is down to 3-4x from 15-20x so I'm happy with current performance. ---------- status: unread -> testing ________________________________________ PyPy bug tracker <tracker@bugs.pypy.org> <https://bugs.pypy.org/issue1328> ________________________________________
Carl Friedrich Bolz <cfbolz@gmx.de> added the comment: I was hoping to just close this issue, but it seems that the jitting of cProfile is simply broken atm. cProfile contains two elidables that are ignored after the recent elidable changes. This is a rather big regression, which we didn't notice as there is neither a benchmark, nor a test_pypy_c test about it. ---------- nosy: +cfbolz priority: performance bug -> critical status: testing -> chatting title: cProfile: slow on built-in function calls -> unbreak cProfile jitting ________________________________________ PyPy bug tracker <tracker@bugs.pypy.org> <https://bugs.pypy.org/issue1328> ________________________________________
Armin Rigo <armin.rigo@gmail.com> added the comment: Should we now really turn ignored @elidables into crashes, and fix those crashes? Didn't wim checked in something about them in cppyy, recently? ________________________________________ PyPy bug tracker <tracker@bugs.pypy.org> <https://bugs.pypy.org/issue1328> ________________________________________
Armin Rigo <armin.rigo@gmail.com> added the comment: ...yes, according to this, we have only two remaining places, both in _lsprof, where we produce the warning "this calls an _elidable_function_, but this contradicts other sources": http://buildbot.pypy.org/builders/pypy-c-jit-linux-x86-64/builds/2030/steps/... I'd suggest we fix them and then turn the warning into an error. ________________________________________ PyPy bug tracker <tracker@bugs.pypy.org> <https://bugs.pypy.org/issue1328> ________________________________________
Armin Rigo <armin.rigo@gmail.com> added the comment: Seems to be resolved after some checkings ending in 75f2886305e0. ---------- status: chatting -> resolved ________________________________________ PyPy bug tracker <tracker@bugs.pypy.org> <https://bugs.pypy.org/issue1328> ________________________________________
participants (3)
-
Armin Rigo -
Carl Friedrich Bolz
-
Daniel Roberts