pypy-2.3.1 memory profiling strategies
Hi all, Not sure if it's appropriate to post here since there's no pypy-user group :) My question is as title, we got memory leak on pypy process and process will down when out of memory, only on production site. Our simplified environment as below: * OS: Centos 6 * pypy-2.3.1 objgraph is seems the only profiling library we can use in this env, and only with its partial function of printing all current objects in memory instead of any further info such as references (.getrefcount not implemented). It turns out we can only see lots of "int", "str", "list" objects seems leaking rather than knowing who are using them or whom they are using. :( Our constraint are * it's hard to change production python runtime since it might affect to our users * we cannot reproduce on other environment "pmap" produced data only shows memory growing in a [anon] block. Please advise if there's other tool/methodologies to attack this problem, thanks a lot in advance :) Best Regards, Jim(洪懷謙)
Hi Phyo, thanks for prompt response :) I didn't state clear that I already tried periodic gc but memory usage still increase over team till out of memory, so we assume it's a real leak cause by python program but we don't have good tool/methodology to hunt it, thanks :) Best Regards, Jim(洪懷謙) On Sun, Aug 2, 2015 at 4:35 AM, Phyo Arkar <phyo.arkarlwin@gmail.com> wrote:
Hi Jim, On 2 August 2015 at 08:57, jim horng <jimhorng@gmail.com> wrote:
I believe you are failing to understand Phyo's answer: PyPy 2.3.1 had some internal memory leaks that were fixed later. If you're running into these, there is nothing you can do but upgrade to a more recent PyPy. There are no Python-level tricks that can free this leaked memory. A bientôt, Armin.
Hi Phyo, thanks for prompt response :) I didn't state clear that I already tried periodic gc but memory usage still increase over team till out of memory, so we assume it's a real leak cause by python program but we don't have good tool/methodology to hunt it, thanks :) Best Regards, Jim(洪懷謙) On Sun, Aug 2, 2015 at 4:35 AM, Phyo Arkar <phyo.arkarlwin@gmail.com> wrote:
Hi Jim, On 2 August 2015 at 08:57, jim horng <jimhorng@gmail.com> wrote:
I believe you are failing to understand Phyo's answer: PyPy 2.3.1 had some internal memory leaks that were fixed later. If you're running into these, there is nothing you can do but upgrade to a more recent PyPy. There are no Python-level tricks that can free this leaked memory. A bientôt, Armin.
participants (3)
-
Armin Rigo -
jim horng -
Phyo Arkar