[pypy-dev] pypy array module memory leak?

[Ben.Young at sungard.com]

Hi Armin,

.NET has an AddMemoryPressure call for just this situation. Perhaps the PyPy GCs need a similar thing?

Thanks,
Ben

-----Original Message-----
From: pypy-dev-bounces+python=theyoungfamily.co.uk at python.org [mailto:pypy-dev-bounces+python=theyoungfamily.co.uk at python.org] On Behalf Of Armin Rigo
Sent: 22 July 2011 12:20
To: Berend De Schouwer
Cc: pypy-dev
Subject: Re: [pypy-dev] pypy array module memory leak?

Hi Berend,

On Fri, Jul 22, 2011 at 12:42 PM, Berend De Schouwer
<berend.deschouwer at ucs-software.co.za> wrote:
> The following program has constant (+- 10 MB) memory usage in CPython,
> but it quickly leaks massive amounts of memory in pypy.

In theory it's not a leak, because the memory is eventually freed.
You can see this by adding "import gc; gc.collect()" in the loop; then
the memory usage remains stable.

Of course in practice it's bad.  The issue is that in order to trigger
a collection, we count the size of the allocated objects --- but we
ignore the fact that the temporary array.array created by a[2001:3000]
has also an attached raw-malloced array of 1000 integers.  Instead we
just count the base size of the array object, which is 4 or 5 words.
As a result we grossly misestimate the time at which we need to
trigger the next collection.

It needs to be fixed, but I'm not sure exactly how to do it generally
(as opposed to just fixing array.array, which doesn't help all other
similar situations).


A bientôt,

Armin.
_______________________________________________
pypy-dev mailing list
pypy-dev at python.org
http://mail.python.org/mailman/listinfo/pypy-dev