[Python-Dev] ast status, memory leaks, etc
Neal Norwitz
nnorwitz at gmail.com
Mon Nov 21 01:14:07 CET 2005
On 11/20/05, "Martin v. Löwis" <martin at v.loewis.de> wrote:
>
> Can somebody please give a quick explanation how valgrind can give
> *any* reasonable leak analysis when obmalloc is used? In the current
> implementation, obmalloc never ever calls free(3), so all pool memory
> should appear to have leaked.
>
> So if valgrind does *not* report all memory as leaked: how does it
> find out?
Thanks for reminding me I wanted to do the next step and test without pymalloc.
Valgrind can't find certain kinds of leaks when pymalloc is holding on
to memory, true. However, remember that lots of allocations are
forwarded to the system malloc(). For example, any request > 256
bytes goes directly to system malloc. Also, PyMem_*() call the system
functions.
The core is pretty clean already, since I've been running Valgrind
pretty regularly over the years. Before Valgrind I used purify going
back to 2000 or 2001. Barry had used purify before me at some point
AFAIK. So nearly all of the leaks have already been fixed. It's
pretty much only new code that starts showing leaks.
To give you an example, I ran the entire regression suite through
Valgrind after configuring --without-pymalloc. I only found 3
additional problems in new code. There was also one problem in older
code (Python/modsupport.c).
The big benefit of running with pymalloc is that it only takes about
1.25 to 1.50 hours to run on my box. When running without pymalloc, I
estimate it takes about 5 times longer. Plus it requires a lot of
extra work since I need to run the tests in batches. I only have 1 GB
of RAM and it takes a lot more than that when running without
pymalloc.
> This is the resizing of the list of arenas, which is a deliberate
> leak. It just happened to be exhausted in this particular call
> stack.
Thanks I was going to look into the resizing and forgot about it.
Running without pymalloc confirmed that there weren't more serious
problems.
n
More information about the Python-Dev
mailing list