[issue11849] glibc allocator doesn't release all free()ed memory

[Charles-François Natali]

Charles-François Natali <neologix at free.fr> added the comment:

> However, there's still another strange regression:
>
> $ ./python -m timeit \
>   -s "n=300000; f=open('10MB.bin', 'rb', buffering=0); b=bytearray(n)" \
>   "f.seek(0);f.readinto(b)"
>
> -> default branch:
> 10000 loops, best of 3: 43 usec per loop
> -> default branch with patch reverted:
> 10000 loops, best of 3: 27.5 usec per loop
>
> FileIO.readinto executes a single read() into the passed buffer.

On my box:
default:
$ ./python -m timeit -s "n=300000; f=open('/tmp/10MB.bin', 'rb');
b=bytearray(n)" "f.seek(0);f.readinto(b)"
1000 loops, best of 3: 640 usec per loop

default without patch ("$ hg revert -r 68258 Objects/obmalloc.c && make"):
$ ./python -m timeit -s "n=300000; f=open('/tmp/10MB.bin', 'rb');
b=bytearray(n)" "f.seek(0);f.readinto(b)"
1000 loops, best of 3: 663 usec per loop

I'm just observing a random variance (but my computer is maybe too
slow to notice).
However, I really don't see how the patch could play a role here.

Concerning the slight performance regression, if it's a problem, I see
two options:
- revert the patch
- replace calls to malloc()/free() by mmap()/munmap() to allocate/free
arenas (but I'm not sure anonymous mappings are supported by every OS
out there, so this might lead to some ugly #ifdef's...)

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue11849>
_______________________________________