[issue3531] file read preallocs 'size' bytes which can cause memory problems

[Andrew Dalke]

Andrew Dalke <dalke at dalkescientific.com> added the comment:

You're right.  I mistook the string implementation for the list one 
which does keep a preallocated section in case of growth.  Strings of 
course don't grow so there's no need for that.

I tracked the memory allocation all the way down to 
obmalloc.c:PyObject_Realloc .  The call goes to realloc(p, nbytes) which 
is a C lib call.  It appears that the memory space is not reallocated.

That was enough to be able to find the python-dev thread "Darwin's 
realloc(...) implementation never shrinks allocations" from Jan. 2005, 
Bob Ippolito's post "realloc.. doesn’t?" 
(http://bob.pythonmac.org/archives/2005/01/01/realloc-doesnt/ ) and Issue1092502 .

Mind you, I also get the problem on FreeBSD 2.6 so it isn't Darwin 
specific.

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue3531>
_______________________________________