Buffer growth, lazy opitimization

Tim Peters tim.one at comcast.net
Wed May 1 15:17:43 EDT 2002


[Stephen D Evans]
> ...
> I have had a look at the 'Interpreter Changes and Fixes' in the Python
> 2.3 documentation (release 0.01)
>
> http://www.python.org/dev/doc/devel/whatsnew/node6.html
>
> Growing string buffers was mentioned as being 'mildly exponential'. A
> very quick look in the Python CVS at python/dist/src/Objects/fileobject.c
? showed that two different methods of buffer reallocation were being used
> in this source file - fixed increment and doubling.

Please be specific.  This is a 2K line file and has many buffer resizing
schemes, depending on context.  As the docs you referenced say, the specific
buffer resizing scheme it's talking about is in get_line(), which is indeed
mildly exponential now (a growth factor of roughly 1.25).  In real life
tests, it made almost no difference compared to the previous fixed-increment
scheme, except (as the docs say) when running the extreme test_bufio in a
debug-mode build.  Note that this routine reads a line of text from a text
file, and the million-character lines test_bufio sets up are (deliberately)
unrealistic.






More information about the Python-list mailing list