[issue2523] binary buffered reading is quadratic

[Antoine Pitrou]

Antoine Pitrou <pitrou at free.fr> added the comment:

Here is a pure Python patch removing the quadratic behaviour and trying
to make read operations generally faster.

Here are some numbers:

./python -m timeit -s "f = open('50KB', 'rb')" "f.seek(0)" "while
f.read(11): pass"

-> py3k without patch: 23.6 msec per loop
-> py3k with patch: 14.5 msec per loop
-> Python 2.5: 4.72 msec per loop

./python -m timeit -s "f = open('50KB', 'rb')" "f.seek(0); f.read()"

-> py3k without patch: 284 usec per loop
-> py3k with patch: 90.1 usec per loop
-> Python 2.5: 33.8 usec per loop

./python -m timeit -s "f = open('100KB', 'rb')" "f.seek(0); f.read()"

-> py3k without patch: 828 usec per loop
-> py3k with patch: 142 usec per loop
-> Python 2.5: 62.5 usec per loop

./python -m timeit -s "f = open('200KB', 'rb')" "f.seek(0); f.read()"

-> py3k without patch: 3.67 msec per loop
-> py3k with patch: 375 usec per loop
-> Python 2.5: 131 usec per loop

And, for the record, with a 10MB file:

./python -m timeit -s "f = open('10MB', 'rb')" "f.seek(0); f.read()"

-> py3k without patch: still running after more than one minute, gave up
-> py3k with patch: 38.6 msec per loop
-> Python 2.5: 20.4 msec per loop

----------
keywords: +patch
Added file: http://bugs.python.org/file10216/binaryio1.patch

__________________________________
Tracker <report at bugs.python.org>
<http://bugs.python.org/issue2523>
__________________________________