[docs] [issue10376] ZipFile unzip is unbuffered
report at bugs.python.org
Fri May 11 00:24:07 CEST 2012
Serhiy Storchaka <storchaka at gmail.com> added the comment:
This is not because zipfile module is unbuffered. This is the difference between expensive function call and cheap bytes slicing. Replace `zf.open(namelist )` to `io.BufferedReader(zf.open(namelist ))` to see the effect of a good buffering. In 3.2 zipfile read() implemented not optimal, so it slower (twice), but in 3.3 it will be almost as fast as using io.BufferedReader. It is still several times more slowly than bytes slicing, but there's nothing you can do with it.
Here is a patch, which is speeds up (+20%) the reading from a zip file by small chunks. Microbenchmark:
./python -m zipfile -c test.zip python
./python -m timeit -n 1 -s "import zipfile;zf=zipfile.ZipFile('test.zip')" "with zf.open('python') as f:" " while f.read(1):pass"
Python 3.3 (vanilla): 1 loops, best of 3: 36.4 sec per loop
Python 3.3 (patched): 1 loops, best of 3: 30.1 sec per loop
Python 3.3 (with io.BufferedReader): 1 loops, best of 3: 30.2 sec per loop
And, for comparison, Python 3.2: 1 loops, best of 3: 74.5 sec per loop
versions: -Python 2.7, Python 3.2
Added file: http://bugs.python.org/file25530/zipfile_optimize_read.patch
Python tracker <report at bugs.python.org>
More information about the docs