[issue10900] bz2 module fails to uncompress large files
Charles-Francois Natali
report at bugs.python.org
Tue Mar 1 12:27:57 CET 2011
Charles-Francois Natali <neologix at free.fr> added the comment:
@Eric.Wolf
Could you try with this:
# Read in anohter chunk of the file
# NOTE: It's possible that an XML tag will be greater than buffsize
# This will break in that situation
- newb = self.fp.read(self.bufpos)
+ newb = self.fp.read(self.buffsize)
Also, could you provide the output of
strace -emmap2,sbrk,brk python <script>
I could be completely wrong, but both in your case and in wrobell's case, there's a lot of _PyBytes_Resize going on, and given how PyObject_Realloc is implemented, this could lead to heavy heap fragmentation.
----------
nosy: +neologix
_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue10900>
_______________________________________
More information about the Python-bugs-list
mailing list