[issue10900] bz2 module fails to uncompress large files

[Eric Wolf]

Eric Wolf <ebwolf at gmail.com> added the comment:

Stupid questions are always worth asking. I did check the MD5 sum earlier and just checked it again (since I copied the file from one machine to another):

ebwolf at ubuntu:/opt$ md5sum /host/full-planet-110115-1800.osm.bz2 
0e3f81ef0dd415d8f90f1378666a400c  /host/full-planet-110115-1800.osm.bz2
ebwolf at ubuntu:/opt$ cat full-planet-110115-1800.osm.bz2.md5 
0e3f81ef0dd415d8f90f1378666a400c  full-planet-110115-1800.osm.bz2

There you have it. I was able to convert the bz2 to gzip with no errors:

bzcat full-planet-110115-1800.osm.bz2 | gzip > full-planet.osm.gz

FYI: This problem came up last year with no resolution:

http://mail.python.org/pipermail/tutor/2010-February/074610.html

Thanks for looking at this. Let me know if there's anything else you'd like me to try. In general, is it best to always read the same number of bytes? And what is the best value to pass for buffering in BZ2File? I just made up something hoping it would work.

I'm still waiting on the bzcat to /dev/null

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue10900>
_______________________________________