Reading a large bz2 textfile exits early

Norman Rieß norman at smash-net.org
Sat Feb 20 23:12:50 CET 2010


Hello,

i am trying to read a large bz2 compressed textfile using the bz2 module.
The file is 1717362770 lines long and 8GB large.
Using this code

source_file = bz2.BZ2File(file, "r")
    for line in source_file:
    print line.strip()

    print "Exiting"
    print "I used file: " + file

the loop exits cleanly after 4311 lines in midline and the prints are
executed.
This happened on two different boxes runnig different brands of linux.
Is there something i miss or should be done differently?

Thank you.

Regards,
Norman




More information about the Python-list mailing list