[Pythonmac-SIG] Reading big text files
Corran Webster
cwebster@math.tamu.edu
Tue, 8 Jun 1999 11:20:30 -0500 (CDT)
> Everything works fine, but with big files (more than 5 MB, and most of my
> PostScript files are bigger than that), I run into memory problems. This is
> the source of my script:
The problem is that readlines() reads the entire file into memory at
once. Try the fileinput module which does much the same, but with lazy
evaluation - it only reads the lines from the file as they are needed.
Change as follows:
> import macfs
> import sys
> import EasyDialogs
import fileinput
> fsspec, ok = macfs.PromptGetFile('File to check end-of-lines in:', 'TEXT')
> if not ok:
> sys.exit(0)
>
> pathname = fsspec.as_pathname()
> fp = open(pathname, 'r')
for x in fileinput.input(fp):
> if x[:9] == "%%Pages: ":
> print "Found", x[9:-1], "pages!"
> break
>
> fp.close()
>
> Anyone an idea how I can handle this problem. I also have some problems with
> the different line-ending style from the different platforms. How should I
> handle this??
Not so sure about this since I haven't run into it recently, but it's a
lmost certainly a FAQ - I'll leave it for others to answer.
Corran