[Pythonmac-SIG] Reading big text files

Corran Webster cwebster@math.tamu.edu
Tue, 8 Jun 1999 11:20:30 -0500 (CDT)


> Everything works fine, but with big files (more than 5 MB, and most of my
> PostScript files are bigger than that), I run into memory problems. This is
> the source of my script:

The problem is that readlines() reads the entire file into memory at
once.  Try the fileinput module which does much the same, but with lazy
evaluation - it only reads the lines from the file as they are needed.

Change as follows:

>  import macfs
>  import sys
>  import EasyDialogs

   import fileinput

>  fsspec, ok = macfs.PromptGetFile('File to check end-of-lines in:', 'TEXT')
>  if not ok:
>   sys.exit(0)
> 
>  pathname = fsspec.as_pathname()
>  fp = open(pathname, 'r')

   for x in fileinput.input(fp):

>   if x[:9] == "%%Pages: ":
>    print "Found", x[9:-1], "pages!"
>    break
> 
>  fp.close()
> 
> Anyone an idea how I can handle this problem. I also have some problems with
> the different line-ending style from the different platforms. How should I
> handle this??

Not so sure about this since I haven't run into it recently, but it's a
lmost certainly a FAQ - I'll leave it for others to answer.

Corran