Memory error due to big input file

[skip at pobox.com]

    phoebe> I have a big text file with size more than 2GB. It turned out
    phoebe> memory error when reading in this file. Here is my python
    phoebe> script, the error occurred at line -- self.fh.readlines().

    phoebe> import math
    phoebe> import time

    phoebe> class textfile:
    phoebe>   def __init__(self,fname):
    phoebe>      self.name=fname
    phoebe>      self.fh=open(fname)
    phoebe>      self.fh.readline()
    phoebe>      self.lines=self.fh.readlines()

Don't do that.  The problem is that you are trying to read the entire file
into memory.  Learn to operate a line (or a few lines) at a time.  Try
something like:

    a = open("/home/sservice/nfbc/GenoData/CompareCalls3.diff")
    for line in a:
        do your per-line work here

-- 
Skip Montanaro - skip at pobox.com - http://www.smontanaro.net/
    when i wake up with a heart rate below 40, i head right for the espresso
    machine. -- chaos @ forums.usms.org