readlines() with large file raises: IOError: [Errno 12] Cannot allocate memory

[Andreas Kostyrka]

On Mon, Nov 22, 2004 at 10:06:24AM +0100, nicogrubert at arcor.de wrote:
> Hi there
>  
> I am trying to read the content of a really large text file (1GByte) and I get
> the following exception if I try to call readlines() on the opened textfile:
> 
>     IOError: [Errno 12] Cannot allocate memory
> 
> 
> This is my code:
> INFILE="/home/myuser/myfile"
> open_infile = open( INFILE, 'r' )
> lines = open_infile.readlines()
> 
> 
> Any idea whats going on there?
> Does pyhon tries to put the content into machine's RAM (512 MB) which is not big enough so an
> exception is raised?
Exactly.

If you just want to read it line by line:
linecount = 0
for line in open_infile:
    linecount += 1
> 
> Thanks in advance,
> Nico
> 
> Arcor-DSL: die echte Flatrate für alle Bandbreiten. Jetzt ohne Einrichtungspreis
> einsteigen oder wechseln. Arcor-DSL ist in vielen Anschlussgebieten verfügbar.
> http://www.arcor.de/home/redir.php/emf-dsl-1
> 
> --
> http://mail.python.org/mailman/listinfo/python-list