readlines() with large file raises: IOError: [Errno 12] Cannot allocate memory
craig at postnewspapers.com.au
Mon Nov 22 10:17:47 CET 2004
On Mon, 2004-11-22 at 17:06, nicogrubert at arcor.de wrote:
> Does pyhon tries to put the content [of the readlines() operation ]
> into machine's RAM (512 MB) which is not big enough so an
> exception is raised?
Pretty much, yeah. A memory allocation failed, probably because of
running out of RAM and swap, as it loaded the entire file into RAM and
made a list out of it.
Consider using file.xreadlines() instead - that uses an iterator to the
job and is much more memory efficient.
More information about the Python-list