MemoryError when list append... plz help

[Steven D'Aprano]

On Tue, 30 Dec 2008 22:02:49 -0800, [BON] wrote:

> ======================
> s=[]
> for i in range(11000-1):
>     for j in range(i+1, 11000):
>         ....
>         s.append(((i,j),sim))
> ======================
> above sim is floating type.
> s.append is totally coducted 60,494,500 times. but this code raise
> MemoryError.
> 
> My computer has 4G RAM.
> i think it's enough. but it doesn't...

Your computer might have 4GB, but how much memory can Python allocate? 
What operating system are you using?

Each time you are appending to the list, you append a tuple:

((i, j), sim)

where sim is a float and i and j are ints. How much memory does each of 
those take?

>>> sys.getsizeof( ((0, 1), 1.1) )
32


So each entry requires 32 bytes. 60 million times 32 bytes = almost 2GB 
alone. Plus the list itself will require (approximately) between 230MB 
and 460MB just for the pointers.

What are you expecting to do with this enormous list, and why do you need 
it all at once?



-- 
Steven