MemoryError when list append... plz help

[Francesco Bochicchio]

[BON] ha scritto:
> ======================
> s=[]
> for i in range(11000-1): 
>     for j in range(i+1, 11000): 
>         ....
>         s.append(((i,j),sim)) 
> ======================
> above sim is floating type.
> s.append is totally coducted 60,494,500 times.
> but this code raise MemoryError.
> 
> My computer has 4G RAM.
> i think it's enough. but it doesn't...
> 
> So, i've tested below code.
> ======================
> a=[] 
> i=0 
> while i<60494500 : 
>     a.append(i) 
>     i+=1
> ======================
> but this code raise also MemoryError.
> 
> How can i resolve this problem?
> please, help...
> 
> Regards,


If you _really_ have to store so many numbers in memory (hint: if you 
are processing them sequentially, you don't need to store all them - use
generators instead) then you may have better luck using mmap module to
create a huge file-memory object, that you can access both as a file and 
as a list, and put numbers in it after packing/unpacking with struct.
Something like this (only marginally tested ):

 >>> memory = mmap.mmap(-1, 60494500*4)
 >>> def memory_put(offset, f):
...     memory[offset*4:(offset+1)*4] = struct.pack( "%f", f )
  >>> def memory_get(offset):
...     return struct.unpack( "f", memory[offset*4:(offset+1)*4] )[0]
 >>> memory_put(12, 3.14 )
 >>> memory_get(12)
3.1400001049041748

Ciao
------
FB