[SciPy-User] Efficient 2-d arrays using standard python?

[David]

On 01/18/2011 07:55 AM, Lutz Maibaum wrote:

>
> How do you measure memory consumption? On my system (Snow Leopard, Python 2.6 64-bit) both expressions seem to use about the same amount of memory, at least if the operating system's Activity Monitor is any indication. Is the standard python integer type on your system 32 bit, as you request explicitly in the numpy version?

It may be true for this special case, but only because the special 
values you are using (0 here). In general, at least for CPython, a list 
of N integers will use *much* more memory than a a numpy array.

Basically, for a numpy array of N 32 bits integers, sizeof(array) = N * 
4 + overhead bytes, where overhead is mostly independent of N, and 
bounded anyway.

For a list of N integers, you have sizeof(list) >= overhead(list) + N * 
sizeof(PyObject*) + N * sizeof(integer), where sizeof(integer) is at 
least 12 bytes (4 bytes for the value, 4 bytes for the reference 
counting, and 4 bytes for the object pointer).

IOW, for a 32 bits machine (the best case), a list of N integers use 
something like 4 times the same space as a numpy array. On a 64 bits 
machine, the size taken by the numpy array will not change so much, 
whereas the size will be almost doubled for the list case (everything 
sees its size doubled except for the integer value itself).

cheers,

David