[SciPy-user] Arrays and strange memory usage ...
Francesc Alted
falted at pytables.org
Wed Sep 3 04:49:23 EDT 2008
A Tuesday 02 September 2008, christophe grimault escrigué:
> Hi,
>
> I have a application that is very demanding in memory ressources. So
> I started to to look closer at python + numpy/scipy as far as memory
> is concerned.
>
> I can't explain the following :
>
> I start my python, + import scipy. A 'top' in the console shows that
> :
>
> PID USER PR NI VIRT RES SHR S %CPU %MEM TIME COMMAND
> 14791 grimault 20 0 21624 8044 3200 S 0 0.4 0:00.43 python
>
> Now after typing :
>
> z = scipy.arange(1000000)
>
> I get :
> 14791 grimault 20 0 25532 11m 3204 S 0 0.6 0:00.44 python
>
> So the memory increased by ~ 7 Mb. I was expecting 4 Mb since the
> data type is int32, giving 4*1000000 = 4 Mb of memory chunk (in C/C++
> at least).
You should look at the "RES" column instead of "VIRT" one. "RES" column
shows the *real* memory that you are consuming. So, in this case, you
have consumed 11MB - 8044KB ~ 3 MB. However, you are undergoing here
the effects of number representation truncation. Your consumed memory
should be rather: 8044KB + 3906KB = 11950KB, but as it is converted to
MB (scale changes happens automatically in 'top' when the figures need
more than 4 digits to be represented), 11959KB is truncated and 950KB
are gone, so this is why the final figure you are seeing is 11MB. This
can be a bit misleading at first sight, but be sure that your machine
(and NumPy) is doing fine and works as expected.
Cheers,
--
Francesc Alted
More information about the SciPy-User
mailing list