Re: [Numpy-discussion] create a numpy array of images
Hi Zach and Sturla,
Well I am a "she" :)) Thanks for your inputs.. I am using 32 bit python as
have so many libraries integrated with it.. and moreover, i plan to put this
volume rendered on a web page or distribute the exe in the end, so want to
be minimal for the memory requirements on the clients' systems..
the physical memory should not be a problem as I have 8GB RAM.. specially
when the images are RGB then it gets into trouble as it adds the 4th
dimension already in my case..
- asmi
---------- Forwarded message ----------
From: Sturla Molden
numpy. as I have like 200 images to stack in the numpy array of say 1024x1344 resolution.. have any idea apart from downsampling?
Take a look at numpy.memmap or h5py [1].
memmap will not help unless he uses 64 bit Python, in which case he can just buy more RAM if he has too little. I suspect he is running out of virtual memory, not physical, for which 64 bit is the easiest solution. It is not possible to compensate for lack of virtual memory (typically 2GB limit with 32 bit) by memory mapping file into the already exhausted memory space. Using a database like h5py will help too, unless he tries to extract them all at once. Sturla
Den 01.02.2011 15:07, skrev Asmi Shah:
Hi Zach and Sturla,
Well I am a "she" :))
I apologize, I did not deduce correct gender from your name :)
Thanks for your inputs.. I am using 32 bit python as have so many libraries integrated with it.. and moreover, i plan to put this volume rendered on a web page or distribute the exe in the end, so want to be minimal for the memory requirements on the clients' systems.. the physical memory should not be a problem as I have 8GB RAM.. specially when the images are RGB then it gets into trouble as it adds the 4th dimension already in my case..
With 32 bit each process only has 2 or 3 GB (dependent on OS setting) of those 8 GB available in user space. If 32 bit is a requirement, you have to store some of the data on disk to prevent exhausting the 2 GB virtual memory limit. Actually, if you have 200 16-bit images in 1024 x 1344 resolution, that is only 525 MB if stored compactly with dtype np.uint16. So check how those images are stored in your stack. Do you use floating point instead of the smallest possible integer? Do you use separate dimensions for RGBA channels? Is there anything else wasting memory besides this stack of images? Do you have a memory leak? Sturla
participants (2)
-
Asmi Shah
-
Sturla Molden