[Numpy-discussion] memory error caused by astype()

Gong, Shawn (Contractor) Shawn.Gong at drdc-rddc.gc.ca
Wed Aug 22 12:36:09 EDT 2007

Hi list,

When I do large array manipulations, I get out-of-memory errors.
For instance if the array size is 5000 by 6000, the following codes use
nearly 1G of RAM. 
Then my PC displays a Python error box.  The try/except won't even catch
it if the error happens in "astype" instead of "array1* array2"

        if ( array1.typecode() in cplx_types ):
            array1 = abs(array1.astype(Numeric.Complex32))
            array1 = array1.astype(Numeric.Float32)

        if ( array2.typecode() in cplx_types ):
            array2 = abs(array2.astype(Numeric.Complex32))
            array2 = array2.astype(Numeric.Float32)

        array1 = Numeric.sqrt(array1) * Numeric.sqrt(array2)
        return array1

        gvutils.error("Memory error occurred\nPlease select a smaller
        return None

My questions are:
1)	Is there a more memory efficient way instead of using astype?
2)	If not, then how do I catch error during astype?
3)	Is there a way in Python that detects the available RAM and
limits the array size before he/she can go ahead with the array
			i.e. detects the available RAM, say 1G
			Assume the worst case - Complex32
			Figure out the array size limit and warn user

Shaw Gong

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20070822/d865b777/attachment.html>

More information about the NumPy-Discussion mailing list