[Numpy-discussion] Catching out-of-memory error before it happens
Chris Barker - NOAA Federal
chris.barker at noaa.gov
Fri Jan 24 10:57:14 EST 2014
c = a + b: 3N
c = a + 2*b: 4N
Does python garbage collect mid-expression? I.e. :
C = (a + 2*b) + b
4 or 5 N?
Also note that when memory gets tight, fragmentation can be a problem. I.e.
if two size-n arrays where just freed, you still may not be able to
allocate a size-2n array. This seems to be worse on windows, not sure why.
a += b: 2N
np.add(a, b, out=a): 2N
b *= 2; a += b: 2N
Note that simply loading a and b requires 2N memory, so the latter code
samples are near-optimal.
And will run quite a bit faster for large arrays--pushing that memory
around takes time.
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the NumPy-Discussion