MemoryError using NumPy

[Emile van Sebille]

Win2k-sp2, Zope2.4, Numeric 20.1.0, Python2.1, 1.5Gb memory.

I'm slinging some large int32 numeric arrays, and at one point do:

Numeric.sum(Numeric.take(myDataArray, indices)).astype(Numeric.Float64)

This causes a temporary jump in memory usage (from ~800Mb to ~1.2Gb)
before settling down again, but it delivers the speed I'm looking for
(~7 secs vs ~30 summing in loop).  All starts well, but soon I get
MemoryError "can't allocate memory for array" only when indices is
large.  I suspect that memory has fragmented such that a sufficiently
large contiguous block is not available to set up the array.  Is this
the likely cause, and if so is there a way to un-fragment memory?  Or
might there be an alternative to Numeric.take that gets to the same
place?

I haven't tested this on the linux side, as the target system is Win2k I
need it to work here.  If it'll help though, I can do that.

Regards,

--

Emile van Sebille
emile at fenx.com

---------