[Numpy-discussion] Is there a pure numpy recipe for this?

[Chris Barker]

On Thu, Mar 27, 2014 at 7:42 AM, RayS <rays at blue-cove.com> wrote:

> I find this interesting, since I work with medical data sets of 100s
> of MB, and regularly run into memory allocation problems when doing a
> lot of Fourrier analysis, waterfalls etc. The per-process limit seems
> to be about 1.3GB on this 6GB quad-i7 with Win7.


This sounds like 32 bit -- have you tried a 64 bit Python_numpy? Nt that
you wont have issues anyway, but you should be abel to do better than
1.3GB...


>  memmaps are also limited to RAM,


I don't think so, no -- but are limited to 2GB (I think)  if you're using a
32 bit process

There is also a compressed array package out there -- I can't remember what
it's called -- but if you have large compressible arrays -- that might help.

-CHB


-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20140327/937c32ff/attachment.html>