Re: [Numpy-discussion] reading large files

On 9/22/06, Bill Baxter wrote:

Do you also have a 64-bit processor? Just checking since you didn't mention it. --bb

On 9/22/06, Svein-Erik.Hamran@ffi.no wrote:

...

I would like to read files >2Gbyte. From earlier posting I believed it should be possible with python 2.5.

I am getting MemoryError when trying to read a file larger than 2Gb.

I am using python2.5 and numpy1.0rc1.

Also, how much memory do you have? You will have to use memory mapping as well if the file size is greater than your memory size. But you should be hopeful. I have been working with > 10 GB files for some time using beta versions of Python 2.5 and numpy-1.0 on a 64-bit AMD machine with 4 GB of memory. Mike -- mike.ressler@alum.mit.edu