[Numpy-discussion] ~2**32 byte tofile()/fromfile() limit in 64-bit Windows?

[Ralf Gommers]

On Thu, Nov 4, 2010 at 5:56 AM, Christoph Gohlke <cgohlke at uci.edu> wrote:
>
>
> On 11/3/2010 1:13 AM, Martin Spacek wrote:
>> I just opened a new ticket (http://projects.scipy.org/numpy/ticket/1660), but I
>> thought I'd bring it up here as well. I can't seem to get tofile() or save() to
>> write anything much bigger than a 2**32 byte array to a file in Py 2.6.6 on
>> 64-bit Windows. They both hang with no errors. Also, fromfile() throws an
>> "IOError: could not seek in file" on>  2**32-1 byte files, although load() seems
>> to work fine on any size of file. I'm a bit surprised I haven't stumbled across
>> these problems before. I've tested as far back as 1.4.1. Is this a known issue?
>>
>> (the ticket has much more detail)
>>
>> Cheers,
>>
>> Martin
>>
>
> I have attached a patch to ticket #1660. Any chance it could be
> considered for numpy 1.5.1?

To me it seems tofile/save being broken for some use cases is a more
serious problem than the other patches proposed. Also the change only
affects 64-bit Windows. So I think it can go in if it's merged in time
(i.e. within a couple of days). I'd prefer someone more familiar with
that particular code to review and merge it.

Cheers,
Ralf