Dan Halbert halbert at halwitz.org
Fri Apr 29 13:06:43 EDT 2011

On Thursday, April 28, 2011 5:22pm, "Dan Halbert" <halbert at halwitz.org> said:
> I'm having trouble loading a large remote .npy file on Windows XP. This is on
> numpy-1.3.0 on Windows XP SP3:
>
>
> will fail with this sort of error being printed:
>     "14328000 items requested but only 54 read"

I have tracked this down: it's not specific to numpy, or even Python, though
there's a Python bug report about it:
"size limit exceeded for read() from network drive"
http://bugs.python.org/issue1478529

The problem stems from a buffer size limit of not quite 64MB for the Windows API

Apparently most of the time the underlying Windows code is cognizant of this limit,
but under certain circumstances, a read from a large networked file is not handled
correctly (I think when there is also an entry in "My Network Places" for the path).
It's quite obscure, and there's nothing that can be fixed at the numpy level.

Dan