[Numpy-discussion] fwrite() failure in PyArray_ToFile

[Charles R Harris]

On Fri, Sep 4, 2009 at 9:22 PM, Sturla Molden <sturla at molden.no> wrote:

> David Warde-Farley skrev:
> >> The odd values might be from the format code in the error message:
> >>
> >>                PyErr_Format(PyExc_ValueError,
> >>                        "%ld requested and %ld written",
> >>                        (long) size, (long) n);
> >>
> >
> > Yes, I saw that. My C is rusty, but wouldn't the cast take care of it?
> > n is of type size_t, which is pretty big, and a cast to long shouldn't
> > be an issue. And if (hopefully) PyErr_Format is correct...
> >
> A long is (usually) 32 bit, even on 64 bit systems. This means that size
> is casted to an integer between - 2**31 and 2*31 - 1. As 2**31 bytes are
> 2 GB, the expression "(long) size" will overflow if a write of 2GB or
> more failed. Thus you get some bogus numbers in the formatted message.
> There is thus a bug in the call to PyErr_Format, as it only works
> reliably on 32 bits systems.
>
>
The size of long depends on the compiler as well as the operating system. On
linux x86_64, IIRC, it is 64 bits, on Windows64 I believe it is 32. Ints
always seem to be 32 bits. But something funny is definitely going on. It
shouldn't be possible to allocate an array bigger than npy_intp, which is a
signed number, and the casts to size_t and such should be safe. But in
anycase, using long in the print statement is not portable and needs fixing.

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20090904/5e338931/attachment.html>