[Numpy-discussion] Problem with absolute value
Charles R Harris
charlesr.harris at gmail.com
Wed Jul 3 12:04:56 EDT 2019
On Wed, Jul 3, 2019 at 9:08 AM Hameer Abbasi <einstein.edison at gmail.com>
wrote:
> Hi,
>
> It turns out you're running into a bit-error. In general, the two's
> complement of -2 ** (n-1) with the bit-length being limited to n bits is
> itself... No way around that. And integers don't set hardware exceptions so
> checking for errors like these is hard as well.
>
> TL;DR: It's an error with how the integer is stored in memory and how
> you're running out of space.
>
> Regards,
> Hameer Abbasi
>
More like the eight bit twos complement of -128 is -128, bytes cannot
represent 128. Matlab used to (still does?) solve this problem by returning
127 instead :) Basically, the data needs more precision. Returning an
unsigned type would lead to it's own problems with unexpected promotions
when the result was used. We could, I suppose, raise a warning, although
that might be considered noisy. If you use `absolute` instead, you can
specify the dtype.
<snip>
Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20190703/ac3d0d3a/attachment.html>
More information about the NumPy-Discussion
mailing list