[Numpy-discussion] Problem with absolute value

[Charles R Harris]

On Wed, Jul 3, 2019 at 9:08 AM Hameer Abbasi <einstein.edison at gmail.com>
wrote:

> Hi,
>
> It turns out you're running into a bit-error. In general, the two's
> complement of -2 ** (n-1) with the bit-length being limited to n bits is
> itself... No way around that. And integers don't set hardware exceptions so
> checking for errors like these is hard as well.
>
> TL;DR: It's an error with how the integer is stored in memory and how
> you're running out of space.
>
> Regards,
> Hameer Abbasi
>

More like the eight bit twos complement of -128 is -128, bytes cannot
represent 128. Matlab used to (still does?) solve this problem by returning
127 instead :) Basically, the data needs more precision. Returning an
unsigned type would lead to it's own problems with unexpected promotions
when the result was used. We could, I suppose, raise a warning, although
that might be considered noisy. If you use `absolute` instead, you can
specify the dtype.

<snip>

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20190703/ac3d0d3a/attachment.html>