Hi,
I’m a bit surprised with the following code:
import numpy as np np.seterr(all='warn') Z = np.array([-128], dtype=np.byte) print(np.abs(Z))
[-128]
Obviously, it does not return the absolute value and I get no warning. Is it something expected ? (numpy 1.16.4, python 3.7.3)
Nicolas
Hi,
It turns out you're running into a bit-error. In general, the two's complement of -2 ** (n-1) with the bit-length being limited to n bits is itself... No way around that. And integers don't set hardware exceptions so checking for errors like these is hard as well.
TL;DR: It's an error with how the integer is stored in memory and how you're running out of space.
Regards, Hameer Abbasi
On 03.07.19, 17:03, "NumPy-Discussion on behalf of Nicolas Rougier" <numpy-discussion-bounces+einstein.edison=gmail.com@python.org on behalf of Nicolas.Rougier@inria.fr> wrote:
Hi,
I’m a bit surprised with the following code:
>>> import numpy as np >>> np.seterr(all='warn') >>> Z = np.array([-128], dtype=np.byte) >>> print(np.abs(Z)) [-128]
Obviously, it does not return the absolute value and I get no warning. Is it something expected ? (numpy 1.16.4, python 3.7.3)
Nicolas _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
On Wed, Jul 3, 2019 at 9:08 AM Hameer Abbasi einstein.edison@gmail.com wrote:
Hi,
It turns out you're running into a bit-error. In general, the two's complement of -2 ** (n-1) with the bit-length being limited to n bits is itself... No way around that. And integers don't set hardware exceptions so checking for errors like these is hard as well.
TL;DR: It's an error with how the integer is stored in memory and how you're running out of space.
Regards, Hameer Abbasi
More like the eight bit twos complement of -128 is -128, bytes cannot represent 128. Matlab used to (still does?) solve this problem by returning 127 instead :) Basically, the data needs more precision. Returning an unsigned type would lead to it's own problems with unexpected promotions when the result was used. We could, I suppose, raise a warning, although that might be considered noisy. If you use `absolute` instead, you can specify the dtype.
<snip>
Chuck
Hi,
On Wed, Jul 3, 2019 at 9:08 AM Charles R Harris charlesr.harris@gmail.com wrote:
On Wed, Jul 3, 2019 at 9:08 AM Hameer Abbasi einstein.edison@gmail.com wrote:
Hi,
It turns out you're running into a bit-error. In general, the two's complement of -2 ** (n-1) with the bit-length being limited to n bits is itself... No way around that. And integers don't set hardware exceptions so checking for errors like these is hard as well.
TL;DR: It's an error with how the integer is stored in memory and how you're running out of space.
Regards, Hameer Abbasi
More like the eight bit twos complement of -128 is -128, bytes cannot represent 128. Matlab used to (still does?) solve this problem by returning 127 instead :) Basically, the data needs more precision. Returning an unsigned type would lead to it's own problems with unexpected promotions when the result was used. We could, I suppose, raise a warning, although that might be considered noisy. If you use `absolute` instead, you can specify the dtype.
I still think this is a major wart in numpy's abs.
I'd really like to add a new function, `uabs` which would return an unsigned int for integer inputs. I'm happy to do a pull request if that also seems sensible to y'all,
Cheers,
Matthew
On Wed, Jul 3, 2019 at 11:39 AM Matthew Brett matthew.brett@gmail.com wrote:
Hi,
On Wed, Jul 3, 2019 at 9:08 AM Charles R Harris charlesr.harris@gmail.com wrote:
On Wed, Jul 3, 2019 at 9:08 AM Hameer Abbasi einstein.edison@gmail.com
wrote:
Hi,
It turns out you're running into a bit-error. In general, the two's
complement of -2 ** (n-1) with the bit-length being limited to n bits is itself... No way around that. And integers don't set hardware exceptions so checking for errors like these is hard as well.
TL;DR: It's an error with how the integer is stored in memory and how
you're running out of space.
Regards, Hameer Abbasi
More like the eight bit twos complement of -128 is -128, bytes cannot
represent 128. Matlab used to (still does?) solve this problem by returning 127 instead :) Basically, the data needs more precision. Returning an unsigned type would lead to it's own problems with unexpected promotions when the result was used. We could, I suppose, raise a warning, although that might be considered noisy. If you use `absolute` instead, you can specify the dtype.
I still think this is a major wart in numpy's abs.
I'd really like to add a new function, `uabs` which would return an unsigned int for integer inputs. I'm happy to do a pull request if that also seems sensible to y'all,
Seems reasonable, it would provide something for discussion. Note that the current abs is in the slot provided by Python numeric types. The main problem is that as soon as the unsigned result is combined with a signed type it will get promoted. Which would not be much of a problem except int64 -> double.
Chuck
Thanks for the quick explanation & answers. I was mostly puzzled by the absence of warning. In C you would get an overflow warning in such a case, no ?
Nicolas
On 3 Jul 2019, at 20:01, Charles R Harris charlesr.harris@gmail.com wrote:
On Wed, Jul 3, 2019 at 11:39 AM Matthew Brett matthew.brett@gmail.com wrote: Hi,
On Wed, Jul 3, 2019 at 9:08 AM Charles R Harris charlesr.harris@gmail.com wrote:
On Wed, Jul 3, 2019 at 9:08 AM Hameer Abbasi einstein.edison@gmail.com wrote:
Hi,
It turns out you're running into a bit-error. In general, the two's complement of -2 ** (n-1) with the bit-length being limited to n bits is itself... No way around that. And integers don't set hardware exceptions so checking for errors like these is hard as well.
TL;DR: It's an error with how the integer is stored in memory and how you're running out of space.
Regards, Hameer Abbasi
More like the eight bit twos complement of -128 is -128, bytes cannot represent 128. Matlab used to (still does?) solve this problem by returning 127 instead :) Basically, the data needs more precision. Returning an unsigned type would lead to it's own problems with unexpected promotions when the result was used. We could, I suppose, raise a warning, although that might be considered noisy. If you use `absolute` instead, you can specify the dtype.
I still think this is a major wart in numpy's abs.
I'd really like to add a new function, `uabs` which would return an unsigned int for integer inputs. I'm happy to do a pull request if that also seems sensible to y'all,
Seems reasonable, it would provide something for discussion. Note that the current abs is in the slot provided by Python numeric types. The main problem is that as soon as the unsigned result is combined with a signed type it will get promoted. Which would not be much of a problem except int64 -> double.
Chuck _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion