On Tue, Oct 11, 2011 at 2:51 PM, Matthew Brett <matthew.brett@gmail.com>wrote:
Hi
On Tue, Oct 11, 2011 at 3:16 PM, Charles R Harris <charlesr.harris@gmail.com> wrote:
On Tue, Oct 11, 2011 at 12:23 PM, Matthew Brett <matthew.brett@gmail.com
wrote:
Hi,
I recently ran into this:
In [68]: arr = np.array(-128, np.int8)
In [69]: arr Out[69]: array(-128, dtype=int8)
In [70]: np.abs(arr) Out[70]: -128
This has come up for discussion before, but no consensus was ever
One solution is for abs to return an unsigned type, but then combining
reached. that
with signed type of the same number of bits will cause both to be cast to higher precision. IIRC, matlab was said to return +127 as abs(-128), which, if true, is quite curious.
octave-3.2.3:1> a = int8([-128, 127]) a =
-128 127
octave-3.2.3:2> abs(a) ans =
127 127
Matlab is the same. That is curious...
See you,
Matthew
Well, it _is_ only off by 0.78%. That should be good enough for government work, right? Ben Root