Hi, Our Debian friends were hammering our code tests before the upcoming freeze, and found the following very odd thing on 32-bit PPC running Debian squeeze and numpy 1.6.2 or current trunk. Consider the following script: <script> import numpy as np arr = np.arange(10, dtype=np.complex256) bs_arr = arr.byteswap().newbyteorder('S') print arr print bs_arr print arr == bs_arr print arr == bs_arr print arr == bs_arr </script> Here is the output from some example runs of this script: np-devel)[mb312@joshlegacy ~/tmp]$ python funny_bs.py [ 0.0+0.0j 1.0+0.0j 2.0+0.0j 3.0+0.0j 4.0+0.0j 5.0+0.0j 6.0+0.0j 7.0+0.0j 8.0+0.0j 9.0+0.0j] [ 0.0+0.0j 1.0+0.0j 2.0+0.0j 3.0+0.0j 4.0+0.0j 5.0+0.0j 6.0+0.0j 7.0+0.0j 8.0+0.0j 9.0+0.0j] [ True False False False False False False False False False] [ True False False False False False False False False False] [ True False False False False False False False False False] (that's the most common result) (np-devel)[mb312@joshlegacy ~/tmp]$ python funny_bs.py [ 0.0+0.0j 1.0+0.0j 2.0+0.0j 3.0+0.0j 4.0+0.0j 5.0+0.0j 6.0+0.0j 7.0+0.0j 8.0+0.0j 9.0+0.0j] [ 0.0+0.0j 1.0+0.0j 2.0+0.0j 3.0+0.0j 4.0+0.0j 5.0+0.0j 6.0+0.0j 7.0+0.0j 8.0+0.0j 9.0+0.0j] [ True True True True True True True True True True] [ True True True True True True True True True True] [ True True True True True True True True True True] (this happens maybe 10% of the time) (np-devel)[mb312@joshlegacy ~/tmp]$ python funny_bs.py [ 0.0+0.0j 1.0+0.0j 2.0+0.0j 3.0+0.0j 4.0+0.0j 5.0+0.0j 6.0+0.0j 7.0+0.0j 8.0+0.0j 9.0+0.0j] [ 0.0+0.0j 1.0+0.0j 2.0+0.0j 3.0+0.0j 4.0+0.0j 5.0+0.0j 6.0+0.0j 7.0+0.0j 8.0+0.0j 9.0+0.0j] [ True True True True True True True True True True] [ True False False False False False False False False False] [ True True True True True True True True True True] (less than 10% of the time - order of True, False prints _of the same comparison_ is random. This only seems to happen with complex256. Is there anything I can do to debug this further? Does anyone want a login to this machine to have a look? See you, Matthew
This looks like a problem with comparisons of floating point numbers rather than a byteswapping problem per-say. Try to use an almost equal comparison instead. -Travis On Jun 20, 2012, at 3:48 PM, Matthew Brett wrote:
Hi,
Our Debian friends were hammering our code tests before the upcoming freeze, and found the following very odd thing on 32-bit PPC running Debian squeeze and numpy 1.6.2 or current trunk.
Consider the following script:
<script> import numpy as np
arr = np.arange(10, dtype=np.complex256) bs_arr = arr.byteswap().newbyteorder('S') print arr print bs_arr print arr == bs_arr print arr == bs_arr print arr == bs_arr </script>
Here is the output from some example runs of this script:
np-devel)[mb312@joshlegacy ~/tmp]$ python funny_bs.py [ 0.0+0.0j 1.0+0.0j 2.0+0.0j 3.0+0.0j 4.0+0.0j 5.0+0.0j 6.0+0.0j 7.0+0.0j 8.0+0.0j 9.0+0.0j] [ 0.0+0.0j 1.0+0.0j 2.0+0.0j 3.0+0.0j 4.0+0.0j 5.0+0.0j 6.0+0.0j 7.0+0.0j 8.0+0.0j 9.0+0.0j] [ True False False False False False False False False False] [ True False False False False False False False False False] [ True False False False False False False False False False]
(that's the most common result)
(np-devel)[mb312@joshlegacy ~/tmp]$ python funny_bs.py [ 0.0+0.0j 1.0+0.0j 2.0+0.0j 3.0+0.0j 4.0+0.0j 5.0+0.0j 6.0+0.0j 7.0+0.0j 8.0+0.0j 9.0+0.0j] [ 0.0+0.0j 1.0+0.0j 2.0+0.0j 3.0+0.0j 4.0+0.0j 5.0+0.0j 6.0+0.0j 7.0+0.0j 8.0+0.0j 9.0+0.0j] [ True True True True True True True True True True] [ True True True True True True True True True True] [ True True True True True True True True True True]
(this happens maybe 10% of the time)
(np-devel)[mb312@joshlegacy ~/tmp]$ python funny_bs.py [ 0.0+0.0j 1.0+0.0j 2.0+0.0j 3.0+0.0j 4.0+0.0j 5.0+0.0j 6.0+0.0j 7.0+0.0j 8.0+0.0j 9.0+0.0j] [ 0.0+0.0j 1.0+0.0j 2.0+0.0j 3.0+0.0j 4.0+0.0j 5.0+0.0j 6.0+0.0j 7.0+0.0j 8.0+0.0j 9.0+0.0j] [ True True True True True True True True True True] [ True False False False False False False False False False] [ True True True True True True True True True True]
(less than 10% of the time - order of True, False prints _of the same comparison_ is random. This only seems to happen with complex256.
Is there anything I can do to debug this further? Does anyone want a login to this machine to have a look?
See you,
Matthew _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Hi, On Wed, Jun 20, 2012 at 1:56 PM, Travis Oliphant <travis@continuum.io> wrote:
This looks like a problem with comparisons of floating point numbers rather than a byteswapping problem per-say. Try to use an almost equal comparison instead.
Is that right - that the byteswapped versions might not be strictly equal to identical numbers but not byteswapped? But I should maybe have been clearer - they also subtract wrongly: <script> import numpy as np arr = np.arange(10, dtype=np.complex256) bs_arr = arr.byteswap().newbyteorder('S') print arr print bs_arr print arr - bs_arr print arr - bs_arr print arr - bs_arr </script> (np-devel)[mb312@joshlegacy ~/tmp]$ python funny_bs.py [ 0.0+0.0j 1.0+0.0j 2.0+0.0j 3.0+0.0j 4.0+0.0j 5.0+0.0j 6.0+0.0j 7.0+0.0j 8.0+0.0j 9.0+0.0j] [ 0.0+0.0j 1.0+0.0j 2.0+0.0j 3.0+0.0j 4.0+0.0j 5.0+0.0j 6.0+0.0j 7.0+0.0j 8.0+0.0j 9.0+0.0j] [ 0.0+0.0j 1.0+0.0j 2.0+0.0j 3.0+0.0j 4.0+0.0j 5.0+0.0j 6.0+0.0j 7.0+0.0j 8.0+0.0j 9.0+0.0j] [ 0.0+0.0j 1.0+0.0j 2.0+0.0j 3.0+0.0j 4.0+0.0j 5.0+0.0j 6.0+0.0j 7.0+0.0j 8.0+0.0j 9.0+0.0j] [ 0.0+0.0j 1.0+0.0j 2.0+0.0j 3.0+0.0j 4.0+0.0j 5.0+0.0j 6.0+0.0j 7.0+0.0j 8.0+0.0j 9.0+0.0j] (wrong) (np-devel)[mb312@joshlegacy ~/tmp]$ python funny_bs.py [ 0.0+0.0j 1.0+0.0j 2.0+0.0j 3.0+0.0j 4.0+0.0j 5.0+0.0j 6.0+0.0j 7.0+0.0j 8.0+0.0j 9.0+0.0j] [ 0.0+0.0j 1.0+0.0j 2.0+0.0j 3.0+0.0j 4.0+0.0j 5.0+0.0j 6.0+0.0j 7.0+0.0j 8.0+0.0j 9.0+0.0j] [ 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j] [ 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j] [ 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j] (right) See you, Matthew
That is clearly wrong and worth a bug report. Travis -- Travis Oliphant (on a mobile) 512-826-7480 On Jun 20, 2012, at 5:00 PM, Matthew Brett <matthew.brett@gmail.com> wrote:
Hi,
On Wed, Jun 20, 2012 at 1:56 PM, Travis Oliphant <travis@continuum.io> wrote:
This looks like a problem with comparisons of floating point numbers rather than a byteswapping problem per-say. Try to use an almost equal comparison instead.
Is that right - that the byteswapped versions might not be strictly equal to identical numbers but not byteswapped?
But I should maybe have been clearer - they also subtract wrongly:
<script> import numpy as np
arr = np.arange(10, dtype=np.complex256) bs_arr = arr.byteswap().newbyteorder('S') print arr print bs_arr print arr - bs_arr print arr - bs_arr print arr - bs_arr </script>
(np-devel)[mb312@joshlegacy ~/tmp]$ python funny_bs.py [ 0.0+0.0j 1.0+0.0j 2.0+0.0j 3.0+0.0j 4.0+0.0j 5.0+0.0j 6.0+0.0j 7.0+0.0j 8.0+0.0j 9.0+0.0j] [ 0.0+0.0j 1.0+0.0j 2.0+0.0j 3.0+0.0j 4.0+0.0j 5.0+0.0j 6.0+0.0j 7.0+0.0j 8.0+0.0j 9.0+0.0j] [ 0.0+0.0j 1.0+0.0j 2.0+0.0j 3.0+0.0j 4.0+0.0j 5.0+0.0j 6.0+0.0j 7.0+0.0j 8.0+0.0j 9.0+0.0j] [ 0.0+0.0j 1.0+0.0j 2.0+0.0j 3.0+0.0j 4.0+0.0j 5.0+0.0j 6.0+0.0j 7.0+0.0j 8.0+0.0j 9.0+0.0j] [ 0.0+0.0j 1.0+0.0j 2.0+0.0j 3.0+0.0j 4.0+0.0j 5.0+0.0j 6.0+0.0j 7.0+0.0j 8.0+0.0j 9.0+0.0j]
(wrong)
(np-devel)[mb312@joshlegacy ~/tmp]$ python funny_bs.py [ 0.0+0.0j 1.0+0.0j 2.0+0.0j 3.0+0.0j 4.0+0.0j 5.0+0.0j 6.0+0.0j 7.0+0.0j 8.0+0.0j 9.0+0.0j] [ 0.0+0.0j 1.0+0.0j 2.0+0.0j 3.0+0.0j 4.0+0.0j 5.0+0.0j 6.0+0.0j 7.0+0.0j 8.0+0.0j 9.0+0.0j] [ 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j] [ 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j] [ 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j]
(right)
See you,
Matthew _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
On Wed, Jun 20, 2012 at 4:00 PM, Matthew Brett <matthew.brett@gmail.com>wrote:
Hi,
On Wed, Jun 20, 2012 at 1:56 PM, Travis Oliphant <travis@continuum.io> wrote:
This looks like a problem with comparisons of floating point numbers rather than a byteswapping problem per-say. Try to use an almost equal comparison instead.
Is that right - that the byteswapped versions might not be strictly equal to identical numbers but not byteswapped?
But I should maybe have been clearer - they also subtract wrongly:
<script> import numpy as np
arr = np.arange(10, dtype=np.complex256) bs_arr = arr.byteswap().newbyteorder('S') print arr print bs_arr print arr - bs_arr print arr - bs_arr print arr - bs_arr </script>
(np-devel)[mb312@joshlegacy ~/tmp]$ python funny_bs.py [ 0.0+0.0j 1.0+0.0j 2.0+0.0j 3.0+0.0j 4.0+0.0j 5.0+0.0j 6.0+0.0j 7.0+0.0j 8.0+0.0j 9.0+0.0j] [ 0.0+0.0j 1.0+0.0j 2.0+0.0j 3.0+0.0j 4.0+0.0j 5.0+0.0j 6.0+0.0j 7.0+0.0j 8.0+0.0j 9.0+0.0j] [ 0.0+0.0j 1.0+0.0j 2.0+0.0j 3.0+0.0j 4.0+0.0j 5.0+0.0j 6.0+0.0j 7.0+0.0j 8.0+0.0j 9.0+0.0j] [ 0.0+0.0j 1.0+0.0j 2.0+0.0j 3.0+0.0j 4.0+0.0j 5.0+0.0j 6.0+0.0j 7.0+0.0j 8.0+0.0j 9.0+0.0j] [ 0.0+0.0j 1.0+0.0j 2.0+0.0j 3.0+0.0j 4.0+0.0j 5.0+0.0j 6.0+0.0j 7.0+0.0j 8.0+0.0j 9.0+0.0j]
(wrong)
(np-devel)[mb312@joshlegacy ~/tmp]$ python funny_bs.py [ 0.0+0.0j 1.0+0.0j 2.0+0.0j 3.0+0.0j 4.0+0.0j 5.0+0.0j 6.0+0.0j 7.0+0.0j 8.0+0.0j 9.0+0.0j] [ 0.0+0.0j 1.0+0.0j 2.0+0.0j 3.0+0.0j 4.0+0.0j 5.0+0.0j 6.0+0.0j 7.0+0.0j 8.0+0.0j 9.0+0.0j] [ 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j] [ 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j] [ 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j]
(right)
See you,
Long doubles on PPC consist of two doubles, so I expect you need to swap both doubles instead of 16 bytes. Strictly speaking, numpy doesn't support non ieee floats. Chuck
Hi, On Wed, Jun 20, 2012 at 3:05 PM, Charles R Harris <charlesr.harris@gmail.com> wrote:
On Wed, Jun 20, 2012 at 4:00 PM, Matthew Brett <matthew.brett@gmail.com> wrote:
Hi,
On Wed, Jun 20, 2012 at 1:56 PM, Travis Oliphant <travis@continuum.io> wrote:
This looks like a problem with comparisons of floating point numbers rather than a byteswapping problem per-say. Try to use an almost equal comparison instead.
Is that right - that the byteswapped versions might not be strictly equal to identical numbers but not byteswapped?
But I should maybe have been clearer - they also subtract wrongly:
<script> import numpy as np
arr = np.arange(10, dtype=np.complex256) bs_arr = arr.byteswap().newbyteorder('S') print arr print bs_arr print arr - bs_arr print arr - bs_arr print arr - bs_arr </script>
(np-devel)[mb312@joshlegacy ~/tmp]$ python funny_bs.py [ 0.0+0.0j 1.0+0.0j 2.0+0.0j 3.0+0.0j 4.0+0.0j 5.0+0.0j 6.0+0.0j 7.0+0.0j 8.0+0.0j 9.0+0.0j] [ 0.0+0.0j 1.0+0.0j 2.0+0.0j 3.0+0.0j 4.0+0.0j 5.0+0.0j 6.0+0.0j 7.0+0.0j 8.0+0.0j 9.0+0.0j] [ 0.0+0.0j 1.0+0.0j 2.0+0.0j 3.0+0.0j 4.0+0.0j 5.0+0.0j 6.0+0.0j 7.0+0.0j 8.0+0.0j 9.0+0.0j] [ 0.0+0.0j 1.0+0.0j 2.0+0.0j 3.0+0.0j 4.0+0.0j 5.0+0.0j 6.0+0.0j 7.0+0.0j 8.0+0.0j 9.0+0.0j] [ 0.0+0.0j 1.0+0.0j 2.0+0.0j 3.0+0.0j 4.0+0.0j 5.0+0.0j 6.0+0.0j 7.0+0.0j 8.0+0.0j 9.0+0.0j]
(wrong)
(np-devel)[mb312@joshlegacy ~/tmp]$ python funny_bs.py [ 0.0+0.0j 1.0+0.0j 2.0+0.0j 3.0+0.0j 4.0+0.0j 5.0+0.0j 6.0+0.0j 7.0+0.0j 8.0+0.0j 9.0+0.0j] [ 0.0+0.0j 1.0+0.0j 2.0+0.0j 3.0+0.0j 4.0+0.0j 5.0+0.0j 6.0+0.0j 7.0+0.0j 8.0+0.0j 9.0+0.0j] [ 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j] [ 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j] [ 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j]
(right)
See you,
Long doubles on PPC consist of two doubles, so I expect you need to swap both doubles instead of 16 bytes. Strictly speaking, numpy doesn't support non ieee floats.
Well - the byteswapping appears to be correct in that the array is displayed with the correct values, but then, when doing a subtraction on the array, most of the time it is incorrect, but whether it is correct or incorrect, appears to be random even with the same variables and memory. Float128 and other numpy dtypes appear to be correct using the same tests. Best, Matthew
Hi, On Wed, Jun 20, 2012 at 3:11 PM, Matthew Brett <matthew.brett@gmail.com> wrote:
Hi,
On Wed, Jun 20, 2012 at 3:05 PM, Charles R Harris <charlesr.harris@gmail.com> wrote:
On Wed, Jun 20, 2012 at 4:00 PM, Matthew Brett <matthew.brett@gmail.com> wrote:
Hi,
On Wed, Jun 20, 2012 at 1:56 PM, Travis Oliphant <travis@continuum.io> wrote:
This looks like a problem with comparisons of floating point numbers rather than a byteswapping problem per-say. Try to use an almost equal comparison instead.
Is that right - that the byteswapped versions might not be strictly equal to identical numbers but not byteswapped?
But I should maybe have been clearer - they also subtract wrongly:
<script> import numpy as np
arr = np.arange(10, dtype=np.complex256) bs_arr = arr.byteswap().newbyteorder('S') print arr print bs_arr print arr - bs_arr print arr - bs_arr print arr - bs_arr </script>
(np-devel)[mb312@joshlegacy ~/tmp]$ python funny_bs.py [ 0.0+0.0j 1.0+0.0j 2.0+0.0j 3.0+0.0j 4.0+0.0j 5.0+0.0j 6.0+0.0j 7.0+0.0j 8.0+0.0j 9.0+0.0j] [ 0.0+0.0j 1.0+0.0j 2.0+0.0j 3.0+0.0j 4.0+0.0j 5.0+0.0j 6.0+0.0j 7.0+0.0j 8.0+0.0j 9.0+0.0j] [ 0.0+0.0j 1.0+0.0j 2.0+0.0j 3.0+0.0j 4.0+0.0j 5.0+0.0j 6.0+0.0j 7.0+0.0j 8.0+0.0j 9.0+0.0j] [ 0.0+0.0j 1.0+0.0j 2.0+0.0j 3.0+0.0j 4.0+0.0j 5.0+0.0j 6.0+0.0j 7.0+0.0j 8.0+0.0j 9.0+0.0j] [ 0.0+0.0j 1.0+0.0j 2.0+0.0j 3.0+0.0j 4.0+0.0j 5.0+0.0j 6.0+0.0j 7.0+0.0j 8.0+0.0j 9.0+0.0j]
(wrong)
(np-devel)[mb312@joshlegacy ~/tmp]$ python funny_bs.py [ 0.0+0.0j 1.0+0.0j 2.0+0.0j 3.0+0.0j 4.0+0.0j 5.0+0.0j 6.0+0.0j 7.0+0.0j 8.0+0.0j 9.0+0.0j] [ 0.0+0.0j 1.0+0.0j 2.0+0.0j 3.0+0.0j 4.0+0.0j 5.0+0.0j 6.0+0.0j 7.0+0.0j 8.0+0.0j 9.0+0.0j] [ 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j] [ 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j] [ 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j]
(right)
See you,
Long doubles on PPC consist of two doubles, so I expect you need to swap both doubles instead of 16 bytes. Strictly speaking, numpy doesn't support non ieee floats.
Well - the byteswapping appears to be correct in that the array is displayed with the correct values, but then, when doing a subtraction on the array, most of the time it is incorrect, but whether it is correct or incorrect, appears to be random even with the same variables and memory.
Float128 and other numpy dtypes appear to be correct using the same tests.
http://projects.scipy.org/numpy/ticket/2174 Best, Matthew
On Wed, Jun 20, 2012 at 4:11 PM, Matthew Brett <matthew.brett@gmail.com>wrote:
Hi,
On Wed, Jun 20, 2012 at 3:05 PM, Charles R Harris <charlesr.harris@gmail.com> wrote:
On Wed, Jun 20, 2012 at 4:00 PM, Matthew Brett <matthew.brett@gmail.com> wrote:
Hi,
On Wed, Jun 20, 2012 at 1:56 PM, Travis Oliphant <travis@continuum.io> wrote:
This looks like a problem with comparisons of floating point numbers rather than a byteswapping problem per-say. Try to use an almost
equal
comparison instead.
Is that right - that the byteswapped versions might not be strictly equal to identical numbers but not byteswapped?
But I should maybe have been clearer - they also subtract wrongly:
<script> import numpy as np
arr = np.arange(10, dtype=np.complex256) bs_arr = arr.byteswap().newbyteorder('S') print arr print bs_arr print arr - bs_arr print arr - bs_arr print arr - bs_arr </script>
(np-devel)[mb312@joshlegacy ~/tmp]$ python funny_bs.py [ 0.0+0.0j 1.0+0.0j 2.0+0.0j 3.0+0.0j 4.0+0.0j 5.0+0.0j 6.0+0.0j 7.0+0.0j 8.0+0.0j 9.0+0.0j] [ 0.0+0.0j 1.0+0.0j 2.0+0.0j 3.0+0.0j 4.0+0.0j 5.0+0.0j 6.0+0.0j 7.0+0.0j 8.0+0.0j 9.0+0.0j] [ 0.0+0.0j 1.0+0.0j 2.0+0.0j 3.0+0.0j 4.0+0.0j 5.0+0.0j 6.0+0.0j 7.0+0.0j 8.0+0.0j 9.0+0.0j] [ 0.0+0.0j 1.0+0.0j 2.0+0.0j 3.0+0.0j 4.0+0.0j 5.0+0.0j 6.0+0.0j 7.0+0.0j 8.0+0.0j 9.0+0.0j] [ 0.0+0.0j 1.0+0.0j 2.0+0.0j 3.0+0.0j 4.0+0.0j 5.0+0.0j 6.0+0.0j 7.0+0.0j 8.0+0.0j 9.0+0.0j]
(wrong)
(np-devel)[mb312@joshlegacy ~/tmp]$ python funny_bs.py [ 0.0+0.0j 1.0+0.0j 2.0+0.0j 3.0+0.0j 4.0+0.0j 5.0+0.0j 6.0+0.0j 7.0+0.0j 8.0+0.0j 9.0+0.0j] [ 0.0+0.0j 1.0+0.0j 2.0+0.0j 3.0+0.0j 4.0+0.0j 5.0+0.0j 6.0+0.0j 7.0+0.0j 8.0+0.0j 9.0+0.0j] [ 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j] [ 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j] [ 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j]
(right)
See you,
Long doubles on PPC consist of two doubles, so I expect you need to swap both doubles instead of 16 bytes. Strictly speaking, numpy doesn't support non ieee floats.
Well - the byteswapping appears to be correct in that the array is displayed with the correct values, but then, when doing a subtraction on the array, most of the time it is incorrect, but whether it is correct or incorrect, appears to be random even with the same variables and memory.
Float128 and other numpy dtypes appear to be correct using the same tests.
Thinking about it, that makes sense because the swapped version is probably incorrect ;) That is, the PPC was (is?) selectable to run either little endian or big endian, so the real test would be if long doubles were portable between machines set up different ways. The only machines I know of are little endian, but IIRC, there was at least one brand that was big endian. However, I suspect we are just reversing the whole 16 bytes, so even though that is pretty much a meaningless thing to do, it should work... Is this something that only happens on 32 bit machines? Chuck
Hi, On Wed, Jun 20, 2012 at 10:43 PM, Charles R Harris <charlesr.harris@gmail.com> wrote:
On Wed, Jun 20, 2012 at 4:11 PM, Matthew Brett <matthew.brett@gmail.com> wrote:
Hi,
On Wed, Jun 20, 2012 at 3:05 PM, Charles R Harris <charlesr.harris@gmail.com> wrote:
On Wed, Jun 20, 2012 at 4:00 PM, Matthew Brett <matthew.brett@gmail.com> wrote:
Hi,
On Wed, Jun 20, 2012 at 1:56 PM, Travis Oliphant <travis@continuum.io> wrote:
This looks like a problem with comparisons of floating point numbers rather than a byteswapping problem per-say. Try to use an almost equal comparison instead.
Is that right - that the byteswapped versions might not be strictly equal to identical numbers but not byteswapped?
But I should maybe have been clearer - they also subtract wrongly:
<script> import numpy as np
arr = np.arange(10, dtype=np.complex256) bs_arr = arr.byteswap().newbyteorder('S') print arr print bs_arr print arr - bs_arr print arr - bs_arr print arr - bs_arr </script>
(np-devel)[mb312@joshlegacy ~/tmp]$ python funny_bs.py [ 0.0+0.0j 1.0+0.0j 2.0+0.0j 3.0+0.0j 4.0+0.0j 5.0+0.0j 6.0+0.0j 7.0+0.0j 8.0+0.0j 9.0+0.0j] [ 0.0+0.0j 1.0+0.0j 2.0+0.0j 3.0+0.0j 4.0+0.0j 5.0+0.0j 6.0+0.0j 7.0+0.0j 8.0+0.0j 9.0+0.0j] [ 0.0+0.0j 1.0+0.0j 2.0+0.0j 3.0+0.0j 4.0+0.0j 5.0+0.0j 6.0+0.0j 7.0+0.0j 8.0+0.0j 9.0+0.0j] [ 0.0+0.0j 1.0+0.0j 2.0+0.0j 3.0+0.0j 4.0+0.0j 5.0+0.0j 6.0+0.0j 7.0+0.0j 8.0+0.0j 9.0+0.0j] [ 0.0+0.0j 1.0+0.0j 2.0+0.0j 3.0+0.0j 4.0+0.0j 5.0+0.0j 6.0+0.0j 7.0+0.0j 8.0+0.0j 9.0+0.0j]
(wrong)
(np-devel)[mb312@joshlegacy ~/tmp]$ python funny_bs.py [ 0.0+0.0j 1.0+0.0j 2.0+0.0j 3.0+0.0j 4.0+0.0j 5.0+0.0j 6.0+0.0j 7.0+0.0j 8.0+0.0j 9.0+0.0j] [ 0.0+0.0j 1.0+0.0j 2.0+0.0j 3.0+0.0j 4.0+0.0j 5.0+0.0j 6.0+0.0j 7.0+0.0j 8.0+0.0j 9.0+0.0j] [ 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j] [ 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j] [ 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j]
(right)
See you,
Long doubles on PPC consist of two doubles, so I expect you need to swap both doubles instead of 16 bytes. Strictly speaking, numpy doesn't support non ieee floats.
Well - the byteswapping appears to be correct in that the array is displayed with the correct values, but then, when doing a subtraction on the array, most of the time it is incorrect, but whether it is correct or incorrect, appears to be random even with the same variables and memory.
Float128 and other numpy dtypes appear to be correct using the same tests.
Thinking about it, that makes sense because the swapped version is probably incorrect ;) That is, the PPC was (is?) selectable to run either little endian or big endian, so the real test would be if long doubles were portable between machines set up different ways. The only machines I know of are little endian, but IIRC, there was at least one brand that was big endian. However, I suspect we are just reversing the whole 16 bytes, so even though that is pretty much a meaningless thing to do, it should work...
Is this something that only happens on 32 bit machines?
The PPC machines I have record themselves as big endian (2 running OSX and 1 running Debian wheezy). I can only get the Debian wheezy machine to misbehave in this way. The original report of the problem was on a POWER7 machine running Debian - wheezy I think. I guess this is 64 bit - Yarik - do you know? See you, Matthew
On Thu, Jun 21, 2012 at 12:11 AM, Matthew Brett <matthew.brett@gmail.com>wrote:
Hi,
On Wed, Jun 20, 2012 at 10:43 PM, Charles R Harris <charlesr.harris@gmail.com> wrote:
On Wed, Jun 20, 2012 at 4:11 PM, Matthew Brett <matthew.brett@gmail.com> wrote:
Hi,
On Wed, Jun 20, 2012 at 3:05 PM, Charles R Harris <charlesr.harris@gmail.com> wrote:
On Wed, Jun 20, 2012 at 4:00 PM, Matthew Brett <
wrote:
Hi,
On Wed, Jun 20, 2012 at 1:56 PM, Travis Oliphant <
wrote:
This looks like a problem with comparisons of floating point numbers rather than a byteswapping problem per-say. Try to use an almost equal comparison instead.
Is that right - that the byteswapped versions might not be strictly equal to identical numbers but not byteswapped?
But I should maybe have been clearer - they also subtract wrongly:
<script> import numpy as np
arr = np.arange(10, dtype=np.complex256) bs_arr = arr.byteswap().newbyteorder('S') print arr print bs_arr print arr - bs_arr print arr - bs_arr print arr - bs_arr </script>
(np-devel)[mb312@joshlegacy ~/tmp]$ python funny_bs.py [ 0.0+0.0j 1.0+0.0j 2.0+0.0j 3.0+0.0j 4.0+0.0j 5.0+0.0j 6.0+0.0j 7.0+0.0j 8.0+0.0j 9.0+0.0j] [ 0.0+0.0j 1.0+0.0j 2.0+0.0j 3.0+0.0j 4.0+0.0j 5.0+0.0j 6.0+0.0j 7.0+0.0j 8.0+0.0j 9.0+0.0j] [ 0.0+0.0j 1.0+0.0j 2.0+0.0j 3.0+0.0j 4.0+0.0j 5.0+0.0j 6.0+0.0j 7.0+0.0j 8.0+0.0j 9.0+0.0j] [ 0.0+0.0j 1.0+0.0j 2.0+0.0j 3.0+0.0j 4.0+0.0j 5.0+0.0j 6.0+0.0j 7.0+0.0j 8.0+0.0j 9.0+0.0j] [ 0.0+0.0j 1.0+0.0j 2.0+0.0j 3.0+0.0j 4.0+0.0j 5.0+0.0j 6.0+0.0j 7.0+0.0j 8.0+0.0j 9.0+0.0j]
(wrong)
(np-devel)[mb312@joshlegacy ~/tmp]$ python funny_bs.py [ 0.0+0.0j 1.0+0.0j 2.0+0.0j 3.0+0.0j 4.0+0.0j 5.0+0.0j 6.0+0.0j 7.0+0.0j 8.0+0.0j 9.0+0.0j] [ 0.0+0.0j 1.0+0.0j 2.0+0.0j 3.0+0.0j 4.0+0.0j 5.0+0.0j 6.0+0.0j 7.0+0.0j 8.0+0.0j 9.0+0.0j] [ 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j] [ 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j] [ 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j]
(right)
See you,
Long doubles on PPC consist of two doubles, so I expect you need to swap both doubles instead of 16 bytes. Strictly speaking, numpy doesn't support non ieee floats.
Well - the byteswapping appears to be correct in that the array is displayed with the correct values, but then, when doing a subtraction on the array, most of the time it is incorrect, but whether it is correct or incorrect, appears to be random even with the same variables and memory.
Float128 and other numpy dtypes appear to be correct using the same tests.
Thinking about it, that makes sense because the swapped version is
matthew.brett@gmail.com> travis@continuum.io> probably
incorrect ;) That is, the PPC was (is?) selectable to run either little endian or big endian, so the real test would be if long doubles were portable between machines set up different ways. The only machines I know of are little endian, but IIRC, there was at least one brand that was big endian. However, I suspect we are just reversing the whole 16 bytes, so even though that is pretty much a meaningless thing to do, it should work...
Is this something that only happens on 32 bit machines?
The PPC machines I have record themselves as big endian (2 running OSX and 1 running Debian wheezy).
I can only get the Debian wheezy machine to misbehave in this way.
The original report of the problem was on a POWER7 machine running Debian - wheezy I think. I guess this is 64 bit - Yarik - do you know?
Looks like I got little/big endian reversed. Anyway, this is very strange. Same compilers on both machines? I don't understand why this wouldn't show up on other machines with float128, and in particular why is should only happen for complex256 and Debian. What is the extended precision type in OSX on PPC? Chuck
Hi, On Wed, Jun 20, 2012 at 11:57 PM, Charles R Harris <charlesr.harris@gmail.com> wrote:
On Thu, Jun 21, 2012 at 12:11 AM, Matthew Brett <matthew.brett@gmail.com> wrote:
Hi,
On Wed, Jun 20, 2012 at 10:43 PM, Charles R Harris <charlesr.harris@gmail.com> wrote:
On Wed, Jun 20, 2012 at 4:11 PM, Matthew Brett <matthew.brett@gmail.com> wrote:
Hi,
On Wed, Jun 20, 2012 at 3:05 PM, Charles R Harris <charlesr.harris@gmail.com> wrote:
On Wed, Jun 20, 2012 at 4:00 PM, Matthew Brett <matthew.brett@gmail.com> wrote:
Hi,
On Wed, Jun 20, 2012 at 1:56 PM, Travis Oliphant <travis@continuum.io> wrote: > This looks like a problem with comparisons of floating point > numbers > rather than a byteswapping problem per-say. Try to use an almost > equal > comparison instead.
Is that right - that the byteswapped versions might not be strictly equal to identical numbers but not byteswapped?
But I should maybe have been clearer - they also subtract wrongly:
<script> import numpy as np
arr = np.arange(10, dtype=np.complex256) bs_arr = arr.byteswap().newbyteorder('S') print arr print bs_arr print arr - bs_arr print arr - bs_arr print arr - bs_arr </script>
(np-devel)[mb312@joshlegacy ~/tmp]$ python funny_bs.py [ 0.0+0.0j 1.0+0.0j 2.0+0.0j 3.0+0.0j 4.0+0.0j 5.0+0.0j 6.0+0.0j 7.0+0.0j 8.0+0.0j 9.0+0.0j] [ 0.0+0.0j 1.0+0.0j 2.0+0.0j 3.0+0.0j 4.0+0.0j 5.0+0.0j 6.0+0.0j 7.0+0.0j 8.0+0.0j 9.0+0.0j] [ 0.0+0.0j 1.0+0.0j 2.0+0.0j 3.0+0.0j 4.0+0.0j 5.0+0.0j 6.0+0.0j 7.0+0.0j 8.0+0.0j 9.0+0.0j] [ 0.0+0.0j 1.0+0.0j 2.0+0.0j 3.0+0.0j 4.0+0.0j 5.0+0.0j 6.0+0.0j 7.0+0.0j 8.0+0.0j 9.0+0.0j] [ 0.0+0.0j 1.0+0.0j 2.0+0.0j 3.0+0.0j 4.0+0.0j 5.0+0.0j 6.0+0.0j 7.0+0.0j 8.0+0.0j 9.0+0.0j]
(wrong)
(np-devel)[mb312@joshlegacy ~/tmp]$ python funny_bs.py [ 0.0+0.0j 1.0+0.0j 2.0+0.0j 3.0+0.0j 4.0+0.0j 5.0+0.0j 6.0+0.0j 7.0+0.0j 8.0+0.0j 9.0+0.0j] [ 0.0+0.0j 1.0+0.0j 2.0+0.0j 3.0+0.0j 4.0+0.0j 5.0+0.0j 6.0+0.0j 7.0+0.0j 8.0+0.0j 9.0+0.0j] [ 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j] [ 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j] [ 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j 0.0+0.0j]
(right)
See you,
Long doubles on PPC consist of two doubles, so I expect you need to swap both doubles instead of 16 bytes. Strictly speaking, numpy doesn't support non ieee floats.
Well - the byteswapping appears to be correct in that the array is displayed with the correct values, but then, when doing a subtraction on the array, most of the time it is incorrect, but whether it is correct or incorrect, appears to be random even with the same variables and memory.
Float128 and other numpy dtypes appear to be correct using the same tests.
Thinking about it, that makes sense because the swapped version is probably incorrect ;) That is, the PPC was (is?) selectable to run either little endian or big endian, so the real test would be if long doubles were portable between machines set up different ways. The only machines I know of are little endian, but IIRC, there was at least one brand that was big endian. However, I suspect we are just reversing the whole 16 bytes, so even though that is pretty much a meaningless thing to do, it should work...
Is this something that only happens on 32 bit machines?
The PPC machines I have record themselves as big endian (2 running OSX and 1 running Debian wheezy).
I can only get the Debian wheezy machine to misbehave in this way.
The original report of the problem was on a POWER7 machine running Debian - wheezy I think. I guess this is 64 bit - Yarik - do you know?
Looks like I got little/big endian reversed. Anyway, this is very strange. Same compilers on both machines?
gcc-4.6.3-1 on the machine I have access to. The POWER7 machine appears to be running an older kernel, so my guess is it has an older gcc too, but I don't have access I'm afraid.
I don't understand why this wouldn't show up on other machines with float128, and in particular why is should only happen for complex256 and Debian. What is the extended precision type in OSX on PPC?
OSX uses double pairs too - and I've been testing on them for a while - so I was also surprised to see this when we got to Debian... See you, Matthew
On Wed, Jun 20, 2012 at 2:48 PM, Matthew Brett <matthew.brett@gmail.com>wrote:
Hi,
Our Debian friends were hammering our code tests before the upcoming freeze, and found the following very odd thing on 32-bit PPC running Debian squeeze and numpy 1.6.2 or current trunk.
Consider the following script:
<script> import numpy as np
arr = np.arange(10, dtype=np.complex256) bs_arr = arr.byteswap().newbyteorder('S') print arr print bs_arr print arr == bs_arr print arr == bs_arr print arr == bs_arr </script>
Here is the output from some example runs of this script:
np-devel)[mb312@joshlegacy ~/tmp]$ python funny_bs.py [ 0.0+0.0j 1.0+0.0j 2.0+0.0j 3.0+0.0j 4.0+0.0j 5.0+0.0j 6.0+0.0j 7.0+0.0j 8.0+0.0j 9.0+0.0j] [ 0.0+0.0j 1.0+0.0j 2.0+0.0j 3.0+0.0j 4.0+0.0j 5.0+0.0j 6.0+0.0j 7.0+0.0j 8.0+0.0j 9.0+0.0j] [ True False False False False False False False False False] [ True False False False False False False False False False] [ True False False False False False False False False False]
(that's the most common result)
(np-devel)[mb312@joshlegacy ~/tmp]$ python funny_bs.py [ 0.0+0.0j 1.0+0.0j 2.0+0.0j 3.0+0.0j 4.0+0.0j 5.0+0.0j 6.0+0.0j 7.0+0.0j 8.0+0.0j 9.0+0.0j] [ 0.0+0.0j 1.0+0.0j 2.0+0.0j 3.0+0.0j 4.0+0.0j 5.0+0.0j 6.0+0.0j 7.0+0.0j 8.0+0.0j 9.0+0.0j] [ True True True True True True True True True True] [ True True True True True True True True True True] [ True True True True True True True True True True]
(this happens maybe 10% of the time)
(np-devel)[mb312@joshlegacy ~/tmp]$ python funny_bs.py [ 0.0+0.0j 1.0+0.0j 2.0+0.0j 3.0+0.0j 4.0+0.0j 5.0+0.0j 6.0+0.0j 7.0+0.0j 8.0+0.0j 9.0+0.0j] [ 0.0+0.0j 1.0+0.0j 2.0+0.0j 3.0+0.0j 4.0+0.0j 5.0+0.0j 6.0+0.0j 7.0+0.0j 8.0+0.0j 9.0+0.0j] [ True True True True True True True True True True] [ True False False False False False False False False False] [ True True True True True True True True True True]
(less than 10% of the time - order of True, False prints _of the same comparison_ is random. This only seems to happen with complex256.
Is there anything I can do to debug this further? Does anyone want a login to this machine to have a look?
Try swapped float128 and see what that looks like. Chuck
participants (3)
-
Charles R Harris -
Matthew Brett -
Travis Oliphant