I'm noticing an inconsistency as to how complex numbers are byteswapped as arrays vs. scalars, and wondering if I'm doing something wrong.
x = np.array([-1j], '<c8') x.tostring().encode('hex') '00000000000080bf' # This is a little-endian representation, in the order (real, imag)
# When I swap the whole array, it swaps each of the (real, imag) parts separately
y = x.byteswap() y.tostring().encode('hex') '00000000bf800000' # and this round-trips fine z = np.fromstring(y.tostring(), dtype='>c8') assert z[0] == -1j
# When I swap the scalar, it seems to swap the entire 8 bytes
y = x[0].byteswap() y.tostring().encode('hex') 'bf80000000000000' # ...and this doesn't round-trip z = np.fromstring(y.tostring(), dtype='>c8') assert z[0] == -1j Traceback (most recent call last): File "<stdin>", line 1, in <module> AssertionError
Any thoughts? Mike -- Michael Droettboom Science Software Branch Operations and Engineering Division Space Telescope Science Institute Operated by AURA for NASA
On Oct 7, 2009, at 10:28 AM, Michael Droettboom wrote:
I'm noticing an inconsistency as to how complex numbers are byteswapped as arrays vs. scalars, and wondering if I'm doing something wrong.
x = np.array([-1j], '<c8') x.tostring().encode('hex') '00000000000080bf' # This is a little-endian representation, in the order (real, imag)
# When I swap the whole array, it swaps each of the (real, imag) parts separately
y = x.byteswap() y.tostring().encode('hex') '00000000bf800000' # and this round-trips fine z = np.fromstring(y.tostring(), dtype='>c8') assert z[0] == -1j
# When I swap the scalar, it seems to swap the entire 8 bytes
y = x[0].byteswap() y.tostring().encode('hex') 'bf80000000000000' # ...and this doesn't round-trip z = np.fromstring(y.tostring(), dtype='>c8') assert z[0] == -1j Traceback (most recent call last): File "<stdin>", line 1, in <module> AssertionError
Any thoughts?
I think this is a bug. You should file a ticket and mark it critical. As I look at the scalar implementation (in gentype_byteswap in scalartypes.c.src), it looks like it's basing it just on the size (Hmm.... I don't know why it's not using the copyswap in the descr field....). This works for many types, but not complex numbers which should have real and imaginary parts handled separately. There are two ways to fix this that I can see: 1) fix the gentype implementation to use the copyswap function pointer from the datatype object 2) over-ride the byteswap in the complex scalar Python type (there is a base-class complex scalar type where it could be placed) to do the right thing. I would probably do #1 if I get a chance to work on it (because strings shouldn't be byteswapped either and they currently are, I see...) x = np.array(['abcd']) Compare: x.byteswap()[0] x[0].byteswap() The work around is to byteswap before extraction: x.byteswap()[0] Thanks for the bug-report. -Travis
On Oct 7, 2009, at 10:28 AM, Michael Droettboom wrote:
I'm noticing an inconsistency as to how complex numbers are byteswapped as arrays vs. scalars, and wondering if I'm doing something wrong.
x = np.array([-1j], '<c8') x.tostring().encode('hex') '00000000000080bf' # This is a little-endian representation, in the order (real, imag)
# When I swap the whole array, it swaps each of the (real, imag) parts separately
y = x.byteswap() y.tostring().encode('hex') '00000000bf800000' # and this round-trips fine z = np.fromstring(y.tostring(), dtype='>c8') assert z[0] == -1j
# When I swap the scalar, it seems to swap the entire 8 bytes
y = x[0].byteswap() y.tostring().encode('hex') 'bf80000000000000' # ...and this doesn't round-trip z = np.fromstring(y.tostring(), dtype='>c8') assert z[0] == -1j Traceback (most recent call last): File "<stdin>", line 1, in <module> AssertionError
Any thoughts?
I just checked a fix for this into SVN (tests still need to be added though...) I can't currently build SVN on my Mac for some reason (I don't know if it has to do with recent changes or not, but I don't have time to track it down right now....the error I'm getting is something about Datetime array scalar types not being defined which seems related to the work Dave and Stefan have been discussing). It's a small change, though, and should work. -Travis
On Oct 8, 2009, at 7:19 AM, Travis Oliphant wrote:
I just checked a fix for this into SVN (tests still need to be added though...)
I can't currently build SVN on my Mac for some reason (I don't know if it has to do with recent changes or not, but I don't have time to track it down right now....the error I'm getting is something about Datetime array scalar types not being defined which seems related to the work Dave and Stefan have been discussing).
I can build from SVN. The problem is I had to check-out again from SVN (and get rid of the old code-generated files --- sure would be nice if there were the equivalent of "make clean" -Travis
On Thu, Oct 8, 2009 at 9:25 PM, Travis Oliphant <oliphant@enthought.com> wrote:
On Oct 8, 2009, at 7:19 AM, Travis Oliphant wrote:
I just checked a fix for this into SVN (tests still need to be added though...)
I can't currently build SVN on my Mac for some reason (I don't know if it has to do with recent changes or not, but I don't have time to track it down right now....the error I'm getting is something about Datetime array scalar types not being defined which seems related to the work Dave and Stefan have been discussing).
I can build from SVN. The problem is I had to check-out again from SVN (and get rid of the old code-generated files --- sure would be nice if there were the equivalent of "make clean"
Note that git clean will clean your working tree, and numscons generates less junk in the working tree. It can be a time saver and quite convenient, David
Thanks! I guess I won't file a bug then ;) Mike Travis Oliphant wrote:
On Oct 7, 2009, at 10:28 AM, Michael Droettboom wrote:
I'm noticing an inconsistency as to how complex numbers are byteswapped as arrays vs. scalars, and wondering if I'm doing something wrong.
x = np.array([-1j], '<c8') x.tostring().encode('hex')
'00000000000080bf' # This is a little-endian representation, in the order (real, imag)
# When I swap the whole array, it swaps each of the (real, imag) parts separately
y = x.byteswap() y.tostring().encode('hex')
'00000000bf800000' # and this round-trips fine
z = np.fromstring(y.tostring(), dtype='>c8') assert z[0] == -1j
# When I swap the scalar, it seems to swap the entire 8 bytes
y = x[0].byteswap() y.tostring().encode('hex')
'bf80000000000000' # ...and this doesn't round-trip
z = np.fromstring(y.tostring(), dtype='>c8') assert z[0] == -1j
Traceback (most recent call last): File "<stdin>", line 1, in <module> AssertionError
Any thoughts?
I just checked a fix for this into SVN (tests still need to be added though...)
I can't currently build SVN on my Mac for some reason (I don't know if it has to do with recent changes or not, but I don't have time to track it down right now....the error I'm getting is something about Datetime array scalar types not being defined which seems related to the work Dave and Stefan have been discussing).
It's a small change, though, and should work.
-Travis
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
-- Michael Droettboom Science Software Branch Operations and Engineering Division Space Telescope Science Institute Operated by AURA for NASA
participants (3)
-
David Cournapeau -
Michael Droettboom -
Travis Oliphant