Mailman 3 byteswapping a complex scalar - NumPy-Discussion

byteswapping a complex scalar

Michael Droettboom

7 Oct 2009 7 Oct '09

3:28 p.m.

I'm noticing an inconsistency as to how complex numbers are byteswapped as arrays vs. scalars, and wondering if I'm doing something wrong.

...

...
...
x = np.array([-1j], '

# When I swap the whole array, it swaps each of the (real, imag) parts separately

...

...
...
y = x.byteswap() y.tostring().encode('hex') '00000000bf800000' # and this round-trips fine z = np.fromstring(y.tostring(), dtype='>c8') assert z[0] == -1j

# When I swap the scalar, it seems to swap the entire 8 bytes

...

...
...
y = x[0].byteswap() y.tostring().encode('hex') 'bf80000000000000' # ...and this doesn't round-trip z = np.fromstring(y.tostring(), dtype='>c8') assert z[0] == -1j Traceback (most recent call last): File "<stdin>", line 1, in <module> AssertionError

Any thoughts? Mike -- Michael Droettboom Science Software Branch Operations and Engineering Division Space Telescope Science Institute Operated by AURA for NASA

Show replies by thread

Travis Oliphant

8 Oct 8 Oct

12:09 p.m.

On Oct 7, 2009, at 10:28 AM, Michael Droettboom wrote:

...

I'm noticing an inconsistency as to how complex numbers are byteswapped as arrays vs. scalars, and wondering if I'm doing something wrong.

...
...
...
x = np.array([-1j], '
# When I swap the whole array, it swaps each of the (real, imag) parts separately

...
...
...
y = x.byteswap() y.tostring().encode('hex') '00000000bf800000' # and this round-trips fine z = np.fromstring(y.tostring(), dtype='>c8') assert z[0] == -1j

# When I swap the scalar, it seems to swap the entire 8 bytes

...
...
...
y = x[0].byteswap() y.tostring().encode('hex') 'bf80000000000000' # ...and this doesn't round-trip z = np.fromstring(y.tostring(), dtype='>c8') assert z[0] == -1j Traceback (most recent call last): File "<stdin>", line 1, in <module> AssertionError

Any thoughts?

I think this is a bug. You should file a ticket and mark it critical. As I look at the scalar implementation (in gentype_byteswap in scalartypes.c.src), it looks like it's basing it just on the size (Hmm.... I don't know why it's not using the copyswap in the descr field....). This works for many types, but not complex numbers which should have real and imaginary parts handled separately. There are two ways to fix this that I can see: 1) fix the gentype implementation to use the copyswap function pointer from the datatype object 2) over-ride the byteswap in the complex scalar Python type (there is a base-class complex scalar type where it could be placed) to do the right thing. I would probably do #1 if I get a chance to work on it (because strings shouldn't be byteswapped either and they currently are, I see...) x = np.array(['abcd']) Compare: x.byteswap()[0] x[0].byteswap() The work around is to byteswap before extraction: x.byteswap()[0] Thanks for the bug-report. -Travis

Travis Oliphant

12:19 p.m.

On Oct 7, 2009, at 10:28 AM, Michael Droettboom wrote:

...

I'm noticing an inconsistency as to how complex numbers are byteswapped as arrays vs. scalars, and wondering if I'm doing something wrong.

...
...
...
x = np.array([-1j], '
# When I swap the whole array, it swaps each of the (real, imag) parts separately

...
...
...
y = x.byteswap() y.tostring().encode('hex') '00000000bf800000' # and this round-trips fine z = np.fromstring(y.tostring(), dtype='>c8') assert z[0] == -1j

# When I swap the scalar, it seems to swap the entire 8 bytes

...
...
...
y = x[0].byteswap() y.tostring().encode('hex') 'bf80000000000000' # ...and this doesn't round-trip z = np.fromstring(y.tostring(), dtype='>c8') assert z[0] == -1j Traceback (most recent call last): File "<stdin>", line 1, in <module> AssertionError

Any thoughts?

I just checked a fix for this into SVN (tests still need to be added though...) I can't currently build SVN on my Mac for some reason (I don't know if it has to do with recent changes or not, but I don't have time to track it down right now....the error I'm getting is something about Datetime array scalar types not being defined which seems related to the work Dave and Stefan have been discussing). It's a small change, though, and should work. -Travis

Travis Oliphant

12:25 p.m.

On Oct 8, 2009, at 7:19 AM, Travis Oliphant wrote:

...

I just checked a fix for this into SVN (tests still need to be added though...)

I can't currently build SVN on my Mac for some reason (I don't know if it has to do with recent changes or not, but I don't have time to track it down right now....the error I'm getting is something about Datetime array scalar types not being defined which seems related to the work Dave and Stefan have been discussing).

I can build from SVN. The problem is I had to check-out again from SVN (and get rid of the old code-generated files --- sure would be nice if there were the equivalent of "make clean" -Travis

Michael Droettboom

5:08 p.m.

Thanks! I guess I won't file a bug then ;) Mike Travis Oliphant wrote:

...

On Oct 7, 2009, at 10:28 AM, Michael Droettboom wrote:

...
I'm noticing an inconsistency as to how complex numbers are byteswapped as arrays vs. scalars, and wondering if I'm doing something wrong.

...
...
...
x = np.array([-1j], '
'00000000000080bf' # This is a little-endian representation, in the order (real, imag)

# When I swap the whole array, it swaps each of the (real, imag) parts separately

...
...
...
y = x.byteswap() y.tostring().encode('hex')

'00000000bf800000' # and this round-trips fine

...
...
...
z = np.fromstring(y.tostring(), dtype='>c8') assert z[0] == -1j

# When I swap the scalar, it seems to swap the entire 8 bytes

...
...
...
y = x[0].byteswap() y.tostring().encode('hex')

'bf80000000000000' # ...and this doesn't round-trip

...
...
...
z = np.fromstring(y.tostring(), dtype='>c8') assert z[0] == -1j

Traceback (most recent call last): File "<stdin>", line 1, in <module> AssertionError

Any thoughts?

I just checked a fix for this into SVN (tests still need to be added though...)

I can't currently build SVN on my Mac for some reason (I don't know if it has to do with recent changes or not, but I don't have time to track it down right now....the error I'm getting is something about Datetime array scalar types not being defined which seems related to the work Dave and Stefan have been discussing).

It's a small change, though, and should work.

-Travis

_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion

-- Michael Droettboom Science Software Branch Operations and Engineering Division Space Telescope Science Institute Operated by AURA for NASA

Travis Oliphant

10:02 p.m.

On Oct 8, 2009, at 12:08 PM, Michael Droettboom wrote:

...

Thanks! I guess I won't file a bug then ;)

Probably still should, actually: Until the tests get committed, the bug is not really "fixed" -Travis

David Cournapeau

9 Oct 9 Oct

5:13 a.m.

On Thu, Oct 8, 2009 at 9:25 PM, Travis Oliphant wrote:

...

On Oct 8, 2009, at 7:19 AM, Travis Oliphant wrote:

...
I just checked a fix for this into SVN (tests still need to be added though...)

I can't currently build SVN on my Mac for some reason (I don't know if it has to do with recent changes or not, but I don't have time to track it down right now....the error I'm getting is something about Datetime array scalar types not being defined which seems related to the work Dave and Stefan have been discussing).

I can build from SVN. The problem is I had to check-out again from SVN (and get rid of the old code-generated files --- sure would be nice if there were the equivalent of "make clean"

Note that git clean will clean your working tree, and numscons generates less junk in the working tree. It can be a time saver and quite convenient, David

5312

Age (days ago)

5314

Last active (days ago)

List overview

Download

6 comments

3 participants

participants (3)

David Cournapeau
Michael Droettboom
Travis Oliphant

byteswapping a complex scalar

Michael Droettboom

Travis Oliphant

Travis Oliphant

Travis Oliphant

Michael Droettboom

Travis Oliphant

David Cournapeau

tags

participants (3)