[AstroPy] FITS image data dtype

[Erik Bray]

On 07/17/2013 05:01 PM, Erik Bray wrote:
> On 07/17/2013 04:52 PM, Miguel de Val-Borro wrote:
>> On Wed, Jul 17, 2013 at 04:28:07PM -0400, Erik Bray wrote:
>>>>> FITS files store arrays in big-endian order natively.  Actually swapping the
>>>>> bytes to native order can be costly, especially when the data is being accessed
>>>>> by way of an mmap'd file (the default in PyFITS 3.1 and up).  And it's generally
>>>>> unnecessary when working within Numpy--it is only necessary if passing those
>>>>> bytes to other software that is expecting values in native byte order.
>>>>
>>>> The matplotlib function imshow shows the original map differently than
>>>> after swapping the byte order to little-endian format.
>>>>
>>>> imshow(hdulist[1].data)
>>>> https://docs.google.com/file/d/0B3Hv-zV2q5IcSjk2amw3ZmM3Mk0/edit
>>>> imshow(hdulist[1].data.byteswap().newbyteorder())
>>>> https://docs.google.com/file/d/0B3Hv-zV2q5IcTWZ2WXduMWttQUE/edit
>>>>
>>>> If I remember correctly I also had problems with numpy array operations
>>>> on the maps with the '>f8' dtype for this kind of image data.
>>>
>>> Hmmm..  This sounds really familiar but I think it's actually a bug in matplotlib.
>>>
>>> Just be sure why don't you make sure that np.all(hdulist[1].data ==
>>> hdulist[1].data.byteswap().newbyteorder())  If that doesn't return True I'll eat
>>> my shoe.
>>
>> Yes, it returns True. It could be a bug in matplotlib and not something
>> related to pyfits, I will try to find more information to debug the
>> problem.  At least it's good to know that swapping the byte order helps
>> to be able to display the image.
>
> I found the old code sent to me from my coworker who had this same problem. I
> remember puzzling over it for some time and never quite figuring out exactly
> what combination of software versions was required to reproduce it.  But indeed,
> it seems like matplotlib is rending the pixel with the max value in the array as
> 0, or close to it.  When I swap the byte order to little-endian it renders
> correctly.  Might be worth bringing up with the matplotlib people (though I can
> confirm that on my installation of mpl 1.2.0 this bug does not manifest).

I think I've tracked it down.  This is about where the bug *was*: 
https://github.com/matplotlib/matplotlib/blob/master/lib/matplotlib/colors.py#L554

But it's fixed in master, at least (and I think in 1.2).

Erik