[AstroPy] question about fpack and astropy.io.fits
Erik Bray
embray at stsci.edu
Thu Oct 9 16:24:28 EDT 2014
Hi Octavi,
On 10/08/2014 11:06 PM, Octavi Fors wrote:
> Hello everyone,
>
> this might sounds like a newbie question, but do you know if fpack compression
> package is supported by astropy.io.fits version 0.4.2?
It depends on what you mean, exactly. It would help to add some clarification
about what *fpack* is--apologies if this isn't new to you but it's worth
mentioning for anyone else reading.
fpack itself (and its counterpart funpack (which I always enjoy reading "fun
pack")) are just command-line programs that write new FITS files with one or
more of the HDUs compressed according the FITS Tile Compression convention--or
decompress them as the case may be.
In other words, the convention states how compressed FITS files should be read
and written, while fpack is simply a program that knows how to write files with
that convention. There's no sense (I can think of) in which it makes sense for
a general purpose FITS library (like astropy.io.fits) to support fpack itself.
It does however support the same compression scheme and can read files
compressed by fpack. And likewise can write compressed files that can be read
by funpack.
> In
> http://docs.astropy.org/en/stable/io/fits/usage/unfamiliar.html#compressed-image-data
> the module "astropy.io.fits.compression" is mentioned to access compressed HDUs,
> but it doesn't say if fpack is supported and how should this package be
> specified in fits.open method.
The docs you linked to at
http://astropy.readthedocs.org/en/stable/io/fits/usage/unfamiliar.html#compressed-image-data
could probably be expanded on with more examples, but as they indicate files
containing compressed HDUs are handled transparently. The compressed HDUs are
accessed the same way as normal uncompressed IMAGE HDUs. The data is
automatically decompressed when it is read. fpack has no role to play in that.
> My images images are 16 and -32bit. and I would like to make use of tiling
> pattern (FZTILE='(n,m)', see fpack manual
> <http://heasarc.gsfc.nasa.gov/FTP/software/fitsio/c/docs/fpackguide.pdf>).
> For the -32bit images, any advise about noise-sensitive scaling (SCALE_FACTOR?)
> and subtractive dithering to achieve high compression ratios while preserving
> the scientific content of data, would be most than welcome.
The FZTILE keyword, as well as the other "Fpack compression directive keywords"
listed in section 3.3 of the fpack guide are not standard FITS keywords and are
only understood by the fpack program itself.
These can be used, for example, in a pipeline where raw data is written out
uncompressed in a FITS file. The file is later processed through fpack which
will compress any HDUs marked with these keywords according to the indicated
settings. The end result of that is a new HDU compatible with the tile
compression convention, and keywords like FZTILE become ZTILEn, or for example
FZALGOR becomes ZCMPTYPE.
In other words, those FZ keywords indicate how the HDU *should* be compressed
when passed through fpack. The keywords on the actually compressed HDU
(ZCMPTYPE, ZTILEn, etc.) indicate how the HDU *was* compressed.
The FZ* keywords have no effect in astropy.io.fits (though it might not be a bad
idea to create a facility that understands them, for creating compressed HDUs
from existing uncompressed HDUs). Instead, to create a compressed HDU you just
create a CompImageHDU object as demonstrated here:
http://astropy.readthedocs.org/en/stable/io/fits/usage/unfamiliar.html#creating-a-compressed-image-hdu
This works the same as other HDU types in the library. Though it does take
several optional keywords that are described in more detail in the API docs:
http://astropy.readthedocs.org/en/stable/io/fits/api/images.html#astropy.io.fits.CompImageHDU
For example to specify a tiling use
>>> hdu = fits.CompImageHDU(data=data, header=header, tile_size=(n, m))
Subtractive dithering is used by default where applicable. It otherwise
supports all the features of the convention except perhaps the ZMASKCMP option
(not for any particular technical reason--it's definitely something that can and
should be done, though I haven't had any requests for it either...)
As to how to find a balance between compression quality and data quality I think
others with more experience may wish to weigh in on that, though I will point
out that documentation for the tiled image convention has some useful tips on that:
http://fits.gsfc.nasa.gov/registry/tilecompression/tilecompression2.3.pdf
Hope that helps,
Erik
More information about the AstroPy
mailing list