[AstroPy] [astropy-dev] ANN: Astropy v0.1

[Erik Bray]

The issue with CFITSIO in pyfits is confusing.  Pyfits only uses pieces 
of it to support tile-compression--everything it needs from CFITSIO to 
do that was cherry-picked into a single file called fitsio.c (the 220K 
file mentioned elsewhere in this thread).

It sort of makes sense that it would do that, as CFITSIO in its entirety 
(including documentation) is about 13 MB uncompressed, and most of that 
is not needed in pyfits.

fitsio.c does document which source files in the original CFITSIO that 
functions were cherry-picked from.  However it also states that these 
functions were "copied and modified" without specifying how or why they 
were modified.  Maybe someone else at STScI has a better handle on the 
history of that.  There have been a couple big fixes to the code in 
fitsio.c that don't seem to have been upstreamed to CFITSIO.

Finally, I don't know exactly what version of CFITSIO the code in pyfits 
was based on, but it hasn't been updated in 3 years.

It might be worthwhile for me to update/refactor this a bit so that 
there's a better record of what CFITSIO version is used, and what has 
been changed.  I think the cherry-picking approach still makes sense for 
pyfits/astropy since CFITSIO is a large-ish library.  I think 
cherry-picking can still work while also supporting linking against a 
system version of CFITSIO.

As for zlib, CFITSIO does include its own copy.  I don't know how the 
Debian package for it is handling that.  Seems like a bit of a mess 
worth trying to unravel.

Erik

On 07/10/2012 11:56 AM, Sergio Pascual wrote:
> Hi, I'm the Fedora packager of pyfits. I wasn't aware that pyfits
> bundles bits of cfitsio. What is funny, because cfitsio also bundles
> zlib...
>
> I would like to remove the cfisio bits and use the system cfitsio
> library , but I'm not sure if it is possible. If the current cfistio
> API is incompatible with pyfits it can be hard to accomplish.
>
> Is it documented somewhere which files were included from cfitsio,
> from which version and what changes, if
> any, were applied to them?
>
>
> Regards, Sergio
>
> 2012/7/10 Erik Bray<embray at stsci.edu>:
>> Hi, thanks for your work on the Debian packaging.  A few replies to your
>> questions inline below:
>>
>> On 07/10/2012 03:38 AM, Olе Streicher wrote:
>>> Dear Astropy developers,
>>>
>>> astropy at liska.ath.cx (Olе Streicher) writes:
>>>> * I would volunteer to do the packaging for Debian (and Ubuntu).
>>>
>>> I now had some time to look a bit deeper into the packaging of astropy;
>>> <http://git.debian.org/?p=debian-science/packages/python-astropy.git>
>>> is the git repository. The package is not yet ready to use, however.
>>>
>>> I have now some questions and comment on the package:
>>>
>>> pyfits
>>> ======
>>>
>>> * Will astropy.io.fits be an integral part of astropy, and pyfits remain
>>>     just as a legacy (just like astropy.wcs resp. pywcs)? Or does astropy
>>>     just include the latest version of pyfits?
>>
>> You can consider astropy.io.fits an eventual replacement for pyfits,
>> though pyfits by itself will still be in development for a while.  I am
>> trying to time things so that astropy.io.fits is as close as possible to
>> an actual release of pyfits.  For that matter, it would be a good idea
>> to include something in the documentation to this effect.
>>
>> To be specific: Astropy 0.1 can be considered a replacement for pyfits
>> 3.1 (which is not yet released, but won't be much different).
>>
>>> * astropy.io.fits also contains some third-party-library code (libz)
>>>     that could be replaced by linking to the library. Many Linux
>>>     distributions unbundle this -- see [1] for Fedora Linux (base of
>>>     Scientific Linux) and [2] for Debian (and Ubuntu). I think that the
>>>     astropy package is a good opportunity to make the code cleaner here
>>>     and move this external code to cextern/ and having some options to use
>>>     external libraries instead (like the wcslib/pywcs issue I already
>>>     mentioned).
>>
>> Agreed.  In addition to libz, io.fits bundles a few *pieces* of cfitsio,
>> but not the entirety of it.  Would you want to unbundle that too?  In
>> any case, moving it to cextern as you suggest.  (I don't remember the
>> conclusion of the wcslib dicussion--was it going to be moved into
>> cextern as well?)
>>
>> The only problem I see is that in some of the zlib and cfitsio files,
>> function names were renamed with a _pyfits_ prefix to prevent binding
>> those symbols to the system versions of those libraries at runtime.  I
>> would need some way to resolve this--perhaps a define in a header file
>> somewhere.  Any ideas?
>>
>>> Packages
>>> ========
>>>
>>> * In the moment, I would just build astropy as one large package. I
>>>     think that it does not make sense to split it up into smaller
>>>     packages. The documentation would be separated, however (it is not
>>>     built yet).
>>
>> Agreed--breaking astropy into multiple packages I think would eliminate
>> the point.  Especially considering future goals to make all the
>> sub-packages more cohesive with each other.
>>
>>> * Since also legacy packages for pyfits, pywcs and vo are build (what
>>>     about asciitable?), I would create for each of them a small legacy
>>>     package that depends on the main package and provides the legacy
>>>     namespace. However, the legacy namespace is only installed if the
>>>     original is not found, which makes a package creation a bit
>>>     instable. It would be nice if the legacy package creation could be
>>>     forced even if the original package is installed. I could do this
>>>     however by patching setup_helpers.py.
>>
>> I'm not sure I understand entirely--if I install the python-pyfits
>> Debian package (or whatever it's called), and then install
>> python-astropy, if it forces installation of the legacy shim for
>> 'pyfits' that would break python-pyfits.
>>
>> Or are you suggesting that python-astropy would specify that it
>> *replaces* python-pyfits and then have it force installation of the
>> legacy shim?  I would caution against that, since it might be replacing
>> python-pyfits with a different, slightly incompatible version of pyfits.
>>    Are there any packages that depend on python-pyfits that this could break?
>>
>> Thanks,
>>
>> Erik
>> _______________________________________________
>> AstroPy mailing list
>> AstroPy at scipy.org
>> http://mail.scipy.org/mailman/listinfo/astropy
>
>
>