From perry at stsci.edu  Mon Mar 21 10:17:54 2005
From: perry at stsci.edu (Perry Greenfield)
Date: Mon, 21 Mar 2005 10:17:54 -0500
Subject: [AstroPy] functional interface for PyFITS?
Message-ID: <82c4d9e7a53ed0ade87acac4ada31ef6@stsci.edu>

Vicki Laidler has persuaded me that PyFITS should have a more 
functional interface as well as it's more object-oriented current 
interface. The gist of the argument is that many astronomers would like 
a simple call to get data or a header and not mess with opening, then 
digging down in the object structure to get at the data. I find that 
hard to argue with. Here are some proposed functions

getdata(filename, [ext, extname, extver, etc])
getheader(filename, [ext, extname, extver, etc])
getdataheader( [ext, extname, extver, etc]) # returns the data and 
header as a tuple
    # i.e., im, hdr = pyfits.getdataheader('mydata.fits')
    # Shorter name better? getdatahdr? getdh?

The defaults for these functions are the primary header if it contains 
data, if not the first extension. Otherwise the user must select the 
extension desired. The semantics are that these are file "atomic" 
operations. The file is opened, the data and/or the header are read, 
and then the file is closed. Memory mapped mode will not be supported 
(at least initially).

I don't see why it should not be possible to use a functional interface 
for writing and updates as well. Likewise, they open and close the 
specified file.

writeto(filename, data, hdr) # simple fits file is created
append(filename, data, hdr,) # appends if exists, created if it doesn't
update(filename, data, hdr, {ext or extname/extver required})
info(filename) # to print information on a FITS file

These functions should pass relevant validation keywords to the 
underlying functions and methods being used.

Comments? Any missing functions?

Perry 



From perry at stsci.edu  Mon Mar 21 10:24:55 2005
From: perry at stsci.edu (Perry Greenfield)
Date: Mon, 21 Mar 2005 10:24:55 -0500
Subject: [AstroPy] Re: functional interface for PyFITS?
In-Reply-To: <423EE6D2.12A17F7C@stsci.edu>
References: <82c4d9e7a53ed0ade87acac4ada31ef6@stsci.edu>
	<423EE6D2.12A17F7C@stsci.edu>
Message-ID: <4510f3c508cd3567cf2d4a557735022b@stsci.edu>


On Mar 21, 2005, at 10:22 AM, Ivo Busko wrote:

> Perry Greenfield wrote:
> <snip>
>> The defaults for these functions are the primary header if it contains
>> data, if not the first extension. Otherwise the user must select the
>> extension desired. The semantics are that these are file "atomic"
> <snip>
>
> In the second case, when there is data, do the first extension header
> also includes the primary header (in the way of the INHERIT keyword)?
>
> -Ivo
>
Initially, no. But I don't see why support for inheritance can't 
eventually be added.

Perry



From busko at stsci.edu  Mon Mar 21 10:22:58 2005
From: busko at stsci.edu (Ivo Busko)
Date: Mon, 21 Mar 2005 10:22:58 -0500
Subject: [AstroPy] Re: functional interface for PyFITS?
References: <82c4d9e7a53ed0ade87acac4ada31ef6@stsci.edu>
Message-ID: <423EE6D2.12A17F7C@stsci.edu>

Perry Greenfield wrote:
<snip>
> The defaults for these functions are the primary header if it contains
> data, if not the first extension. Otherwise the user must select the
> extension desired. The semantics are that these are file "atomic"
<snip>

In the second case, when there is data, do the first extension header 
also includes the primary header (in the way of the INHERIT keyword)?

-Ivo


From erwin at iac.es  Mon Mar 21 16:23:32 2005
From: erwin at iac.es (Peter Erwin)
Date: Mon, 21 Mar 2005 21:23:32 +0000
Subject: [AstroPy] functional interface for PyFITS?
In-Reply-To: <82c4d9e7a53ed0ade87acac4ada31ef6@stsci.edu>
References: <82c4d9e7a53ed0ade87acac4ada31ef6@stsci.edu>
Message-ID: <p06200709be64e970f4d4@[161.72.92.174]>

>Vicki Laidler has persuaded me that PyFITS should have a more 
>functional interface as well as it's more object-oriented current 
>interface. The gist of the argument is that many astronomers would 
>like a simple call to get data or a header and not mess with 
>opening, then digging down in the object structure to get at the 
>data. I find that hard to argue with. Here are some proposed 
>functions
>
>getdata(filename, [ext, extname, extver, etc])
>getheader(filename, [ext, extname, extver, etc])
>getdataheader( [ext, extname, extver, etc]) # returns the data and 
>header as a tuple
>    # i.e., im, hdr = pyfits.getdataheader('mydata.fits')
>    # Shorter name better? getdatahdr? getdh?

[snip]

I think this is an excellent idea!

As for "getdataheader":
I'd avoid making the name *too* short and cryptic ("getdh" is too short); one
of the nice things about Python is that if you want to avoid typing too much,
you can define your own shorthand names, e.g.:

>>>  gdh = pyfits.getdataheader
and then:
>>>  im, hdr = gdh('mydata.fits')

(Stick it in your PYTHONSTARTUP file if you think you'll be using it a lot.)


>Comments? Any missing functions?

It might be nice to have direct functional access to header values; something
along the lines of

>>>  getheadervalue(filename, keyword, ...)  # returns header value as a string

(There are often times when that's all I want out of a FITS file.)

    -- Peter
-- 
=============================================================
Peter Erwin                   Instituto de Astrofisica de Canarias
erwin at iac.es                  C/ Via Lactea s/n
tel. +34 922 605 238          38200 La Laguna, Tenerife, Spain
fax  +34 922 605 210          http://www.iac.es/galeria/erwin


From laidler at stsci.edu  Mon Mar 21 17:57:40 2005
From: laidler at stsci.edu (Victoria G. Laidler)
Date: Mon, 21 Mar 2005 17:57:40 -0500
Subject: [AstroPy] Re: functional interface for PyFITS?
In-Reply-To: <82c4d9e7a53ed0ade87acac4ada31ef6@stsci.edu>
References: <82c4d9e7a53ed0ade87acac4ada31ef6@stsci.edu>
Message-ID: <423F5164.6030801@stsci.edu>

Perry Greenfield wrote:

> Vicki Laidler has persuaded me that PyFITS should have a more 
> functional interface as well as it's more object-oriented current 
> interface. The gist of the argument is that many astronomers would 
> like a simple call to get data or a header and not mess with opening, 
> then digging down in the object structure to get at the data. I find 
> that hard to argue with. Here are some proposed functions
>
> getdata(filename, [ext, extname, extver, etc])
> getheader(filename, [ext, extname, extver, etc])
> getdataheader( [ext, extname, extver, etc]) # returns the data and 
> header as a tuple
>    # i.e., im, hdr = pyfits.getdataheader('mydata.fits')
>    # Shorter name better? getdatahdr? getdh?

Can I suggest dropping "getdataheader" and replacing it with a keyword 
argument to getdata?
img=getdata('foo.fits')
img,h=getdata('foo.fits',hdr=True)

Will the filename syntax support specifying the extension that way? (I 
forget whether pyfits already does this)
img=getdata('foo.fits[2]')  for the 2nd extension, as a valid alternative to
img=getdata('foo.fits',ext=2)

>
> The defaults for these functions are the primary header if it contains 
> data, if not the first extension. Otherwise the user must select the 
> extension desired. 

Sounds good.

> The semantics are that these are file "atomic" operations. The file is 
> opened, the data and/or the header are read, and then the file is 
> closed. Memory mapped mode will not be supported (at least initially).
>
> I don't see why it should not be possible to use a functional 
> interface for writing and updates as well. Likewise, they open and 
> close the specified file.
>
> writeto(filename, data, hdr) # simple fits file is created
> append(filename, data, hdr,) # appends if exists, created if it doesn't
> update(filename, data, hdr, {ext or extname/extver required})
> info(filename) # to print information on a FITS file 

These sound great. "writeto" should create a minimal header if none is 
provided.

I like Peter's suggested addition too, with a couple of comments:

>>>>/  getheadervalue(filename, keyword, ...)  # returns header value as a string/
>
How about pyfits.getval(filename,keyword, ...) for its name?

And please, how about returning the keyword value in a properly typed 
variable? The data type is encoded in the FITS header after all! Might 
as well make use of it,instead of making the user float it, int it, or 
whathaveyou.

Vicki Laidler




From jh at oobleck.astro.cornell.edu  Tue Mar 22 09:30:42 2005
From: jh at oobleck.astro.cornell.edu (Joe Harrington)
Date: Tue, 22 Mar 2005 09:30:42 -0500
Subject: [AstroPy] Re: functional interface for PyFITS?
In-Reply-To: <423F5164.6030801@stsci.edu> (laidler@stsci.edu)
References: <82c4d9e7a53ed0ade87acac4ada31ef6@stsci.edu>
	<423F5164.6030801@stsci.edu>
Message-ID: <200503221430.j2MEUgSL019922@oobleck.astro.cornell.edu>

In IDL, it is very common to inline a FITS keyword read into a math
expression:

x = 7 * sxpar(file, keyword) + sqrt(grandmothersage)

So, I strongly agree that having the python equivalent return a
properly-typed value is important.  There might also be an option to
return the naked string.

> img=getdata('foo.fits[2]')  for the 2nd extension, as a valid alternative to

Be careful here.  How do you distinguish that from a file whose name
is literally foo.fits[2]?  And if you do 'foo.fits'[2], how do you
distinguish that from the second element of a string array (bar, in
this case): getdata(bar[2])?  The first is contrived but possible in
the grand universe of file-naming schemes.  The second I use *all the
time* to address a list of filenames in sequence.  Stick with
getdata('foo.fits', ext=2).  It's just a few extra characters longer
than the shorthand, and is much clearer.

--jh--


From jpb at pha.jhu.edu  Tue Mar 22 09:41:12 2005
From: jpb at pha.jhu.edu (John Blakeslee)
Date: Tue, 22 Mar 2005 09:41:12 -0500 (EST)
Subject: [AstroPy] Re: functional interface for PyFITS?
In-Reply-To: <200503221430.j2MEUgSL019922@oobleck.astro.cornell.edu>
References: <82c4d9e7a53ed0ade87acac4ada31ef6@stsci.edu>
	<423F5164.6030801@stsci.edu>
	<200503221430.j2MEUgSL019922@oobleck.astro.cornell.edu>
Message-ID: <Pine.LNX.4.58.0503220934030.21462@acs1>


> > img=getdata('foo.fits[2]')  for the 2nd extension, as a valid alternative
> 
> Be careful here.  How do you distinguish that from a file whose name
> is literally foo.fits[2]?  And if you do 'foo.fits'[2], how do you
> distinguish that from the second element of a string array (bar, in
> this case): getdata(bar[2])?  The first is contrived but possible in
> the grand universe of file-naming schemes.  The second I use *all the
> time* to address a list of filenames in sequence.  Stick with
> getdata('foo.fits', ext=2).  It's just a few extra characters longer
> than the shorthand, and is much clearer.

I want to agree whole-heartedly with this comment!
We use pyfits a lot and find it amazingly useful,
and the syntax fairly clear and consistent.  Allowing
something like 'foo.fits[2]' in a function call to 
refer to the 2nd extenstion in the file would open
a Pandora's box of inconsistency and confusion, or 
at least set a very bad precedent.

JB



From ptak at pha.jhu.edu  Tue Mar 22 10:51:05 2005
From: ptak at pha.jhu.edu (Andrew Ptak)
Date: Tue, 22 Mar 2005 10:51:05 -0500 (EST)
Subject: [AstroPy] Re: functional interface for PyFITS?
In-Reply-To: <Pine.LNX.4.58.0503220934030.21462@acs1>
Message-ID: <Pine.LNX.4.44.0503221041590.1304-100000@skysrv.pha.jhu.edu>

I missed the original post so I'm sorry if this was already mentioned, 
but in fairness I just want to add that this format for selecting 
extensions has been in use for a while in high-energy fits software, most 
notably ftools and ciao.  For both of these, you can give either the name 
of the extension or the extension number in brackets.  So that format is 
"natural" for some of us, and might help with converting shell scripts to 
pure python.  I wonder if there is anything in the fits standard 
concerning filenames. But of course this is a minor issue and not allowing 
it would be ok.

Andy Ptak



On Tue, 22 Mar 2005, John Blakeslee wrote:

> 
> > > img=getdata('foo.fits[2]')  for the 2nd extension, as a valid alternative
> > 
> > Be careful here.  How do you distinguish that from a file whose name
> > is literally foo.fits[2]?  And if you do 'foo.fits'[2], how do you
> > distinguish that from the second element of a string array (bar, in
> > this case): getdata(bar[2])?  The first is contrived but possible in
> > the grand universe of file-naming schemes.  The second I use *all the
> > time* to address a list of filenames in sequence.  Stick with
> > getdata('foo.fits', ext=2).  It's just a few extra characters longer
> > than the shorthand, and is much clearer.
> 
> I want to agree whole-heartedly with this comment!
> We use pyfits a lot and find it amazingly useful,
> and the syntax fairly clear and consistent.  Allowing
> something like 'foo.fits[2]' in a function call to 
> refer to the 2nd extenstion in the file would open
> a Pandora's box of inconsistency and confusion, or 
> at least set a very bad precedent.
> 
> JB
> 
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.net
> http://www.scipy.net/mailman/listinfo/astropy
> 



From npirzkal at eso.org  Tue Mar 22 11:59:40 2005
From: npirzkal at eso.org (Nor)
Date: Tue, 22 Mar 2005 11:59:40 -0500
Subject: [AstroPy] Re: functional interface for PyFITS?
In-Reply-To: <200503221430.j2MEUgSL019922@oobleck.astro.cornell.edu>
References: <82c4d9e7a53ed0ade87acac4ada31ef6@stsci.edu>
	<423F5164.6030801@stsci.edu>
	<200503221430.j2MEUgSL019922@oobleck.astro.cornell.edu>
Message-ID: <e1692fdfc451947e316fc0e90e17481d@eso.org>

Default should be to check if foo.fits[2] is a valid filename, read it 
if it is, otherwise read the second extension of foo.fits.

On Mar 22, 2005, at 9:30 AM, Joe Harrington wrote:

> Be careful here.  How do you distinguish that from a file whose name
> is literally foo.fits[2]?  And if you do 'foo.fits'[2], how do you
> distinguish that from the second element of a string array (bar, in
> this case): getdata(bar[2])



From jh at oobleck.astro.cornell.edu  Tue Mar 22 12:06:14 2005
From: jh at oobleck.astro.cornell.edu (Joe Harrington)
Date: Tue, 22 Mar 2005 12:06:14 -0500
Subject: [AstroPy] Re: functional interface for PyFITS?
In-Reply-To: <e1692fdfc451947e316fc0e90e17481d@eso.org> (message from Nor on
	Tue, 22 Mar 2005 11:59:40 -0500)
References: <82c4d9e7a53ed0ade87acac4ada31ef6@stsci.edu>
	<423F5164.6030801@stsci.edu>
	<200503221430.j2MEUgSL019922@oobleck.astro.cornell.edu>
	<e1692fdfc451947e316fc0e90e17481d@eso.org>
Message-ID: <200503221706.j2MH6EVM020794@oobleck.astro.cornell.edu>

> Default should be to check if foo.fits[2] is a valid filename, read it 
> if it is, otherwise read the second extension of foo.fits.

No, that kind of fuzzy-logic behavior is very hard to follow if you're
writing a script and handling error behavior.  What if you have both
foo.fits and foo.fits[2] as filenames, and both have extensions?  How
do you tell which you got, if you weren't sure you had both?  You'd
have to have a relatively complex status variable that knew and
reported all the possible combinations, and you'd have to check it and
process all the possible errors.  Yuk.

--jh--


From mccannwj at pha.jhu.edu  Tue Mar 22 12:13:53 2005
From: mccannwj at pha.jhu.edu (William Jon McCann)
Date: Tue, 22 Mar 2005 12:13:53 -0500
Subject: [AstroPy] Re: functional interface for PyFITS?
In-Reply-To: <e1692fdfc451947e316fc0e90e17481d@eso.org>
References: <82c4d9e7a53ed0ade87acac4ada31ef6@stsci.edu>	<423F5164.6030801@stsci.edu>	<200503221430.j2MEUgSL019922@oobleck.astro.cornell.edu>
	<e1692fdfc451947e316fc0e90e17481d@eso.org>
Message-ID: <42405251.2030004@pha.jhu.edu>

Nor wrote:
> Default should be to check if foo.fits[2] is a valid filename, read it 
> if it is, otherwise read the second extension of foo.fits.

Goodness, no.  What if foo.fits[2] is a valid filename and you wish to 
read the second extension of foo.fits.  Also, a "valid filename" is 
system dependent.

This convention was a terrible idea to start with and it should not be 
propagated.

A better way to go about this might be to define a "fits://" URI scheme.

Jon


From jh at oobleck.astro.cornell.edu  Tue Mar 22 12:18:34 2005
From: jh at oobleck.astro.cornell.edu (Joe Harrington)
Date: Tue, 22 Mar 2005 12:18:34 -0500
Subject: [AstroPy] Re: functional interface for PyFITS?
In-Reply-To: <42405251.2030004@pha.jhu.edu> (message from William Jon McCann
	on Tue, 22 Mar 2005 12:13:53 -0500)
References: <82c4d9e7a53ed0ade87acac4ada31ef6@stsci.edu>	<423F5164.6030801@stsci.edu>	<200503221430.j2MEUgSL019922@oobleck.astro.cornell.edu>
	<e1692fdfc451947e316fc0e90e17481d@eso.org>
	<42405251.2030004@pha.jhu.edu>
Message-ID: <200503221718.j2MHIYsh020901@oobleck.astro.cornell.edu>

URI prefixes are for communication protocols, not for filetypes.
What's wrong with just ext=2?  It's simple.  It's elegant.  It's
easily understood.  It doesn't take up bandwidth on mailing lists... ;-)

--jh--


From laidler at stsci.edu  Tue Mar 22 12:22:57 2005
From: laidler at stsci.edu (Victoria G. Laidler)
Date: Tue, 22 Mar 2005 12:22:57 -0500
Subject: [AstroPy] Re: functional interface for PyFITS?: foo.fits[2]
In-Reply-To: <Pine.LNX.4.44.0503221041590.1304-100000@skysrv.pha.jhu.edu>
References: <Pine.LNX.4.44.0503221041590.1304-100000@skysrv.pha.jhu.edu>
Message-ID: <42405471.8050702@stsci.edu>

I've been googling a bit and have not yet found whether the FITS 
standard either mandates or recommends support for the 'foo.fits[2]' 
syntax to specify the extension.

In addition to ftools and ciao, IRAF also supports this method for 
filename specification.

PyFITS does not support it presently.

Andrew Ptak wrote:

>I missed the original post so I'm sorry if this was already mentioned, 
>but in fairness I just want to add that this format for selecting 
>extensions has been in use for a while in high-energy fits software, most 
>notably ftools and ciao.  For both of these, you can give either the name 
>of the extension or the extension number in brackets.  So that format is 
>"natural" for some of us, and might help with converting shell scripts to 
>pure python.  I wonder if there is anything in the fits standard 
>concerning filenames. But of course this is a minor issue and not allowing 
>it would be ok.
>
>Andy Ptak
>
>
>
>On Tue, 22 Mar 2005, John Blakeslee wrote:
>
>  
>
>>>>img=getdata('foo.fits[2]')  for the 2nd extension, as a valid alternative
>>>>        
>>>>
>>>Be careful here.  How do you distinguish that from a file whose name
>>>is literally foo.fits[2]?  And if you do 'foo.fits'[2], how do you
>>>distinguish that from the second element of a string array (bar, in
>>>this case): getdata(bar[2])?  The first is contrived but possible in
>>>the grand universe of file-naming schemes.  The second I use *all the
>>>time* to address a list of filenames in sequence.  Stick with
>>>getdata('foo.fits', ext=2).  It's just a few extra characters longer
>>>than the shorthand, and is much clearer.
>>>      
>>>
>>I want to agree whole-heartedly with this comment!
>>We use pyfits a lot and find it amazingly useful,
>>and the syntax fairly clear and consistent.  Allowing
>>something like 'foo.fits[2]' in a function call to 
>>refer to the 2nd extenstion in the file would open
>>a Pandora's box of inconsistency and confusion, or 
>>at least set a very bad precedent.
>>
>>JB
>>
>>_______________________________________________
>>AstroPy mailing list
>>AstroPy at scipy.net
>>http://www.scipy.net/mailman/listinfo/astropy
>>
>>    
>>
>
>_______________________________________________
>AstroPy mailing list
>AstroPy at scipy.net
>http://www.scipy.net/mailman/listinfo/astropy
>  
>




From mccannwj at pha.jhu.edu  Tue Mar 22 12:23:08 2005
From: mccannwj at pha.jhu.edu (William Jon McCann)
Date: Tue, 22 Mar 2005 12:23:08 -0500
Subject: [AstroPy] Re: functional interface for PyFITS?
In-Reply-To: <200503221718.j2MHIYsh020901@oobleck.astro.cornell.edu>
References: <82c4d9e7a53ed0ade87acac4ada31ef6@stsci.edu>	<423F5164.6030801@stsci.edu>	<200503221430.j2MEUgSL019922@oobleck.astro.cornell.edu>	<e1692fdfc451947e316fc0e90e17481d@eso.org>	<42405251.2030004@pha.jhu.edu>
	<200503221718.j2MHIYsh020901@oobleck.astro.cornell.edu>
Message-ID: <4240547C.3020005@pha.jhu.edu>

Joe Harrington wrote:
> URI prefixes are for communication protocols, not for filetypes.
> What's wrong with just ext=2?  It's simple.  It's elegant.  It's
> easily understood.  It doesn't take up bandwidth on mailing lists... ;-)

Completely, agree.

I was suggesting if you want to change the definition of a filename for 
FITS files then you should create a fits:// scheme.  However, this would 
be silly.

Jon


From npirzkal at eso.org  Tue Mar 22 12:24:28 2005
From: npirzkal at eso.org (Nor)
Date: Tue, 22 Mar 2005 12:24:28 -0500
Subject: [AstroPy] Re: functional interface for PyFITS?
In-Reply-To: <200503221718.j2MHIYsh020901@oobleck.astro.cornell.edu>
References: <82c4d9e7a53ed0ade87acac4ada31ef6@stsci.edu>	<423F5164.6030801@stsci.edu>	<200503221430.j2MEUgSL019922@oobleck.astro.cornell.edu>
	<e1692fdfc451947e316fc0e90e17481d@eso.org>
	<42405251.2030004@pha.jhu.edu>
	<200503221718.j2MHIYsh020901@oobleck.astro.cornell.edu>
Message-ID: <84458d5a326c6533e6593b6bb42c23cb@eso.org>

because iraf returns filenames with the syntax we have been discussing 
and that getting things tied up with python and pyfits then requires 
some parsing of the filename, etc... which is what in my opinion a high 
level pyfits function call should handle for me.
Call it fuzzy logic if you want, and it is (thank to IRAF ....) , but I 
am sick of parsing filenames in python when writing pyraf scripts and 
would like to see this done for me... :-)


On Mar 22, 2005, at 12:18 PM, Joe Harrington wrote:

> URI prefixes are for communication protocols, not for filetypes.
> What's wrong with just ext=2?  It's simple.  It's elegant.  It's
> easily understood.  It doesn't take up bandwidth on mailing lists... 
> ;-)
>
> --jh--



From focke at slac.stanford.edu  Tue Mar 22 12:38:32 2005
From: focke at slac.stanford.edu (Warren Focke)
Date: Tue, 22 Mar 2005 09:38:32 -0800 (PST)
Subject: [AstroPy] Re: functional interface for PyFITS?: foo.fits[2]
In-Reply-To: <42405471.8050702@stsci.edu>
References: <Pine.LNX.4.44.0503221041590.1304-100000@skysrv.pha.jhu.edu>
	<42405471.8050702@stsci.edu>
Message-ID: <Pine.LNX.4.58.0503220930020.5215@auriga.slac.stanford.edu>


The FITS standard (http://archive.stsci.edu/fits/fits_standard/) is almost
mute on the subject of file names, the closest it comes is C.2.1: "Tapes
may be either ANSI standard labeled or unlabeled. Unlabeled tapes are
preferred." and C.2.2: "Conventions regarding labels for physical media
containing FITS files have not been established for other media."
Considering that they were pretty explicitly thinking of 1/2" reel-to-reel
tapes and stacks of punch cards when they wrote the standard, they really
couldn't mandate much about names.

The foo.fits[2] interpretation is built into CFITSIO, any app that uses
that lib (including Python code using pcfitsio) inherits the behavior
automatically.

It is kinda goofy, though.  Causes problems w/ tcsh, too.

Warren Focke

On Tue, 22 Mar 2005, Victoria G. Laidler wrote:

> I've been googling a bit and have not yet found whether the FITS
> standard either mandates or recommends support for the 'foo.fits[2]'
> syntax to specify the extension.
>
> In addition to ftools and ciao, IRAF also supports this method for
> filename specification.
>
> PyFITS does not support it presently.
>
> Andrew Ptak wrote:
>
> >I missed the original post so I'm sorry if this was already mentioned,
> >but in fairness I just want to add that this format for selecting
> >extensions has been in use for a while in high-energy fits software, most
> >notably ftools and ciao.  For both of these, you can give either the name
> >of the extension or the extension number in brackets.  So that format is
> >"natural" for some of us, and might help with converting shell scripts to
> >pure python.  I wonder if there is anything in the fits standard
> >concerning filenames. But of course this is a minor issue and not allowing
> >it would be ok.
> >
> >Andy Ptak
> >
> >
> >
> >On Tue, 22 Mar 2005, John Blakeslee wrote:
> >
> >
> >
> >>>>img=getdata('foo.fits[2]')  for the 2nd extension, as a valid alternative
> >>>>
> >>>>
> >>>Be careful here.  How do you distinguish that from a file whose name
> >>>is literally foo.fits[2]?  And if you do 'foo.fits'[2], how do you
> >>>distinguish that from the second element of a string array (bar, in
> >>>this case): getdata(bar[2])?  The first is contrived but possible in
> >>>the grand universe of file-naming schemes.  The second I use *all the
> >>>time* to address a list of filenames in sequence.  Stick with
> >>>getdata('foo.fits', ext=2).  It's just a few extra characters longer
> >>>than the shorthand, and is much clearer.
> >>>
> >>>
> >>I want to agree whole-heartedly with this comment!
> >>We use pyfits a lot and find it amazingly useful,
> >>and the syntax fairly clear and consistent.  Allowing
> >>something like 'foo.fits[2]' in a function call to
> >>refer to the 2nd extenstion in the file would open
> >>a Pandora's box of inconsistency and confusion, or
> >>at least set a very bad precedent.
> >>
> >>JB
> >>
> >>_______________________________________________
> >>AstroPy mailing list
> >>AstroPy at scipy.net
> >>http://www.scipy.net/mailman/listinfo/astropy
> >>
> >>
> >>
> >
> >_______________________________________________
> >AstroPy mailing list
> >AstroPy at scipy.net
> >http://www.scipy.net/mailman/listinfo/astropy
> >
> >
>
>
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.net
> http://www.scipy.net/mailman/listinfo/astropy
>


From jh at oobleck.astro.cornell.edu  Tue Mar 22 12:42:52 2005
From: jh at oobleck.astro.cornell.edu (Joe Harrington)
Date: Tue, 22 Mar 2005 12:42:52 -0500
Subject: [AstroPy] Re: functional interface for PyFITS?: foo.fits[2]
In-Reply-To: <Pine.LNX.4.58.0503220930020.5215@auriga.slac.stanford.edu>
	(message from Warren Focke on Tue, 22 Mar 2005 09:38:32 -0800 (PST))
References: <Pine.LNX.4.44.0503221041590.1304-100000@skysrv.pha.jhu.edu>
	<42405471.8050702@stsci.edu>
	<Pine.LNX.4.58.0503220930020.5215@auriga.slac.stanford.edu>
Message-ID: <200503221742.j2MHgqd0021034@oobleck.astro.cornell.edu>

There are lots of examples of really bad coding and computer behavior
in IRAF.  We switched to Python to get *away* from IRAF and its
numerous headaches, not to bring them with us.  CFITSIO did not have
the best-considered design, either.  Please, let's not pollute this
package with all the bad examples we can find from all the other code
in the world.  That will make it totally unusable.  Save that stuff
for compatibility functions.

--jh--



From erwin at iac.es  Tue Mar 22 14:18:13 2005
From: erwin at iac.es (Peter Erwin)
Date: Tue, 22 Mar 2005 19:18:13 +0000
Subject: [AstroPy] Re: functional interface for PyFITS?
Message-ID: <p0620070fbe661e43bce0@[161.72.92.174]>

At 5:57 PM -0500 3/21/05, Victoria G. Laidler wrote:

[snip]

>>  getdata(filename, [ext, extname, extver, etc])
>>  getheader(filename, [ext, extname, extver, etc])
>>  getdataheader( [ext, extname, extver, etc]) # returns the data and 
>>header as a tuple
>>     # i.e., im, hdr = pyfits.getdataheader('mydata.fits')
>>     # Shorter name better? getdatahdr? getdh?
>
>Can I suggest dropping "getdataheader" and replacing it with a 
>keyword argument to getdata?
>img=getdata('foo.fits')
>img,h=getdata('foo.fits',hdr=True)

This seems reasonable, though I would like the option of a separate "getheader"
function which just retrieves the header.


>I like Peter's suggested addition too, with a couple of comments:
>
>>>>>/  getheadervalue(filename, keyword, ...)  # returns header 
>>>>>value as a string/
>>
>How about pyfits.getval(filename,keyword, ...) for its name?
>
>And please, how about returning the keyword value in a properly 
>typed variable? The data type is encoded in the FITS header after 
>all! Might as well make use of it,instead of making the user float 
>it, int it, or whathaveyou.

Oops.  I've been spending too much time trying to parse string output from
IRAF commands, that's why I assumed string output.  Yes, given that 
the data type
*is* in the FITS header, properly typed output makes sense.

(And "getval" would work as a function name, as long as no one is planning on
implementing a function to directly retrieve, say, *pixel* values from the data
itself...)

    -- Peter
-- 
=============================================================
Peter Erwin                   Instituto de Astrofisica de Canarias
erwin at iac.es                  C/ Via Lactea s/n
tel. +34 922 605 238          38200 La Laguna, Tenerife, Spain
fax  +34 922 605 210          http://www.iac.es/galeria/erwin


From npirzkal at eso.org  Tue Mar 22 14:25:52 2005
From: npirzkal at eso.org (Nor)
Date: Tue, 22 Mar 2005 14:25:52 -0500
Subject: [AstroPy] Re: functional interface for PyFITS?: foo.fits[2]
In-Reply-To: <200503221742.j2MHgqd0021034@oobleck.astro.cornell.edu>
References: <Pine.LNX.4.44.0503221041590.1304-100000@skysrv.pha.jhu.edu>
	<42405471.8050702@stsci.edu>
	<Pine.LNX.4.58.0503220930020.5215@auriga.slac.stanford.edu>
	<200503221742.j2MHgqd0021034@oobleck.astro.cornell.edu>
Message-ID: <3f5988204792365a7ade71f52ac77e81@eso.org>

>  Please, let's not pollute this
> package with all the bad examples we can find from all the other code
> in the world.  That will make it totally unusable.  Save that stuff
> for compatibility functions.

That is certainly overstating the case a bit too much. I was happy to 
see Perry was considering a more pragmatic approach to using pyfits to 
read and write FITS files. I thought we were trying to talk about a set 
of high level functions to get things done faster within our current 
work frameworks, and not start defining what proper FITS filenames 
should be. Frankly, as a user, I could not care less. I am just looking 
for a set of functions to allow me to use pyfits using single line 
commands instead of multiple lines of codes, and that could easily be 
useable within pyraf and pyraf scripts. This is actually why I often 
continue to use pcfitsio. I suspect that this is how most users would 
perceive this issue and why many of my collaborators see pyfits as too 
cumbersome to use.
Let me ask a simple question: how many files do you currently have on 
your hard disk with [1] or [2] at the end of their file name? Maybe we 
should start allowing unicode in filenames too? :-) (yes, I am joking).

Anyway, these were my $0.02, feel free to do what you want with them 
... It is clear that we will not agree, isn't it... :-)

On Mar 22, 2005, at 12:42 PM, Joe Harrington wrote:

> There are lots of examples of really bad coding and computer behavior
> in IRAF.  We switched to Python to get *away* from IRAF and its
> numerous headaches, not to bring them with us.  CFITSIO did not have
> the best-considered design, either.  Please, let's not pollute this
> package with all the bad examples we can find from all the other code
> in the world.  That will make it totally unusable.  Save that stuff
> for compatibility functions.
>
> --jh--
>
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.net
> http://www.scipy.net/mailman/listinfo/astropy



From jpb at pha.jhu.edu  Tue Mar 22 13:40:21 2005
From: jpb at pha.jhu.edu (John Blakeslee)
Date: Tue, 22 Mar 2005 13:40:21 -0500 (EST)
Subject: [AstroPy] Re: functional interface for PyFITS?: foo.fits[2]
In-Reply-To: <200503221742.j2MHgqd0021034@oobleck.astro.cornell.edu>
References: <Pine.LNX.4.44.0503221041590.1304-100000@skysrv.pha.jhu.edu>
	<42405471.8050702@stsci.edu>
	<Pine.LNX.4.58.0503220930020.5215@auriga.slac.stanford.edu>
	<200503221742.j2MHgqd0021034@oobleck.astro.cornell.edu>
Message-ID: <Pine.LNX.4.58.0503221339410.29001@acs1>


> There are lots of examples of really bad coding and computer behavior
> in IRAF.  We switched to Python to get *away* from IRAF and its
> numerous headaches, not to bring them with us.  CFITSIO did not have
> the best-considered design, either.  Please, let's not pollute this
> package with all the bad examples we can find from all the other code
> in the world.  That will make it totally unusable.  Save that stuff
> for compatibility functions.

Amen, brother.



From perry at stsci.edu  Tue Mar 22 20:38:13 2005
From: perry at stsci.edu (Perry Greenfield)
Date: Tue, 22 Mar 2005 20:38:13 -0500
Subject: [AstroPy] Re: functional interface for PyFITS?
In-Reply-To: <Pine.LNX.4.58.0503221339410.29001@acs1>
Message-ID: <NEBBIJKBMLDBLNCEEFOCKEIHFJAA.perry@stsci.edu>

First, I'm heartened to see all the discussion this
question engendered.

1) It's very clear that a functional interface is very
much desired and needed.

So we will go ahead and implement a initial version of
something close to what was proposed quickly. Yes, we'll 
add a getheadervalue function as well, and try the approach
of using a keyword flag to add a header return object to
the getdata function (while keeping the getheader function
as well). We welcome any further suggestions about what
would be useful functions.

2) There is some disagreement about whether to support
appending extension numbers to the filename string.

I'm inclined to be skeptical for the importance of supporting
this. After all, is:

getdata("file.fits",3)

really much more to type than

getdata("file.fits[3]")?

(ahem)
 
Besides, the former lends itself far more easily to iteration
(inefficient as it may be to repeatedly open and close the file,
but that's beside the issue) with variables. I'm inclined to
put this feature off until I see a more convincing use case
than familiarity with IRAF or CFITSIO usage). I suppose it 
may argue for providing a convenience function to parse such
strings to pull out the useful elements to be used with PyFITS
but I don't think we need to make this a built in behavior
of these pyfits functions--yet anyway.

Perry



From ptak at pha.jhu.edu  Tue Mar 22 21:29:23 2005
From: ptak at pha.jhu.edu (Andrew Ptak)
Date: Tue, 22 Mar 2005 21:29:23 -0500 (EST)
Subject: [AstroPy] Re: functional interface for PyFITS?
In-Reply-To: <NEBBIJKBMLDBLNCEEFOCKEIHFJAA.perry@stsci.edu>
Message-ID: <Pine.LNX.4.44.0503222043440.13866-100000@skysrv.pha.jhu.edu>

Right, I just was mentioning that there was existing usage of the bracket 
approach, even though to some it seemed ridiculous, and some very minute 
population might expect it if it were allowed by the fits specification.  
Of course anyone using pyfits would be able to use whatever is adopted 
without any problem.  Maybe something actually worth discussing (and I 
remember some traffic along these lines a little while back, plus I only 
skimmed the initial posts on this) is to what extent to emulate idlastro. 
Many astronomers coming over to python will be doing so from IDL and 
that library.  It would be cool to be able to run the following:
data =  readfits(filename, h, ext=1)
n = sxpar(h, "naxis2")
x = tbget(h, data, "x")
y = tbget(h, data, "y")

>From the looks of things this could easily be accomplished via aliasing 
(or at worst simple wrappers). 

Andy


On Tue, 22 Mar 2005, Perry Greenfield wrote:

> First, I'm heartened to see all the discussion this
> question engendered.
> 
> 1) It's very clear that a functional interface is very
> much desired and needed.
> 
> So we will go ahead and implement a initial version of
> something close to what was proposed quickly. Yes, we'll 
> add a getheadervalue function as well, and try the approach
> of using a keyword flag to add a header return object to
> the getdata function (while keeping the getheader function
> as well). We welcome any further suggestions about what
> would be useful functions.
> 
> 2) There is some disagreement about whether to support
> appending extension numbers to the filename string.
> 
> I'm inclined to be skeptical for the importance of supporting
> this. After all, is:
> 
> getdata("file.fits",3)
> 
> really much more to type than
> 
> getdata("file.fits[3]")?
> 
> (ahem)
>  
> Besides, the former lends itself far more easily to iteration
> (inefficient as it may be to repeatedly open and close the file,
> but that's beside the issue) with variables. I'm inclined to
> put this feature off until I see a more convincing use case
> than familiarity with IRAF or CFITSIO usage). I suppose it 
> may argue for providing a convenience function to parse such
> strings to pull out the useful elements to be used with PyFITS
> but I don't think we need to make this a built in behavior
> of these pyfits functions--yet anyway.
> 
> Perry
> 
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.net
> http://www.scipy.net/mailman/listinfo/astropy
> 




From cstawarz at head.cfa.harvard.edu  Thu Mar 24 08:53:40 2005
From: cstawarz at head.cfa.harvard.edu (Chris Stawarz)
Date: Thu, 24 Mar 2005 08:53:40 -0500
Subject: [AstroPy] AstroPy at PyCon
Message-ID: <200503241353.j2ODreAM021958@devel12.cfa.harvard.edu>

Hi,

I'm at PyCon this week and was wondering if any other list members who are 
attending would be interested in meeting up and chatting a bit.  Primarily, 
I'd be interested in comparing the Python-related development going on at 
STScI and elsewhere with the S-Lang stuff we're doing for Chandra, but it'd 
also be nice just to meet people and say hello.

If there's a lot of interest, we could try to set up an open space, or it 
could just be something informal (over lunch or whatever).


Thanks,
Chris




From barrett at stsci.edu  Thu Mar 24 10:59:24 2005
From: barrett at stsci.edu (Paul Barrett)
Date: Thu, 24 Mar 2005 10:59:24 -0500
Subject: [AstroPy] AstroPy at PyCon
Message-ID: <2b0173b2.e2210cbc.81a0400@comet.stsci.edu>



---- Original message ----
>Date: Thu, 24 Mar 2005 08:53:40 -0500
>From: Chris Stawarz <cstawarz at head.cfa.harvard.edu>  
>Subject: [AstroPy] AstroPy at PyCon  
>To: astropy at scipy.net
>
>Hi,
>
>I'm at PyCon this week and was wondering if any other list members who are 
>attending would be interested in meeting up and chatting a bit. 
Primarily, 
>I'd be interested in comparing the Python-related development going on at 
>STScI and elsewhere with the S-Lang stuff we're doing for Chandra, but
it'd 
>also be nice just to meet people and say hello.
>
>If there's a lot of interest, we could try to set up an open space, or it 
>could just be something informal (over lunch or whatever).
>

Most of the STScI SSB group is at the table near the doors in the Grand
Ballroom.  Stop by and chat if you like.

 -- Paul
Paul Barrett, PhD      Space Telescope Science Institute
Phone: 410-338-4475    ESS/Science Software Group
FAX:   410-338-4767    Baltimore, MD 21218



From perry at stsci.edu  Mon Mar 28 11:24:00 2005
From: perry at stsci.edu (Perry Greenfield)
Date: Mon, 28 Mar 2005 11:24:00 -0500
Subject: [AstroPy] Re: functional interface for PyFITS?
In-Reply-To: <1c1629e27526f9b7d85d4202e5ca6b82@eso.org>
References: <NEBBIJKBMLDBLNCEEFOCKEIHFJAA.perry@stsci.edu>
	<1c1629e27526f9b7d85d4202e5ca6b82@eso.org>
Message-ID: <6c993bd0e1dd951fb3f63537a524c595@stsci.edu>


On Mar 23, 2005, at 8:57 AM, Nor wrote:

> Hi Perry,
>
> 	I do agree that it *looks* as simple. The problem though is that many 
> IRAF tasks use the [2] and when you write Pyraf scripts you end up 
> going back and forth from the two notation which has the very negative 
> consequence of requiring more coding, to parse the filename, removal 
> of [] etc..., and ends up costing a lot of time and introduce many 
> bugs.
> 	I would encourage you to at least allow for both to be implemented, 
> maybe using different functions. Otherwise, as has been done in the 
> past, people will just write their own wrappers to your high level 
> functions to pyfits. What was the point of starting this discussion in 
> the first place then? pyfits is fine the way it is... :-)
>
> 		n

How about allowing a layered module on top of pyfits that permits this 
convention? This way the purists can use pyfits and those that wish to 
allow this usage can use the layered module.

Any suggestions for a name of such a module?

Perry



From laidler at stsci.edu  Wed Mar 30 12:31:34 2005
From: laidler at stsci.edu (laidler at stsci.edu)
Date: Wed, 30 Mar 2005 12:31:34 -0500
Subject: [AstroPy] Draft specification for PyFITS functional
 interface
Message-ID: <268a48b8.e5407981.81d4c00@comet.stsci.edu>

After some iterations among the STScI group, here is a draft
specification for these convenience functions. 



Proposed specifications for functional user interface for PyFITS
29 March 05

Functions to include:
--------------------
Reading: getdata, getheader, getval
  getdata returns a numarray (image extension) or recarray(table
  extension) object by default; if the keyword argument
'header' is
  specified with a value other than None, it returns a
(data,header)
  tuple. .

  getheader returns a PyFITS header object.

  getval returns a variable of the correct type containing the
data.

Writing: writeto, update, append
  Return nothing.

Informational: info
  Returns a (multiline) string.

Exception conditions:
--------------------

File does not exist:
~~~~~~~~~~~~~~~~~~~
Read, info, and update functions will raise an exception of the 
specified file
does not exist.

Ambiguity in specifying the extension:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
All ambiguous specifications will raise an exception, eg
  getdata('myfile.fits',ext=(sci,1),extname='err',extver=2)

Attempting to read data that is not found:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

All the read functions will raise an exception if the
requested data
is not found in the file, that is:
  - if there is no such file
  - if there is no such extension
  - if the specified extension contains no data (for getdata)
  - if the header of the specified extension does not contain the
  specified keyword (for getval)

There is one special case which will be handled without raising an
exception:
   - If getdata is called without specifying an extension, it will
   return data from the primary extension.
     - If the primary has no data, it will return data from
the first
     extension.
       - If the first extension has no data, either, it will
raise an
       exception.

Overwriting an existing file:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
writeto will raise an exception if the specified filename already
exists.


Specifying the extension:
------------------------

Pure PyFITS will not support specifying the extension as part
of the
filename string. This functionality will, however, be
supported in a
layered module.

If no extension is specified, the primary HDU will be used.

Several keyword arguments will be provided.
All keyword arguments will be lower case.

Using the ext keyword:
~~~~~~~~~~~~~~~~~~~~~~
getdata('myfile.fits', ext=1)         - extension number
getdata('myfile.fits', ext='sci')     - extension name
getdata('myfile.fits', ext=('sci',2)) - extension name and version

Using the extname keyword:
~~~~~~~~~~~~~~~~~~~~~~~~~~
getdata('myfiles.fits',extname='sci') - selects first encountered
                                        'sci' extension

Using the extver keyword:
~~~~~~~~~~~~~~~~~~~~~~~~~
getdata('myfile.fits',extname='sci',extver=2)



Keyword syntax will *not* be enforced....:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Although keyword syntax will be recommended, it will not be
enforced.
The order of the positional arguments will be filename, ext,
[extver].

Thus the following examples will also work:
  getdata('myfile.fits',3)
  getdata('myfile.fits','sci')
  getdata('myfile.fits','sci',2)
  getdata('myfile.fits',('sci',2))

....except for getval and update:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Since there is no obvious order of parameters for getval and
update,
keyword syntax *will* be enforced for these functions.


Keyword inheritance:
-------------------
Pure PyFITS will not handle inheritance of keywords. This
functionality may, however, be supported by a separate function,
merge_headers, provided in a layered module.




From jh at oobleck.astro.cornell.edu  Wed Mar 30 14:07:04 2005
From: jh at oobleck.astro.cornell.edu (Joe Harrington)
Date: Wed, 30 Mar 2005 14:07:04 -0500
Subject: [AstroPy] Draft specification for PyFITS functional
 interface
In-Reply-To: <268a48b8.e5407981.81d4c00@comet.stsci.edu> (laidler@stsci.edu)
References: <268a48b8.e5407981.81d4c00@comet.stsci.edu>
Message-ID: <200503301907.j2UJ74DX015570@oobleck.astro.cornell.edu>

>Overwriting an existing file:
>~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>writeto will raise an exception if the specified filename already
>exists.

There should be a flag to force overwriting.

There should be an exception for a write failure, including a failure
to delete the old file if overwriting is forced.

--jh--


From hodge at stsci.edu  Wed Mar 30 14:22:07 2005
From: hodge at stsci.edu (Phil Hodge)
Date: Wed, 30 Mar 2005 14:22:07 -0500
Subject: [AstroPy] Draft specification for PyFITS functional interface
In-Reply-To: <268a48b8.e5407981.81d4c00@comet.stsci.edu>
References: <268a48b8.e5407981.81d4c00@comet.stsci.edu>
Message-ID: <424AFC5F.4080308@stsci.edu>

>  Reading: getdata, getheader, getval
>  getdata returns a numarray (image extension) or recarray(table
>  extension) object by default; if the keyword argument 'header' is
>  specified with a value other than None, it returns a (data,header)
>  tuple. .
                                                                               
 
If the syntax header=x is allowed, what would that mean?
I would suggest header=False (or 0) for the default, and
header=True (or non-zero) to indicate the header should be
included.
                                                                               
 
What does getval return if the keyword is HISTORY, COMMENT or ""
and there are multiple such keywords in the header?  (I presume
it returns the first one, but this should be specified.)
                                                                               
 
I think the specifications should list all the functions
and their calling sequences together in one section.  I didn't
see any calling sequences except in examples.
                                                                               
 
>  Overwriting an existing file:
>  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>  writeto will raise an exception if the specified filename already
>  exists.

Could update() be used to append a new extension to an existing
file?  Does writeto() have an argument to specify whether the
header/data should be written to the primary or first extension
(when the data block is an array)?
                                                                                

>  Pure PyFITS will not handle inheritance of keywords.
                                                                                

Good!
                                                                                

Phil



From jh at oobleck.astro.cornell.edu  Wed Mar 30 14:28:53 2005
From: jh at oobleck.astro.cornell.edu (Joe Harrington)
Date: Wed, 30 Mar 2005 14:28:53 -0500
Subject: [AstroPy] Draft specification for PyFITS functional interface
In-Reply-To: <424AFC5F.4080308@stsci.edu> (message from Phil Hodge on Wed, 30
	Mar 2005 14:22:07 -0500)
References: <268a48b8.e5407981.81d4c00@comet.stsci.edu>
	<424AFC5F.4080308@stsci.edu>
Message-ID: <200503301928.j2UJSr9L015678@oobleck.astro.cornell.edu>

I hope getval will handle ESO's HIERARCH keywords.  I'm at this moment
updating sxpar() in IDL to do that.

--jh--


From barrett at stsci.edu  Wed Mar 30 14:42:00 2005
From: barrett at stsci.edu (Paul Barrett)
Date: Wed, 30 Mar 2005 14:42:00 -0500
Subject: [AstroPy] Draft specification for PyFITS functional interface
In-Reply-To: <268a48b8.e5407981.81d4c00@comet.stsci.edu>
References: <268a48b8.e5407981.81d4c00@comet.stsci.edu>
Message-ID: <424B0108.2000802@stsci.edu>

laidler at stsci.edu wrote:
> After some iterations among the STScI group, here is a draft
> specification for these convenience functions. 
> 
> 
> 
> Proposed specifications for functional user interface for PyFITS
> 29 March 05
> 
> Functions to include:
> --------------------
> Reading: getdata, getheader, getval
>   getdata returns a numarray (image extension) or recarray(table
>   extension) object by default; if the keyword argument
> 'header' is
>   specified with a value other than None, it returns a
> (data,header)
>   tuple. .
> 
>   getheader returns a PyFITS header object.
> 
>   getval returns a variable of the correct type containing the
> data.
> 
> Writing: writeto, update, append
>   Return nothing.
> 
> Informational: info
>   Returns a (multiline) string.

I'm wondering if these convenience functions aren't better off in a 
separate python module that imports pyfits.  This approach would keep 
the pyfits interface simple and objected-oriented, and would signal that 
the procedural interface is being used.  Command line versions of these 
convenience functions could also exist, so that users could easily 
extract information from a FITS file.

  -- Paul

-- 
Paul Barrett, PhD      Space Telescope Science Institute
Phone: 410-338-4475    ESS/Science Software Branch
FAX:   410-338-4767    Baltimore, MD 21218


From erwin at iac.es  Wed Mar 30 14:56:15 2005
From: erwin at iac.es (Peter Erwin)
Date: Wed, 30 Mar 2005 20:56:15 +0100
Subject: [AstroPy] Draft specification for PyFITS functional interface
In-Reply-To: <424B0108.2000802@stsci.edu>
References: <268a48b8.e5407981.81d4c00@comet.stsci.edu>
	<424B0108.2000802@stsci.edu>
Message-ID: <p06200732be70b35dc946@[161.72.92.174]>

At 2:42 PM -0500 3/30/05, Paul Barrett wrote:
>I'm wondering if these convenience functions aren't better off in a 
>separate python module that imports pyfits.  This approach would 
>keep the pyfits interface simple and objected-oriented, and would 
>signal that the procedural interface is being used.  Command line 
>versions of these convenience functions could also exist, so that 
>users could easily extract information from a FITS file.

Wouldn't that make things more confusing for users, with two modules to worry
about instead of one?  I like the idea that if I want to do things 
with FITS files,
all I need to do is import pyfits; then I can worry about whether I want to use
a simple functional approach, or something object-oriented, as the need
arises.

    -- Peter
-- 
=============================================================
Peter Erwin                   Instituto de Astrofisica de Canarias
erwin at iac.es                  C/ Via Lactea s/n
tel. +34 922 605 238          38200 La Laguna, Tenerife, Spain
fax  +34 922 605 210          http://www.iac.es/galeria/erwin


From barrett at stsci.edu  Wed Mar 30 15:05:38 2005
From: barrett at stsci.edu (Paul Barrett)
Date: Wed, 30 Mar 2005 15:05:38 -0500
Subject: [AstroPy] Draft specification for PyFITS functional interface
In-Reply-To: <p06200732be70b35dc946@[161.72.92.174]>
References: <268a48b8.e5407981.81d4c00@comet.stsci.edu>	<424B0108.2000802@stsci.edu>
	<p06200732be70b35dc946@[161.72.92.174]>
Message-ID: <424B0692.1080202@stsci.edu>

Peter Erwin wrote:
> At 2:42 PM -0500 3/30/05, Paul Barrett wrote:
> 
>> I'm wondering if these convenience functions aren't better off in a 
>> separate python module that imports pyfits.  This approach would keep 
>> the pyfits interface simple and objected-oriented, and would signal 
>> that the procedural interface is being used.  Command line versions of 
>> these convenience functions could also exist, so that users could 
>> easily extract information from a FITS file.
> 
> 
> Wouldn't that make things more confusing for users, with two modules to 
> worry
> about instead of one?  I like the idea that if I want to do things with 
> FITS files,
> all I need to do is import pyfits; then I can worry about whether I want 
> to use
> a simple functional approach, or something object-oriented, as the need
> arises.

Yes, if you consider them as separate distinct modules.  However, maybe 
we should consider creating a pyfits package which contains the basic OO 
interface as it now stands and adds a procedural interface module and a 
set of command-line programs for convenience.  This approach is similar 
to that used by matplotlib, which I think works well.

  -- Paul

-- 
Paul Barrett, PhD      Space Telescope Science Institute
Phone: 410-338-4475    ESS/Science Software Branch
FAX:   410-338-4767    Baltimore, MD 21218


From npirzkal at eso.org  Wed Mar 30 15:07:46 2005
From: npirzkal at eso.org (Nor)
Date: Wed, 30 Mar 2005 15:07:46 -0500
Subject: [AstroPy] Draft specification for PyFITS functional interface
In-Reply-To: <424B0108.2000802@stsci.edu>
References: <268a48b8.e5407981.81d4c00@comet.stsci.edu>
	<424B0108.2000802@stsci.edu>
Message-ID: <19c59b26bd6d84500d5fba53a03157e4@eso.org>

Keeping the two separated would be my preference as well. It would 
leave pyfits unpolluted by the "quick and dirty" high level interface 
(qpyfits? :-) )

On Mar 30, 2005, at 2:42 PM, Paul Barrett wrote:
>
> I'm wondering if these convenience functions aren't better off in a 
> separate python module that imports pyfits.  This approach would keep 
> the pyfits interface simple and objected-oriented, and would signal 
> that the procedural interface is being used.  Command line versions of 
> these convenience functions could also exist, so that users could 
> easily extract information from a FITS file.
>
>  -- Paul
>
> -- 
> Paul Barrett, PhD      Space Telescope Science Institute
> Phone: 410-338-4475    ESS/Science Software Branch
> FAX:   410-338-4767    Baltimore, MD 21218
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.net
> http://www.scipy.net/mailman/listinfo/astropy



From laidler at stsci.edu  Wed Mar 30 15:08:43 2005
From: laidler at stsci.edu (laidler at stsci.edu)
Date: Wed, 30 Mar 2005 15:08:43 -0500
Subject: [AstroPy] Draft specification for PyFITS
 functional interface
Message-ID: <5883dfe0.e54edc76.81dc100@comet.stsci.edu>



---- Original message ----
>Date: Wed, 30 Mar 2005 20:56:15 +0100
>From: Peter Erwin <erwin at iac.es>  
>Subject: Re: [AstroPy] Draft specification for PyFITS
functional interface  
>To: astropy at scipy.net
>
>At 2:42 PM -0500 3/30/05, Paul Barrett wrote:
>>I'm wondering if these convenience functions aren't better
off in a 
>>separate python module that imports pyfits.  This approach
would 
>>keep the pyfits interface simple and objected-oriented, and
would 
>>signal that the procedural interface is being used.  Command
line 
>>versions of these convenience functions could also exist, so
that 
>>users could easily extract information from a FITS file.
>
>Wouldn't that make things more confusing for users, with two
modules to worry
>about instead of one?  I like the idea that if I want to do
things 
>with FITS files,
>all I need to do is import pyfits; then I can worry about
whether I want to use
>a simple functional approach, or something object-oriented,
as the need
>arises.

I agree with Peter - to my mind, one of the strengths of
Python is that it supports both functional and object-oriented
programming, so that it's easy and painless to switch back and
forth or mix them as the need arises. In an interactive
session, I might be very likely to use the functional routines
as long as things are going as I expect, and then want to
switch to the OO ones for cases where things are going wrong
and I need more direct control of the file's innards.


From hsu at stsci.edu  Wed Mar 30 15:14:41 2005
From: hsu at stsci.edu (Jin-chung Hsu)
Date: Wed, 30 Mar 2005 15:14:41 -0500
Subject: [AstroPy] Re: HIERARCH
In-Reply-To: <20050330195417.664463EB5A@www.scipy.com>
References: <20050330195417.664463EB5A@www.scipy.com>
Message-ID: <424B08B1.2070601@stsci.edu>


>Date: Wed, 30 Mar 2005 14:28:53 -0500
>From: Joe Harrington <jh at oobleck.astro.cornell.edu>
>Subject: Re: [AstroPy] Draft specification for PyFITS functional
>	interface
>To: hodge at stsci.edu
>Cc: astropy at scipy.net
>Message-ID: <200503301928.j2UJSr9L015678 at oobleck.astro.cornell.edu>
>
>I hope getval will handle ESO's HIERARCH keywords.  I'm at this moment
>updating sxpar() in IDL to do that.
>
>--jh--
>
>  
>
It will not be initially as the underlying PyFITS code does not support 
HIERARCH yet.

JC Hsu



From jh at oobleck.astro.cornell.edu  Wed Mar 30 15:36:45 2005
From: jh at oobleck.astro.cornell.edu (Joe Harrington)
Date: Wed, 30 Mar 2005 15:36:45 -0500
Subject: [AstroPy] Re: HIERARCH
In-Reply-To: <424B08B1.2070601@stsci.edu> (message from Jin-chung Hsu on Wed, 
	30 Mar 2005 15:14:41 -0500)
References: <20050330195417.664463EB5A@www.scipy.com>
	<424B08B1.2070601@stsci.edu>
Message-ID: <200503302036.j2UKaj2B015926@oobleck.astro.cornell.edu>

>It will not be initially as the underlying PyFITS code does not support 
>HIERARCH yet.

It's not hard to support extracting the values of individual HIERARCH
keywords, and it's a very valuable thing to do since there are now a
lot of data that use them.  I just implemented it for IDL's sxpar
using regular expressions in a couple of hours.  I've included a
patch, just for illustrative purposes, so you can see how I cased it
in the regex.  The full routine is here if you care:
http://oobleck.astro.cornell.edu/pub/sxpar.pro

Getting the whole hierarchy in some reasonable structure is more of a
stunt, and less useful.  People care about the individual values.

--jh--

*** sxpar.pro   2005-03-30 15:22:39.436418953 -0500
--- /home/local/share/idlastro-6.0/pro/fits/sxpar.pro   2002-03-12 00:27:25.000000000 -0500
***************
*** 115,121 ****
  ;       W. Landsman Dec 2001, Optional /SILENT keyword to suppress warnings
  ;       W. Landsman/D. Finkbeiner  Mar 2002  Make sure extracted vectors
  ;             of mixed data type are returned with the highest type.
- ;     J. Harrington 30 Mar 2005 Extended to read individual HIERARCH keys.
  ;-
  ;----------------------------------------------------------------------
   if N_params() LT 2 then begin
--- 115,120 ----
***************
*** 151,164 ****
              vector = 1                  ;Flag for vector output
              name_length = strlen(nam)   ;Length of name
              num_length = 8 - name_length        ;Max length of number portion 
!
!             if num_length LE 0 and strmid(nam, 0, 8) NE 'HIERARCH' then  $
                    message, 'Keyword length must be 8 characters or less'

! ;  Otherwise, DO NOT extend NAME with blanks to eight characters.

      endif else begin
!                 ;while strlen(nam) LT 8 do nam = nam + ' ' ;Make 8 chars long
                  vector = 0
      endelse

--- 150,162 ----
              vector = 1                  ;Flag for vector output
              name_length = strlen(nam)   ;Length of name
              num_length = 8 - name_length        ;Max length of number portion 
!             if num_length LE 0 then  $
                    message, 'Keyword length must be 8 characters or less'

! ;  Otherwise, extend NAME with blanks to eight characters.

      endif else begin
!                 while strlen(nam) LT 8 do nam = nam + ' ' ;Make 8 chars long
                  vector = 0
      endelse

***************
*** 167,173 ****
  ;  a number.  Store the positions of the located keywords in NFOUND, and the
  ;  value of the number field in NUMBER.

!         histnam = (nam eq 'HISTORY') or (nam eq 'COMMENT') or (nam eq '')
          if N_elements(start) EQ 0 then start = -1l
          start = long(start[0])
          if (not vector) and (start GE 0) then begin
--- 165,171 ----
  ;  a number.  Store the positions of the located keywords in NFOUND, and the
  ;  value of the number field in NUMBER.

!         histnam = (nam eq 'HISTORY ') or (nam eq 'COMMENT ') or (nam eq '')
          if N_elements(start) EQ 0 then start = -1l
          start = long(start[0])
          if (not vector) and (start GE 0) then begin
***************
*** 176,194 ****
              nheader = N_elements(hdr)
              mn = (start - precheck)  > 0
              mx = (start + postcheck) < nheader-1
!             keyword = stregex(hdr[mn:mx], /extract, /subexpr, $
! '^(........)=|^(HIERARCH[^=]*)|^(COMMENT )|^(HISTORY )|(^        )|(^END     )')
!             keyword = keyword[1:*, *]
!             keyword = keyword[where(keyword ne '')]
          endif else begin
              restart:
              start   = -1l
!             keyword = stregex(hdr, /extract, /subexpr, $
! '^(........)=|^(HIERARCH[^=]*)|^(COMMENT )|^(HISTORY )|(^        )|(^END     )')
!             keyword = keyword[1:*, *]
!             keyword = keyword[where(keyword ne '')]
          endelse
-         keyword = strtrim(keyword)

          if vector then begin
              nfound = where(strpos(keyword,nam) GE 0, matches)
--- 174,185 ----
              nheader = N_elements(hdr)
              mn = (start - precheck)  > 0
              mx = (start + postcheck) < nheader-1
!             keyword = strmid(hdr[mn:mx], 0, 8)
          endif else begin
              restart:
              start   = -1l
!             keyword = strmid( hdr, 0, 8)
          endelse

          if vector then begin
              nfound = where(strpos(keyword,nam) GE 0, matches)
***************
*** 224,230 ****

   if matches GT 0 then begin
    line = hdr[nfound]
!   svalue = strtrim((stregex(line, '[^=]*=(.*)', /extract, /subexpr))[1, *], 2)    if histnam then $
          value = strtrim(strmid(line,8,71),2) else for i = 0,matches-1 do begin        if ( strmid(svalue[i],0,1) EQ "'" ) then begin   ;Is it a string?
--- 215,221 ----

   if matches GT 0 then begin
    line = hdr[nfound]
!   svalue = strtrim( strmid(line,9,71),2)
    if histnam then $
          value = strtrim(strmid(line,8,71),2) else for i = 0,matches-1 do begin        if ( strmid(svalue[i],0,1) EQ "'" ) then begin   ;Is it a string?



From perry at stsci.edu  Wed Mar 30 15:42:47 2005
From: perry at stsci.edu (Perry Greenfield)
Date: Wed, 30 Mar 2005 15:42:47 -0500
Subject: [AstroPy] Draft specification for PyFITS functional interface
In-Reply-To: <19c59b26bd6d84500d5fba53a03157e4@eso.org>
References: <268a48b8.e5407981.81d4c00@comet.stsci.edu>
	<424B0108.2000802@stsci.edu>
	<19c59b26bd6d84500d5fba53a03157e4@eso.org>
Message-ID: <4896e96866f295608cdb0dbbd2da86e7@stsci.edu>


On Mar 30, 2005, at 3:07 PM, Nor wrote:

> Keeping the two separated would be my preference as well. It would 
> leave pyfits unpolluted by the "quick and dirty" high level interface 
> (qpyfits? :-) )
>
> On Mar 30, 2005, at 2:42 PM, Paul Barrett wrote:
>>
>> I'm wondering if these convenience functions aren't better off in a 
>> separate python module that imports pyfits.  This approach would keep 
>> the pyfits interface simple and objected-oriented, and would signal 
>> that the procedural interface is being used.  Command line versions 
>> of these convenience functions could also exist, so that users could 
>> easily extract information from a FITS file.
>>
>>
These are good points, the problem then that has to be faced is that a 
choice has to be made regarding filename syntax since I don't think 
having 3 separate modules is sensible (to include or not include 
handling of cases like input.fits[1]). My read is that if there are 
only two modules, it must be included. What say those that despise this 
usage? Which is worse:

oo-only pyfits and iraf/cfitsio-convention-contaminated xxxfits?
oo+functional pyfits and  iraf/cfitsio-convention-contaminated xxxfits?

As for names, hmmm...

ifits
pifits
qfits
shufits er...

I think I find qpyfits getting too long.

Also, the interactive/functional version I think should either not 
expose the pyfits.open function  or it should rename it (fitsopen?) so 
that from xxxfits import * may be used by those that wish to.

Comments?






From hsu at stsci.edu  Wed Mar 30 16:15:05 2005
From: hsu at stsci.edu (Jin-chung Hsu)
Date: Wed, 30 Mar 2005 16:15:05 -0500
Subject: [AstroPy] Re: HIERARCH
Message-ID: <424B16D9.7020700@stsci.edu>

Joe Harrington wrote:

>> It will not be initially as the underlying PyFITS code does not 
>> support HIERARCH yet.
>>   
>
>
> It's not hard to support extracting the values of individual HIERARCH
> keywords, and it's a very valuable thing to do since there are now a
> lot of data that use them.  I just implemented it for IDL's sxpar
> using regular expressions in a couple of hours.  I've included a
> patch, just for illustrative purposes, so you can see how I cased it
> in the regex.  The full routine is here if you care:
> http://oobleck.astro.cornell.edu/pub/sxpar.pro
>
> Getting the whole hierarchy in some reasonable structure is more of a
> stunt, and less useful.  People care about the individual values.
>
> --jh--
>  
>
Thank you for the quick reply.  I agree that it is not a big deal if 
just reading the value.
But, if I want to support it fully, I also need to be able to write it, 
index it etc. in PyFITS.

The way the keywords are now implemented in PyFITS is pretty involved 
because of efficiency issue (minimal parsing by using a keyword list).  
Also what if people combine it with the complicted case of CONTINUE 
cards?  These are the reasons I hesitate to support it fully in PyFITS.

It is possible to support it in getval(), but that looks funny if PyFITS 
itself does not.

JC Hsu



From jh at oobleck.astro.cornell.edu  Wed Mar 30 16:13:55 2005
From: jh at oobleck.astro.cornell.edu (Joe Harrington)
Date: Wed, 30 Mar 2005 16:13:55 -0500
Subject: [AstroPy] Draft specification for PyFITS functional interface
In-Reply-To: <4896e96866f295608cdb0dbbd2da86e7@stsci.edu> (message from Perry
	Greenfield on Wed, 30 Mar 2005 15:42:47 -0500)
References: <268a48b8.e5407981.81d4c00@comet.stsci.edu>
	<424B0108.2000802@stsci.edu>
	<19c59b26bd6d84500d5fba53a03157e4@eso.org>
	<4896e96866f295608cdb0dbbd2da86e7@stsci.edu>
Message-ID: <200503302113.j2ULDthP016162@oobleck.astro.cornell.edu>

I don't see the point of putting these routines in a separate package.
If someone's annoyed by the psychic stink of having functional
routines in an OO package, please get over it!  What I care about is
that the bizarre IRAF/cfitsio behavior not be applied to strings a
user calls the routines with unless the user turns it on deliberately.
That could be by calling a compatibility function or by a switch to
the procedural function.  For example, I have no problem with:

x=foo('input.fits[1]', irafext=1)
x=fooraf('input.fits[1]')

etc., where foo() is the main function for doing the operation.  But,

x=foo('input.fits[1]')

should give a file-not-found error unless I have a file of that name.
You could be super-helpful and print a suggestion in that case that
the user might want to turn on the irafext flag.

--jh--



From hsu at stsci.edu  Wed Mar 30 16:37:04 2005
From: hsu at stsci.edu (Jin-chung Hsu)
Date: Wed, 30 Mar 2005 16:37:04 -0500
Subject: [AstroPy] Re: HIERARCH
In-Reply-To: <200503302104.j2UL44aH016124@oobleck.astro.cornell.edu>
References: <20050330195417.664463EB5A@www.scipy.com>
	<424B08B1.2070601@stsci.edu>
	<200503302036.j2UKaj2B015926@oobleck.astro.cornell.edu>
	<424B1131.10301@stsci.edu>
	<200503302104.j2UL44aH016124@oobleck.astro.cornell.edu>
Message-ID: <424B1C00.5080904@stsci.edu>

Joe Harrington wrote:

>Anyway, I'll give my usual the-bottom-line-is-the-only-line reply: I
>don't care what you do, I care that I can read HIERARCH keywords.  I
>can't imagine that parsing keywords will ever require super-efficient
>coding on a computer built since, say, 1995.  So, if that's standing
>
We had a case that there were tens of thousands keywords and that forced 
us to come up with a more efficient but somewhat limiting scheme.

>in the way of supporting this basic capability, please sacrifice it.
>I don't understand what the issue can be, and don't want to learn the
>code internals well enough to understand it.  I just want to be able
>to read keywords like I now can in IDL:
>
Well, we do have to do all the work.  I am confident that if we just 
implement the read, someone else will come around and ask for the write.
Another reason why it will be non-trial amount of work is that the 
verification will be quite different.

>airmass =  sxpar(header, 'HIERARCH ESO TEL AIRM START')
>
>to get the airmass of my observation.  Why can't you just treat the
>whole string as a single unit and treat it like any other keyword?
>  
>
Don't mean to get into the detail.  But shouldn't the "keyword" in this 
be "ESO TEL AIRM START" without the "HIERARCH" (or maybe even without 
the "ESO")?

We can take another look and see how much work it will take.  But 
implementation in the very near future is unlikely.

JC



From erwin at iac.es  Wed Mar 30 17:14:42 2005
From: erwin at iac.es (Peter Erwin)
Date: Wed, 30 Mar 2005 23:14:42 +0100
Subject: [AstroPy] Draft specification for PyFITS  functional interface
In-Reply-To: <5883dfe0.e54edc76.81dc100@comet.stsci.edu>
References: <5883dfe0.e54edc76.81dc100@comet.stsci.edu>
Message-ID: <p06200734be70d1b0e4af@[161.72.92.174]>

At 3:08 PM -0500 3/30/05, <laidler at stsci.edu> wrote:
>I agree with Peter - to my mind, one of the strengths of
>Python is that it supports both functional and object-oriented
>programming, so that it's easy and painless to switch back and
>forth or mix them as the need arises. In an interactive
>session, I might be very likely to use the functional routines
>as long as things are going as I expect, and then want to
>switch to the OO ones for cases where things are going wrong
>and I need more direct control of the file's innards.

Well, obviously, I agree with this ;-)

It's actually what I was going to add, but you beat me to it -- one
of the things that makes Python so useful is its flexibility and *lack*
of forced adherence to a particular programming style.  Complaints
that it would be "polluting" the pyfits interface make no sense to me,
I'm afraid.  (My personal perspective is that it would make pyfits
*more useful* and easier to start using; I've been slightly leery of
digging into pyfits because an object-oriented approach seems
slightly excessive for the things I usually want to do with FITS files.)
Many of the Python standard libraries are similarly "unclean", and
more useful because of it.

As for the ".fits[1]" issue -- I'd find it somewhat convenient to be
able to use that syntax, but it's not a big issue for me.  So if you're
looking for votes on:

At 3:42 PM -0500 3/30/05, Perry Greenfield wrote:
>Which is worse:
>oo-only pyfits and iraf/cfitsio-convention-contaminated xxxfits?
>oo+functional pyfits and  iraf/cfitsio-convention-contaminated xxxfits?

then my votes is that the *first* in that list is worse...

    -- Peter

-- 
=============================================================
Peter Erwin                   Instituto de Astrofisica de Canarias
erwin at iac.es                  C/ Via Lactea s/n
tel. +34 922 605 238          38200 La Laguna, Tenerife, Spain
fax  +34 922 605 210          http://www.iac.es/galeria/erwin


From jh at oobleck.astro.cornell.edu  Wed Mar 30 17:15:49 2005
From: jh at oobleck.astro.cornell.edu (Joe Harrington)
Date: Wed, 30 Mar 2005 17:15:49 -0500
Subject: [AstroPy] Re: HIERARCH
In-Reply-To: <424B1C00.5080904@stsci.edu> (message from Jin-chung Hsu on Wed, 
	30 Mar 2005 16:37:04 -0500)
References: <20050330195417.664463EB5A@www.scipy.com>
	<424B08B1.2070601@stsci.edu>
	<200503302036.j2UKaj2B015926@oobleck.astro.cornell.edu>
	<424B1131.10301@stsci.edu>
	<200503302104.j2UL44aH016124@oobleck.astro.cornell.edu>
	<424B1C00.5080904@stsci.edu>
Message-ID: <200503302215.j2UMFnxt016354@oobleck.astro.cornell.edu>

>We had a case that there were tens of thousands keywords and that forced 
>us to come up with a more efficient but somewhat limiting scheme.

I'm a bit worried that you're so quick to abandon maintainable and
extensible code for efficiency, given just one pathological case.
Since HIERARCH is used by all ESO headers, we lose a substantial
fraction of European astronomers if we don't support it.  FITS is such
a broken standard that it's going to be possible to construct FITS
files that are hard or slow to process.  For those cases, the user is
advised try a better format, like HDF.

> I am confident that if we just 
> implement the read, someone else will come around and ask for the
> write.

Perhaps.  So give them a simple string write.  It should be equally
easy to implement.  The hard thing is to actually implement a data
structure read/write from the hierarchy, with all the elements typed
correctly, and then more functions to parse and get info out of that.
But, relatively few people will want to do that, compared to the many
(thousands?) like me who just want to get and set individual values in
the header, and who are happy giving the whole HIERARCH ESO... string
as a keyword.

Mind you I don't object to your implementing the whole thing, but
implementing even just a stop-gap read in the very near future is
important if we want to get Europeans to use Python.  Perhaps ESO
could be contacted to provide code to support their keywords.  If you
publicize the need, someone who needs the capability will step
forward.  Use the open source development model to your advantage.

>But shouldn't the "keyword" in this 
>be "ESO TEL AIRM START" without the "HIERARCH" (or maybe even without 
>the "ESO")?

I don't care, I just wanna read the sucker so I can analyze my
data. ;-) I'm playing a little naiive here, but that's the user
perspective.  I'd *rather* deal with these keywords as single strings
that give me one value back, instead of extracting a large data
structure and addressing it in an OO way.  OO notation is cumbersome
to type interactively.

In sxpar, I implemented it as a single long keyword string.  I had to
special-case the regex to allow only max-8-char keywords and HIERARCH
keywords, plus COMMENT, HISTORY, END, and 8 blanks.  But, it was all
done in one regex.  Here's the code to read all the keywords in the
header, sans trailing spaces or equals, and without loops:

        keyword = stregex(hdr[mn:mx], /extract, /subexpr, $
'^(........)=|^(HIERARCH[^=]*)|^(COMMENT )|^(HISTORY )|(^        )|(^END     )')
        keyword = keyword[1:*, *]
        keyword = keyword[where(keyword ne '')]
        keyword = strtrim(keyword)

Then, you just search for a keyword that matches your string, pull the
matching line from the header, and parse the value.

--jh--


From hsu at stsci.edu  Wed Mar 30 18:18:32 2005
From: hsu at stsci.edu (Jin-chung Hsu)
Date: Wed, 30 Mar 2005 18:18:32 -0500
Subject: [AstroPy] Re: HIERARCH
In-Reply-To: <200503302215.j2UMFnxt016354@oobleck.astro.cornell.edu>
References: <20050330195417.664463EB5A@www.scipy.com>
	<424B08B1.2070601@stsci.edu>
	<200503302036.j2UKaj2B015926@oobleck.astro.cornell.edu>
	<424B1131.10301@stsci.edu>
	<200503302104.j2UL44aH016124@oobleck.astro.cornell.edu>
	<424B1C00.5080904@stsci.edu>
	<200503302215.j2UMFnxt016354@oobleck.astro.cornell.edu>
Message-ID: <424B33C8.7070002@stsci.edu>


>I'm a bit worried that you're so quick to abandon maintainable and
>extensible code for efficiency, given just one pathological case.
>
Please not be so quick to make your judgement based on its face value.  
This is just one of the reasons.

>Since HIERARCH is used by all ESO headers, we lose a substantial
>fraction of European astronomers if we don't support it.  
>
Maybe.  I have no idea how many astronomers use ESO data and how many of 
them do access the HIERARCH keywords.  We did get 2 or 3 such requests 
in the last year or two. 

>>I am confident that if we just 
>>implement the read, someone else will come around and ask for the
>>write.
>>    
>>
>
>Perhaps.  So give them a simple string write.  It should be equally
>easy to implement.  The hard thing is to actually implement a data
>structure read/write from the hierarchy, with all the elements typed
>
That's right.  It is not hard to just write the card image.  In fact, it 
can be done now.  But I was alluding to people who want to do:

hdu.header['very long keyword']='a new value'

>>But shouldn't the "keyword" in this 
>>be "ESO TEL AIRM START" without the "HIERARCH" (or maybe even without 
>>the "ESO")?
>>    
>>
>
>I don't care, I just wanna read the sucker so I can analyze my
>
Well, I have heard the other way too.  So, we do need a "standard".  
Other users may want to give their input?

JC




From jh at oobleck.astro.cornell.edu  Wed Mar 30 18:46:28 2005
From: jh at oobleck.astro.cornell.edu (Joe Harrington)
Date: Wed, 30 Mar 2005 18:46:28 -0500
Subject: [AstroPy] Re: HIERARCH
In-Reply-To: <424B33C8.7070002@stsci.edu> (message from Jin-chung Hsu on Wed, 
	30 Mar 2005 18:18:32 -0500)
References: <20050330195417.664463EB5A@www.scipy.com>
	<424B08B1.2070601@stsci.edu>
	<200503302036.j2UKaj2B015926@oobleck.astro.cornell.edu>
	<424B1131.10301@stsci.edu>
	<200503302104.j2UL44aH016124@oobleck.astro.cornell.edu>
	<424B1C00.5080904@stsci.edu>
	<200503302215.j2UMFnxt016354@oobleck.astro.cornell.edu>
	<424B33C8.7070002@stsci.edu>
Message-ID: <200503302346.j2UNkS5H016649@oobleck.astro.cornell.edu>

>I have no idea how many astronomers use ESO data and how many of 
>them do access the HIERARCH keywords.

I would estimate the number as roughly all the observational
astronomers in Europe and Chile, plus Americans and others like me who
collaborate with them.  Since the Europeans have comparatively few
spacecraft, a larger fraction is ground-based observers than in the
US.

In their headers, almost all the useful info is in the HIERARCH
keywords.  Airmass, position of the telescope, observing conditions,
wavelength, image scale, etc.  They do give time and exposure time in
the regular keywords.  There are over 400 keywords in their headers,
of which about 30 are not HIERARCH keywords.  See
ftp://oobleck.astro.cornell.edu/outgoing/jh/ESOhdr.fits
for an example.

>hdu.header['very long keyword']='a new value'

I just don't understand why this presents any problem to python.  If
the keyword starts with "HIERARCH", turn off length checking and treat
it like any other keyword, wherever length is checked.  What am I
missing?

--jh--


From perry at stsci.edu  Wed Mar 30 20:11:38 2005
From: perry at stsci.edu (Perry Greenfield)
Date: Wed, 30 Mar 2005 20:11:38 -0500
Subject: [AstroPy] Re: HIERARCH
In-Reply-To: <200503302346.j2UNkS5H016649@oobleck.astro.cornell.edu>
Message-ID: <NEBBIJKBMLDBLNCEEFOCIEKAFJAA.perry@stsci.edu>

 
> >hdu.header['very long keyword']='a new value'> 
> I just don't understand why this presents any problem to python.  If
> the keyword starts with "HIERARCH", turn off length checking and treat
> it like any other keyword, wherever length is checked.  What am I
> missing?
> 
I'm sure we'll be able to add this capability to PyFITS. We have some
things to take care of first, but it is on our list of features to
add (and not in the greatly distant future either).

The primary issue is whether or not to automatically invoke this
convention when updating a header that isn't already using it. I tend
to think not, that it requires some special action to invoke it (since
it isn't official FITS. Reading will happen automatically (that's pretty
straightforward I think). So the details of enabling its use will
need to be thought out a bit.

Perry  



From jh at oobleck.astro.cornell.edu  Wed Mar 30 20:24:52 2005
From: jh at oobleck.astro.cornell.edu (Joe Harrington)
Date: Wed, 30 Mar 2005 20:24:52 -0500
Subject: [AstroPy] Re: HIERARCH
In-Reply-To: <NEBBIJKBMLDBLNCEEFOCIEKAFJAA.perry@stsci.edu>
References: <NEBBIJKBMLDBLNCEEFOCIEKAFJAA.perry@stsci.edu>
Message-ID: <200503310124.j2V1OqCp016968@oobleck.astro.cornell.edu>

>add (and not in the greatly distant future either).

That's good to hear!

>The primary issue is whether or not to automatically invoke this
>convention when updating a header that isn't already using it.

I am not intimate with the HIERARCH extension, but I don't think that
there is any difference between a header that has the extension but
doesn't happen to have any HIERARCH keywords in it, and one that
doesn't have the extension.  If that's the case, and a user adds a
HIERARCH keyword to a non-HIERARCH header, you can assume they know
what they are doing and enable it without some funky switch (it won't
happen by accident).  If it's not the case, clearly you don't want to
write HIERARCH-enabled headers for all files.

--jh--


From perry at stsci.edu  Wed Mar 30 21:09:48 2005
From: perry at stsci.edu (Perry Greenfield)
Date: Wed, 30 Mar 2005 21:09:48 -0500
Subject: [AstroPy] Re: HIERARCH
In-Reply-To: <200503310124.j2V1OqCp016968@oobleck.astro.cornell.edu>
References: <NEBBIJKBMLDBLNCEEFOCIEKAFJAA.perry@stsci.edu>
	<200503310124.j2V1OqCp016968@oobleck.astro.cornell.edu>
Message-ID: <0f5b52f88fdc081c6c493cf92178f4e7@stsci.edu>


On Mar 30, 2005, at 8:24 PM, Joe Harrington wrote:

>> add (and not in the greatly distant future either).
>
> That's good to hear!
>
>> The primary issue is whether or not to automatically invoke this
>> convention when updating a header that isn't already using it.
>
> I am not intimate with the HIERARCH extension, but I don't think that
> there is any difference between a header that has the extension but
> doesn't happen to have any HIERARCH keywords in it, and one that
> doesn't have the extension.  If that's the case, and a user adds a
> HIERARCH keyword to a non-HIERARCH header, you can assume they know
> what they are doing and enable it without some funky switch (it won't
> happen by accident).  If it's not the case, clearly you don't want to
> write HIERARCH-enabled headers for all files.
>
I'm not so sure. Users can mistakenly write (through typos or use of 
variables) keywords that are longer than 8 characters without realizing 
they doing so. In such cases it is probably a good thing to raise an 
error. Basically you have two different situations, one where users 
want a strict fits header, and expect to be warned when they have 
non-conforming keywords, and the other case where they don't want to be 
bothered when they are writing long keyword names. So this situation is 
murkier from the point of view of what should be done. You know what 
you want, but it isn't necessarily what other users want for default 
behavior.

Perry



From perry at stsci.edu  Wed Mar 30 21:39:15 2005
From: perry at stsci.edu (Perry Greenfield)
Date: Wed, 30 Mar 2005 21:39:15 -0500
Subject: [AstroPy] Draft specification for PyFITS  functional interface
In-Reply-To: <p06200734be70d1b0e4af@[161.72.92.174]>
References: <5883dfe0.e54edc76.81dc100@comet.stsci.edu>
	<p06200734be70d1b0e4af@[161.72.92.174]>
Message-ID: <bc2442f7397c8b987efd4832dcb11610@stsci.edu>

OK, I'm persuaded that the right thing to do is only have one module: 
pyfits.

1) the functional interface will support keywords specifying whether 
the the
iraf convention will apply or not.

2) Rather than make this value default to one value or another within 
the function,
we'll try going with a module default that a user can use to change 
globally.
There are drawbacks to this (a user can cause the behavior of scripts 
written
assuming the other default to behave differently if they really depend 
on this
distinction), but I think one camp or the other is going to find using 
the
keyword interactively to be a major hassle to bypass the default 
behavior.
I'm not crazy about adding extra sets of functions for the other 
behavior.
We can always do that later if this solution proves to be unworkable.

3) we will add a fitsopen function to alias to open and use the __all__
mechanism to hide open when doing a "from pyfits import *" so that open
does not override the built-in open.

I hope that settles it (fingers crossed).

Perry

On Mar 30, 2005, at 5:14 PM, Peter Erwin wrote:

> At 3:08 PM -0500 3/30/05, <laidler at stsci.edu> wrote:
>> I agree with Peter - to my mind, one of the strengths of
>> Python is that it supports both functional and object-oriented
>> programming, so that it's easy and painless to switch back and
>> forth or mix them as the need arises. In an interactive
>> session, I might be very likely to use the functional routines
>> as long as things are going as I expect, and then want to
>> switch to the OO ones for cases where things are going wrong
>> and I need more direct control of the file's innards.
>
> Well, obviously, I agree with this ;-)
>
> It's actually what I was going to add, but you beat me to it -- one
> of the things that makes Python so useful is its flexibility and *lack*
> of forced adherence to a particular programming style.  Complaints
> that it would be "polluting" the pyfits interface make no sense to me,
> I'm afraid.  (My personal perspective is that it would make pyfits
> *more useful* and easier to start using; I've been slightly leery of
> digging into pyfits because an object-oriented approach seems
> slightly excessive for the things I usually want to do with FITS 
> files.)
> Many of the Python standard libraries are similarly "unclean", and
> more useful because of it.
>
> As for the ".fits[1]" issue -- I'd find it somewhat convenient to be
> able to use that syntax, but it's not a big issue for me.  So if you're
> looking for votes on:
>
> At 3:42 PM -0500 3/30/05, Perry Greenfield wrote:
>> Which is worse:
>> oo-only pyfits and iraf/cfitsio-convention-contaminated xxxfits?
>> oo+functional pyfits and  iraf/cfitsio-convention-contaminated 
>> xxxfits?
>
> then my votes is that the *first* in that list is worse...
>
>    -- Peter



From loredo at astro.cornell.edu  Wed Mar 30 17:36:21 2005
From: loredo at astro.cornell.edu (Tom Loredo)
Date: Wed, 30 Mar 2005 17:36:21 -0500 (EST)
Subject: [AstroPy] Draft specification for PyFITS  functional interface
Message-ID: <200503302236.j2UMaLr09908@laplace.astro.cornell.edu>


Hi folks-

> one
> of the things that makes Python so useful is its flexibility and *lack*
> of forced adherence to a particular programming style.  Complaints
> that it would be "polluting" the pyfits interface make no sense to me

I can't speak for the authors, so perhaps they'll speak for themselves
on this.

That said... if the split is made, are you really going to complain
about having to type:

  import qpyfits, pyfits

versus:

  import pyfits

(or the two "*" imports) on the occassions when you want both interfaces?

As a developer, I think the package idea makes sense, (1) to help
"compartmentalize" development (in that it reflects a logical division
between the OO stuff that does the work, and the functional stuff that
is only an interface to the OO stuff), and (2) to prevent *namespace*
pollution.  Especially interactively, I can be lazy:

  from pyfits import *

If I only use the OO stuff, why should I fill my namespace (esp. in a
module that might itself get imported elsewhere) with all the other
stuff?  If you believe that users who use the functional interface will
often want the OO stuff, we can simply arrange that the qpyfits (or
whatever) module import the OO stuff into its namespace, so if you do:

  from qpyfits import *

you have everything (I'm not advocating this necessarily, but
it could be a way to have and eat the cake).  Or we can define
pyfits this way, but have under it a separate oofits module that
we could import by itself (pyfits.oofits?) when we write a module
or script that just uses the OO stuff.  pyfits.py would just
"suck" the oofits stuff into its namespace (perhaps by judicious
use of __all__ within the various modules).

I don't know what the big deal is regarding this.  Any of the
possible choices will be easy to live with.  FITS stuff is
such a pain (perhaps not incredibly difficult, but definitely
no fun!) that I'm just grateful the PyFITS team is doing
*anything* that takes that load off my back!  Yea team!

Cheers,
Tom


From William.T.Bridgman.1 at gsfc.nasa.gov  Thu Mar 31 09:02:06 2005
From: William.T.Bridgman.1 at gsfc.nasa.gov (W.T. Bridgman)
Date: Thu, 31 Mar 2005 09:02:06 -0500
Subject: [AstroPy] Draft specification for PyFITS  functional interface
In-Reply-To: <200503302236.j2UMaLr09908@laplace.astro.cornell.edu>
References: <200503302236.j2UMaLr09908@laplace.astro.cornell.edu>
Message-ID: <01fe66d634b7a515766be3d2fc25b72f@gsfc.nasa.gov>

Or, to put it more succinctly:

"If I wanted to do procedural programming, I'd stick with IDL."  :^)

I vote for separate modules.  Don't load down the OO module with 
functional wrappers.

Tom
On Mar 30, 2005, at 5:36 PM, Tom Loredo wrote:

>
> Hi folks-
>
>> one
>> of the things that makes Python so useful is its flexibility and 
>> *lack*
>> of forced adherence to a particular programming style.  Complaints
>> that it would be "polluting" the pyfits interface make no sense to me
>
> I can't speak for the authors, so perhaps they'll speak for themselves
> on this.
>
> That said... if the split is made, are you really going to complain
> about having to type:
>
>   import qpyfits, pyfits
>
> versus:
>
>   import pyfits
>
> (or the two "*" imports) on the occassions when you want both 
> interfaces?
>
> As a developer, I think the package idea makes sense, (1) to help
> "compartmentalize" development (in that it reflects a logical division
> between the OO stuff that does the work, and the functional stuff that
> is only an interface to the OO stuff), and (2) to prevent *namespace*
> pollution.  Especially interactively, I can be lazy:
>
>   from pyfits import *
>
> If I only use the OO stuff, why should I fill my namespace (esp. in a
> module that might itself get imported elsewhere) with all the other
> stuff?  If you believe that users who use the functional interface will
> often want the OO stuff, we can simply arrange that the qpyfits (or
> whatever) module import the OO stuff into its namespace, so if you do:
>
>   from qpyfits import *
>
> you have everything (I'm not advocating this necessarily, but
> it could be a way to have and eat the cake).  Or we can define
> pyfits this way, but have under it a separate oofits module that
> we could import by itself (pyfits.oofits?) when we write a module
> or script that just uses the OO stuff.  pyfits.py would just
> "suck" the oofits stuff into its namespace (perhaps by judicious
> use of __all__ within the various modules).
>
> I don't know what the big deal is regarding this.  Any of the
> possible choices will be easy to live with.  FITS stuff is
> such a pain (perhaps not incredibly difficult, but definitely
> no fun!) that I'm just grateful the PyFITS team is doing
> *anything* that takes that load off my back!  Yea team!
>
> Cheers,
> Tom
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.net
> http://www.scipy.net/mailman/listinfo/astropy
>
>
--
Dr. William T."Tom" Bridgman               Scientific Visualization 
Studio
Global Science & Technology, Inc.          NASA/Goddard Space Flight 
Center
Email: William.T.Bridgman.1 at gsfc.nasa.gov  Code 610.3
Phone: 301-286-1346                        Greenbelt, MD 20771
FAX:   301-286-1634                        http://svs.gsfc.nasa.gov/



From jh at oobleck.astro.cornell.edu  Thu Mar 31 09:45:58 2005
From: jh at oobleck.astro.cornell.edu (Joe Harrington)
Date: Thu, 31 Mar 2005 09:45:58 -0500
Subject: [AstroPy] Re: HIERARCH
In-Reply-To: <0f5b52f88fdc081c6c493cf92178f4e7@stsci.edu> (message from Perry
	Greenfield on Wed, 30 Mar 2005 21:09:48 -0500)
References: <NEBBIJKBMLDBLNCEEFOCIEKAFJAA.perry@stsci.edu>
	<200503310124.j2V1OqCp016968@oobleck.astro.cornell.edu>
	<0f5b52f88fdc081c6c493cf92178f4e7@stsci.edu>
Message-ID: <200503311445.j2VEjwqI024313@oobleck.astro.cornell.edu>

>I'm not so sure. Users can mistakenly write (through typos or use of 
>variables) keywords that are longer than 8 characters without realizing 
>they doing so. In such cases it is probably a good thing to raise an 
>error.

Perry, we're in agreement.  I don't want pyfits to accept STARTTIME as
a keyword.  But, I want it to accept HIERARCH ESO STARTTIME into a
header already containing HIERARCH keywords, without complaint.  I
don't think anyone is going to type "HIERARCH" before anything
accidentally.  As far as I know, the HIERARCH extension is the only
long-keyword extension in wide use, and it is in very wide use.  That
might justify special-casing "HIERARCH".  But there might be a more
general way.

Maybe the way to do this is to have a "strict" flag as a property of
the header (separate from actual header entries like SIMPLE=T), and to
clear it if there is a non-standard extension or a recoverable error
in the file when read.  Aside from HIERARCH, there are many other
cases where large amounts of data are in nearly-conforming files.  For
example, the IRTF produced files for a few years in the late '90s with
a final card that read:

END     =

That case is non-standard, but easily recoverable.  If users wanted,
they could standardize those headers and then reset the strict flag.
HIERARCH headers would clear that flag when read as well.  Adding
keywords conforming to a known "gray" convention like HIERARCH would
error out unless the strict flag were cleared, which it would be for
any header that already contained a HIERARCH keyword.  So, people
reading and writing HIERARCH headers wouldn't be bothered with
manually clearing the strict flag, but adding a HIERARCH keyword to a
non-HIERARCH header would require manually clearing the flag first.

But, far more important is that it accept HIERARCH keywords under some
conditions, whether or not one has to turn it on.

--jh--


From laidler at stsci.edu  Thu Mar 31 09:58:44 2005
From: laidler at stsci.edu (Victoria G. Laidler)
Date: Thu, 31 Mar 2005 09:58:44 -0500
Subject: [AstroPy] Draft specification for PyFITS  functional interface
In-Reply-To: <200503302236.j2UMaLr09908@laplace.astro.cornell.edu>
References: <200503302236.j2UMaLr09908@laplace.astro.cornell.edu>
Message-ID: <424C1024.3030104@stsci.edu>

Hi Tom,

>That said... if the split is made, are you really going to complain
>about having to type:
>
>  import qpyfits, pyfits
>
>versus:
>
>  import pyfits
>
>(or the two "*" imports) on the occassions when you want both interfaces?
>
>As a developer, I think the package idea makes sense, (1) to help
>"compartmentalize" development (in that it reflects a logical division
>between the OO stuff that does the work, and the functional stuff that
>is only an interface to the OO stuff), and (2) to prevent *namespace*
>pollution.  Especially interactively, I can be lazy:
>
>  from pyfits import *
>
Honestly, people who are worried about namespace pollution *shouldn't use*

from foo import *

in the first place, whether foo is pyfits or anything else.

Every style guide or similar document I've seen strongly recommends 
against this syntax.

Of course we all get lazy sometimes, but it's logically inconsistent to 
complain about namespace pollution that only happens if you import *.




From ptak at pha.jhu.edu  Thu Mar 31 11:07:50 2005
From: ptak at pha.jhu.edu (Andrew Ptak)
Date: Thu, 31 Mar 2005 11:07:50 -0500
Subject: [AstroPy] Re: HIERARCH
In-Reply-To: <200503311445.j2VEjwqI024313@oobleck.astro.cornell.edu>
References: <NEBBIJKBMLDBLNCEEFOCIEKAFJAA.perry@stsci.edu>
	<200503310124.j2V1OqCp016968@oobleck.astro.cornell.edu>
	<0f5b52f88fdc081c6c493cf92178f4e7@stsci.edu>
	<200503311445.j2VEjwqI024313@oobleck.astro.cornell.edu>
Message-ID: <1112285271.9834.14.camel@bit.localdomain>


> Maybe the way to do this is to have a "strict" flag as a property of
> the header (separate from actual header entries like SIMPLE=T), and to
> clear it if there is a non-standard extension or a recoverable error
> in the file when read.  

It would also make sense to add a "strict" keyword to header functions,
to turn toggle FITS conformance testing (or exceptions).  I haven't had
a chance to use pyfits yet (mainly due to poor performance of numarray
with small arrays... is this still a problem, by the way?), but this
makes me wonder if there already are fits conformance exceptions.

Andy




From hsu at stsci.edu  Thu Mar 31 11:15:51 2005
From: hsu at stsci.edu (Jin-chung Hsu)
Date: Thu, 31 Mar 2005 11:15:51 -0500
Subject: [AstroPy] Re: HIERARCH
In-Reply-To: <1112285271.9834.14.camel@bit.localdomain>
References: <NEBBIJKBMLDBLNCEEFOCIEKAFJAA.perry@stsci.edu>	
	<200503310124.j2V1OqCp016968@oobleck.astro.cornell.edu>	
	<0f5b52f88fdc081c6c493cf92178f4e7@stsci.edu>	
	<200503311445.j2VEjwqI024313@oobleck.astro.cornell.edu>
	<1112285271.9834.14.camel@bit.localdomain>
Message-ID: <424C2237.80806@stsci.edu>

Andrew Ptak wrote:

>>Maybe the way to do this is to have a "strict" flag as a property of
>>the header (separate from actual header entries like SIMPLE=T), and to
>>clear it if there is a non-standard extension or a recoverable error
>>in the file when read.  
>>    
>>
>
>It would also make sense to add a "strict" keyword to header functions,
>to turn toggle FITS conformance testing (or exceptions).  I haven't had
>a chance to use pyfits yet (mainly due to poor performance of numarray
>with small arrays... is this still a problem, by the way?), but this
>makes me wonder if there already are fits conformance exceptions.
>
>Andy
>
We have implemented a flexible verification scheme to conform to FITS 
standards in the more recent
versions (0.9 and later).  The principle is "forgiving" in input, strict 
enforcing standards by default (but can be reset to "ignore" or "fix" 
the non-standards) in output.

HIERARCH is kind of in the middle.  It is NOT a FITS standard.

JC



From perry at stsci.edu  Thu Mar 31 11:24:34 2005
From: perry at stsci.edu (Perry Greenfield)
Date: Thu, 31 Mar 2005 11:24:34 -0500
Subject: [AstroPy] Re: HIERARCH
In-Reply-To: <1112285271.9834.14.camel@bit.localdomain>
References: <NEBBIJKBMLDBLNCEEFOCIEKAFJAA.perry@stsci.edu>
	<200503310124.j2V1OqCp016968@oobleck.astro.cornell.edu>
	<0f5b52f88fdc081c6c493cf92178f4e7@stsci.edu>
	<200503311445.j2VEjwqI024313@oobleck.astro.cornell.edu>
	<1112285271.9834.14.camel@bit.localdomain>
Message-ID: <6da6b67843c97c520151bde81afc17eb@stsci.edu>


On Mar 31, 2005, at 11:07 AM, Andrew Ptak wrote:
>>
>
> It would also make sense to add a "strict" keyword to header functions,
> to turn toggle FITS conformance testing (or exceptions).  I haven't had
> a chance to use pyfits yet (mainly due to poor performance of numarray
> with small arrays... is this still a problem, by the way?), but this

That all depends (as usual). Ufuncs are generally 4 times slower but it
all depends on the specific thing you are doing with small arrays 
(ufuncs,
indexing, array functions, matrix libraries, etc.). Recent improvements
have made it closer to 2 time slower (but not released). If you are 
seeing
numbers much different than a factor of 4 slower (based on v1.2.3) let 
us
know, there are likely various things that can be fixed that we just may
not know about until someone points them out.

> makes me wonder if there already are fits conformance exceptions.
>
> Andy
>
>