[SciPy-User] [AstroPy] Co-ordinating Python astronomy libraries?

[Perry Greenfield]

On Jul 6, 2010, at 2:36 PM, Thomas Robitaille wrote:

> Hi all,
>
> Unlike several people who have replied so far, I have not been  
> involved in general purpose astronomy libraries, but rather a few  
> small packages, including:
>
> - APLpy (FITS image plotting built on matplotlib) at http://aplpy.sourceforge.net 
>  (co-developed with Eli Bressert)
> - ATpy (Seamless multi-format table handler) at http://atpy.sourceforge.net 
>  (also co-developed with Eli Bressert)
> - IDLSave (To read IDL save files into python) at http://idlsave.sourceforge.net
> - python-montage (Montage wrapper) at http://python-montage.sourceforgenet
>
> The main reason for keeping these separate rather than group them  
> into a single package was that each of these packages accomplishes a  
> well-defined task, and we were not prepared to develop all the other  
> tools needed in a general-purpose library. However, I think that  
> from a user point of view, it would be nice to have something more  
> unified.
>
> The main point I want to make is that we need to distinguish between  
> merging all development into a single repository, and bundling  
> packages. There are cases where merging the development of packages  
> does not make sense. For example, in the case of IDLSave, the module  
> was originally developed for the astronomy community, but in fact  
> can be used by any scientist that uses IDL. By developing it as part  
> of a general astronomy libraries makes it less likely that non- 
> astronomers will find it and use it. Another example is Tom  
> Aldcroft's asciitable module (http://cxc.harvard.edu/contrib/asciitable/ 
> ). This was developed to read in ASCII tables, but was not actually  
> designed in an astronomy specific ways, since ASCII tables are of  
> course not limited to astronomy. In the case of such a package,  
> developing it as part of a general astronomy library would be  
> detrimental, but bundling it as part of a general package might be  
> desirable.
>
In principle yes, some of these things are generic. But having  
libraries in a common repository doesn't preclude distributing them  
separately from astronomy.

I also tend to thing that premature fragmentation is as bad a problem  
as being way too monolithic. I'd tend to wait until it was clear we  
had too much stuff in one repository before worrying about how to  
split things up. Sometimes it never becomes an issue. If some items  
grow non-astronomical contributors, they can go to scipy (or something  
similar).

There are some advantages to a common repository:

1) We become more aware of areas of commonality and that foster  
greater sense of making things work better together.
2) Eventually it would help make for more consistent documentation and  
style, and ways of integrating the documentation.
3) Making the process of integrating common or redundant functionality  
easier later (I'll say a little about that in a separate email)

And there are downsides of course:

1) Not everyone wants to use the same version control software (svn,  
git, hg...) or wiki, or issue tracker.
2) Coordination with other developers and projects is more work than  
doing things on your own.

[...]

> 3. An 'all-in-one' package that would also include a full Python  
> distribution, with numpy, scipy, matplotlib, etc - essentially an  
> alternative to EPD for astronomy, e.g. 'APD (Astronomy Python  
> Distribution)'. It could even come with a custom ipython prompt with  
> all packges pre-loaded. All dependencies would be included.
>
In this area STScI and Gemini are attempting to do something like this  
now as a joint project. We are just starting on this effort, but we  
hope to  make a fairly easy to install distribution that includes most  
core tools astronomers would need (or something that would be easy to  
install optional items on). But we don't want to talk too much about  
this until we have something to have people try (I  think Gemini would  
like to have something basic by the end of the year). We intend to  
include IRAF and the common external IRAF packages as well.

> This would then cater to the different levels of python users in  
> Astronomy. One quick note is that if we follow this or a similar  
> model of bundling some packages without merging development  
> repositories, is that developers of small packages will need to be  
> more careful what license they release their code under, to ensure  
> they can be bundled and re-released (but this should be trivial).
>
Yes, licensing is one of the important issues to deal with...

Perry