[AstroPy] Co-ordinating Python astronomy libraries?

[Thomas Robitaille]

Hi all,

After reading all the replies, I have the following suggestion to make.

The model scipy follows is to have a 'core' scipy package, and 'scikit' packages which share a common namespace, and which are meant as addons to the scipy package, but have not yet (or might never) make it to the main scipy package:

http://www.scipy.org/scipy/scikits/

"Scipy Toolkits are independent and seperately installable projects hosted under a common namespace. Packages that are distributed in this way are here (instead of in monolithic scipy) for at least one of three general reasons. Each of these reasons use the same high-level namespace (scikits)."

I think we can use this model, and that the following approach can be used here, namely:

- start with a very basic 'astropy' package, with e.g. support for FITS/WCS
- agree to coordinate astronomy packages with a common namespace (e.g. 'astrokit', so for example, APLpy would become astrokit.aplpy). This can help us manage the namespace (as suggested in Joe Harrington's email)
- as astrokit modules mature, they can (if the authors are willing) be merged into the main 'astropy' package, once they have met a number of conditions, including e.g. unit tests, sphinx documentation, limited dependencies (e.g. numpy/scipy/matplotlib, and any package in the 'astropy' package), and compatible license.

The advantage of this model is that this encourages the growth from the bottom up of a core astronomy package, which is manageable, as well as the independent development of other packages. It also means that the core package will be quite stable, because it will only accrete 'astrokit' modules as they become stable and mature. At the same time, it encourages developers to make their own innovative astrokit, but without commitment from the maintainers of the core package to accrete it in future.

In passing, this also leaves the possibility for those who want to develop meta-pacakges of astrokit modules. Also, it will make it easier for those who want to build fully fledged astronomy python distributions.

There have been many ideas passed around in the mailing list, but I think that some are maybe too ambitious. I do think that the idea suggested above is much more manageable. The concrete steps would be:

- setup a central repository for the core packages, as well as astrokit pacakges (although we should leave the option for people to develop astrokit packages outside this repository too, and rely on trust and communication to avoid namespace clashes)
- start a core package with e.g. FITS and WCS support (e.g. pyfits + pywcs)
- set up a list of 'registered' astrokit names to avoid conflict.
- set up a list of recommendations and requirements for astrokit pacakges
- encourage developers of existing packages to use this namespace and follow the guidelines. This would be very little work in some cases, which is why I think it could work.

Sharing a namespace is (I think) the first step to showing that we are willing to work on something coherent, and users would only see two top-level namespaces - astropy and astrokit, which would be a great improvement over the current situation where it is not immediately obvious which packages are astronomy related.

Comments/suggestions/criticism welcome!

Cheers,

Tom

On Jun 30, 2010, at 5:48 PM, James Turner wrote:

> Dear Python users in astronomy,
> 
> At SciPy 2009, I arranged an astronomy BoF where we discussed the
> fact that there are now a number of astronomy libraries for Python
> floating around and maybe it would be good to collect more code into
> a single place. People seemed receptive to this idea and weren't sure
> why it hasn't already happened, given that there has been an Astrolib
> page at SciPy for some years now, with an associated SVN repository:
> 
>   http://scipy.org/AstroLib
> 
> After the meeting last August, I was supposed to contact the mailing
> list and some library authors I had talked to previously, to discuss
> this further. My apologies for taking 10 months to do that! I did
> draft an email the day after the BoF, but then we ran into a hurdle
> with setting up new committers to the AstroLib repository (which has
> taken a lot longer than expected to resolve), so it seemed a bad
> time to suggest that new people start using it.
> 
> To discuss these issues further, we'd like to encourage everyone to
> sign up for the AstroPy mailing list if you are not already on it.
> The traffic is just a few messages per month.
> 
>   http://lists.astropy.scipy.org/mailman/listinfo/astropy
> 
> We (the 2009 BoF group) would also like to hear on the list about
> why people have decided to host their own astronomy library (eg. not
> being aware of the one at SciPy). Are you interested in contributing
> to Astrolib? Do you have any other comments or concerns about
> co-ordinating tools? Our motivation is to make libraries easy to
> find and install, allow sharing code easily, help rationalize
> available functionality and fill in what's missing. A standard
> astronomy library with a single set of documentation should be more
> coherent and easier to maintain. The idea is not to limit authors'
> flexibility of take ownership of their code -- the sub-packages
> can still be maintained by different people.
> 
> If you're at SciPy this week, Perry Greenfield and I would be happy
> to talk to you. If you would like to add your existing library to
> Astrolib, please contact Perry Greenfield or Mark Sienkiewicz at
> STScI for access (contact details at http://scipy.org/AstroLib).
> Note that the repository is being moved to a new server this week,
> after which the URLs will be updated at scipy.org.
> 
> Thanks!
> 
> James Turner (Gemini).
> 
> Bcc: various library authors
> 
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org
> http://mail.scipy.org/mailman/listinfo/astropy