[AstroPy] Proliferating py-astro-libs

[Erik Tollerud]

I definitely agree with Marshall's starting point that consolidation
would be great, but I'd like to weigh in with a slightly different
point-of-view of the "small modules" approach.  The danger of
*independent* small modules is that it can lead to *huge* versioning
problems and reproducibility issues.  This is one of the most
frustrating problems I fight with on a seemingly daily basis with IDL.

Certain applications are much better for small modules - for example,
pyfits is great as a stand-alone tool because it really is just used
by other modules and there's little need for the other modules to
"talk back" to pyfits, as it were. pywcs is similar (although not
quite as much so, given the need for coordinate systems).  On the
other hand, consider the situation of a set of classes designed to
represent photometric bands (and associated photometry routines).  It
makes a lot of sense to represent these bands in a similar way to how
you would represent a spectrum.  And if you want to have another class
for an SED, you need to both have the SED understand spectra and
photometry, *and* there are many use cases where you want to treat the
SED as a spectrum of sorts.  This is where small modules for each of
these things suddenly become frustrating because you spend more time
dealing with module/package management then you do with the actual
science.

This *could* by solved via an EPD-like approach, but those require
that each of the sub-modules be fairly carefully/consistently
packaged, and it seems like that just doesn't happen in astronomy.  It
also makes it much harder to coordinate development, because the
versioning problems are only solved at every release.  Most python
packages don't really have that complicated dependencies, unlike the
astronomy case where conceivable a given module depends on many
different modules in the scenario outlined above.

So the vision I have for astropysics is to house all such modules in
one *package*, while still implementing them as separate modules
within that package.  As Thomas points out, combining this with the
excellent capabilities of git and github means different people can
work on different modules on different branches, while still keeping
it a single package. This is akin to the idea of the Goddard IDL
library (an approach that is clearly working fairly well with the
community), while still being pythonic and up-to-date in terms of project
management.

With this in mind, I agree with Wolfgang's suggestion of a consistent
set of base classes... But note that this is also already implemented
in many places in astropysics, given that it was my original design
philosophy (in fact, Wolfgang, I would strongly urge you to look at
the astropysics.spec.Spectrum class before writing your own - I
designed it with exactly the idea you had in mind.).

I think the key to addresses most of these concerns is a
community-driven package using the tools and methods that open source
software people have spent decades refining.  I think these models are
something that python-based astronomy would do well to learn from and
make use of for our own needs. Of course I personally like the idea of
doing this
with astropysics, given that it's my baby, but the broader ideas for the
community certainly don't have to be tied to that particular package.



On Fri, Jun 10, 2011 at 2:57 AM, Stefan Czesla
<stefan.czesla at hs.uni-hamburg.de> wrote:
> Dear all,
>
> with great interest, I follow the discussion on how packet proliferation for
> astronomy
> should take place. I would like to comment on some issues. Marshall kicked
> off the
> discussion with a little provocation (not too critical btw.). I shall first
> say that PyAstronomy
> (and the other mentioned projects, except pyastrolib, as I know), is not an
> "attempt at building a core set of routines ported from the IDL library";
> that is the smallest
> part at least of PyA. Nonetheless, I agree with Marshall that something is
> wrong with
> the development model currently pursued by "the community".
>
> People will always tend to
> implement there own small wavelength conversion routines etc. until they do
> not find a solution
> that convinces them that it is worth to be installed and "learned". Not
> until they are convinced that
> the latter is more convenient than doing it with a quick hack, we will see a
> successful project covering that.
> I agree that currently no particular project has gained enough ground to
> take this role. Taro is quite right in
> pointing out that there is no de facto standard yet in Python, and as the
> community has not gathered behind
> one project, I do not know whether we can already speak of real community
> projects. Considering that Taro
> probably did also not exactly find what was needed, the choice of
> "rolling out [] own ports" seems quite natural.
>
> Behind forming a beautiful spiral stands the question of what makes a
> successful open-source project - not
> raised for the first time here. Although I do not know, I believe that
> documentation and in particular examples (i.e., simplicity and
> usability) is a key to convince users to give it a try.  From the developer
> side, I assume that project leadership is crucial.
> As long as the state is fuzzy like now, releasing "just another" project
> does at least not hurt anybody.
>
> So - I am curious to see how this evolves,
>
> yours,
> Stefan
>
>
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org
> http://mail.scipy.org/mailman/listinfo/astropy
>
>



-- 
Erik Tollerud