[AstroPy] Proliferating py-astro-libs

James Turner jturner at gemini.edu
Mon Jun 13 13:25:13 EDT 2011


It seems that several of us would really like to improve
collaboration on Python libraries but have been struggling to pull
it off. I've raised the same issue on this list in past months, but my
focus has unavoidably been on other things and since I'm wary of
shouting a lot without contributing much, I haven't really been able
to keep the discussion alive...

I tend to agree with Mark and Stefan about the question of leadership.
Perry & co. at Space Telescope deserve recognition for getting us this
far with things like PyFITS and PyRAF. Others have taken the initiative
with things like astronomical plotting and documentation sprints. We're
still lacking a bit of coherence though, which (as Mark suggests) is
likely to involve one or a few dedicated, energetic, knowledgeable,
hands-on developer(s) who can glue things together. Those people need
to be employed by someone, though, to ensure stability & continuity
(fortunately there's already a bit of that going on at STScI, eg. with
Mike and Matplotlib). Personally, I have the motivation but have not
had the time/independence (and might not be assertive enough).
Apparently we do have several energetic authors in the community now
(like Thomas & Eli), but each with their own project.

A couple of years ago, a number of us at the observatories submitted a
white paper to the Decadal Survey, pointing out the need for more
co-ordinated funding so that we can have people who focus on cross-
institutional platform development & support. The report from the
committee did give a nod to our concerns and their importance, but
stopped short of making any recommendation, which basically means "good
luck with that". Meanwhile, at Gemini we have had our own problems to
deal with, which make it very difficult for me to propose something
internally beyond working with STScI on the distribution of
dependencies that Perry mentioned. Perhaps someone obtaining a grant
for this is not out of the question though.

I would like it if we could get together organically behind Astrolib,
but sometimes it's difficult to get people away from their immediate
priorities to focus on that, even within my own institution. If we
could get people dedicated to it, though, it could become indispensable
enough to attract and co-ordinate more effort. I'm just not sure how we
get started at this point and my personal options for tackling the
problem seem limited given the overarching funding transition at Gemini
and the intense focus on projects that are needed to make that work...

Cheers,

James.


On 10/06/11 09:48, Perry Greenfield wrote:
>
> On Jun 9, 2011, at 11:12 PM, Thomas Robitaille wrote:
>
>> I just wanted to also add that (in agreement with Marshall) I'm all
>> in favor of many small modules that accomplish a particular task
>> well, rather than packages that aim for a 'do-it-all' approach and
>> fall short. It's always possible to bundle small packages together
>> afterwards, and I don't mean merge development, but instead just
>> bundling the packages for installation (kind of like EPD). I think
>> that is the easiest approach for all of us.
>>
>> Maybe in the long run, a specific set of core packages will emerge
>> as essential and we can then talk about truly merging them into a
>> scipy-like package, but for now, I think the race is still on. And
>> after all, there's nothing to say we *have* to achieve the same
>> setup as in IDL.
>
> It sure seems to me that the time is ripe to start trying to coalesce
> some of the ongoing efforts.
>
> Mind you, I don't think it is necessarily good to start with only one
> version. Allowing a few different approaches initially has its
> benefits. You get to see more approaches and ideas in play and having
> experience with them is very helpful in deciding which one is more
> productive. And sometimes there is room for more than one in the long
> run. The different approaches may have their own niches. But it is
> hard to imagine any long-term need for more than two or three
> different approaches.
>
> Early on there are some pragmatic needs for different approaches. For
> example, having a fairly "literal" translation of IDL tools into
> Python has its benefit. It is very useful for those that would like to
> migrate IDL code, and given the existing IDL versions, make it much
> easier to test their correctness. But I don't see this as a substitute
> for a good set of modular tools that have a better object design and
> consistent interfaces with other modules. Doing this is more work and
> will take more time. So a need for both approaches is likely. Some
> could argue the same for replacing IRAF tasks.
>
> All this is much easier said than done of course.
>
> I wish STScI had more resources to devote to this than we've actually
> had. We've been planning to do more on this front than we've actually
> done. Things come up repeatedly that ruin these plans. But it may be
> worth saying where some of our current efforts are going that may
> overlap some of these other efforts.
>
> 1) We've been planning (along with Gemini, and particularly James
> Turner), to try to develop some Sage-like installation package that
> would make it easy to install all the basic tools for most
> astronomers. We had hoped to have a beta version out, but one of the
> people working on this left at the end of last year, and we've not
> been able to replace that person. We are going to continue this effort
> with existing staff though. Hopefully in a few months we'll have
> something to try out.
>
> 2) There is a recognition that a more serious effort needs to be made
> to replace IRAF functionality. Perhaps one of the benefits of a JWST
> delay is that it will allow us to do some of that work more
> explicitly. But we would not do it by replacing IRAF tasks one-for-one
> but coming up with an entirely different approach which has to start
> from the bottom up (the end result could have applications that mostly
> emulate IRAF tasks, but also provide much more modular tools).
>
> 3) More specifically, we are currently focussing on how to handle WCS
> issues in a more general way than they are handled in FITS. If there
> is interest, perhaps we should say more about the intended approach.
> This is particularly important for replacing spectrographic tools in
> IRAF, and this is where we are starting our effort.
>
> 4) We need a way of saving these WCS models, and FITS is not the way.
> We are looking at an alternate data format, not just for WCS models,
> but for data in general [gasp!]. Work has begun on this too.
>
> 5) A lot of our recent work has been on pysynphot and ETCs. We plan on
> making the computational part of our ETCs a released tool. But I'm
> also wondering if we can generalize the pysynphot spectral models for
> more general use in spectral tools.
>
> 6) We have been working on a framework for making pipelines easier to
> build and configure. That won't be ready for at least a few months,
> but could well be of general interest and use.
>
> But besides these things, I would like to see if we can't begin the
> effort of narrowing some of the underlying libraries being used. FITS
> WCS is one obvious area that seems ripe for that.
>
> But the community ought to identify one or two areas that are of the
> most interest in consolidating (let's start small). What should we
> start with? Focus is important in making any progress in this area.
>
> Perry
>
>
>
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org
> http://mail.scipy.org/mailman/listinfo/astropy



More information about the AstroPy mailing list