[AstroPy] Proliferating py-astro-libs

Perry Greenfield perry at stsci.edu
Mon Jun 13 14:00:39 EDT 2011

That's a good idea.


On Jun 13, 2011, at 1:43 PM, James Turner wrote:

> Maybe we should hold an AstroPy conference, where we can discuss
> co-ordination, get to know each other better and even sit down and
> work on libraries together (like at SciPy). That might help generate
> a bit of momentum. Some of us have had meetings before that were
> full of ideas that didn't go anywhere, but I don't think it has to
> be that way if active people on the ground are talking to one another
> rather than having institutions present their plans and try to
> negotiate at a high level.
> On 13/06/11 13:25, James Turner wrote:
>> It seems that several of us would really like to improve
>> collaboration on Python libraries but have been struggling to pull
>> it off. I've raised the same issue on this list in past months, but  
>> my
>> focus has unavoidably been on other things and since I'm wary of
>> shouting a lot without contributing much, I haven't really been able
>> to keep the discussion alive...
>> I tend to agree with Mark and Stefan about the question of  
>> leadership.
>> Perry & co. at Space Telescope deserve recognition for getting us  
>> this
>> far with things like PyFITS and PyRAF. Others have taken the  
>> initiative
>> with things like astronomical plotting and documentation sprints.  
>> We're
>> still lacking a bit of coherence though, which (as Mark suggests) is
>> likely to involve one or a few dedicated, energetic, knowledgeable,
>> hands-on developer(s) who can glue things together. Those people need
>> to be employed by someone, though, to ensure stability & continuity
>> (fortunately there's already a bit of that going on at STScI, eg.  
>> with
>> Mike and Matplotlib). Personally, I have the motivation but have not
>> had the time/independence (and might not be assertive enough).
>> Apparently we do have several energetic authors in the community now
>> (like Thomas & Eli), but each with their own project.
>> A couple of years ago, a number of us at the observatories  
>> submitted a
>> white paper to the Decadal Survey, pointing out the need for more
>> co-ordinated funding so that we can have people who focus on cross-
>> institutional platform development & support. The report from the
>> committee did give a nod to our concerns and their importance, but
>> stopped short of making any recommendation, which basically means  
>> "good
>> luck with that". Meanwhile, at Gemini we have had our own problems to
>> deal with, which make it very difficult for me to propose something
>> internally beyond working with STScI on the distribution of
>> dependencies that Perry mentioned. Perhaps someone obtaining a grant
>> for this is not out of the question though.
>> I would like it if we could get together organically behind Astrolib,
>> but sometimes it's difficult to get people away from their immediate
>> priorities to focus on that, even within my own institution. If we
>> could get people dedicated to it, though, it could become  
>> indispensable
>> enough to attract and co-ordinate more effort. I'm just not sure  
>> how we
>> get started at this point and my personal options for tackling the
>> problem seem limited given the overarching funding transition at  
>> Gemini
>> and the intense focus on projects that are needed to make that  
>> work...
>> Cheers,
>> James.
>> On 10/06/11 09:48, Perry Greenfield wrote:
>>> On Jun 9, 2011, at 11:12 PM, Thomas Robitaille wrote:
>>>> I just wanted to also add that (in agreement with Marshall) I'm all
>>>> in favor of many small modules that accomplish a particular task
>>>> well, rather than packages that aim for a 'do-it-all' approach and
>>>> fall short. It's always possible to bundle small packages together
>>>> afterwards, and I don't mean merge development, but instead just
>>>> bundling the packages for installation (kind of like EPD). I think
>>>> that is the easiest approach for all of us.
>>>> Maybe in the long run, a specific set of core packages will emerge
>>>> as essential and we can then talk about truly merging them into a
>>>> scipy-like package, but for now, I think the race is still on. And
>>>> after all, there's nothing to say we *have* to achieve the same
>>>> setup as in IDL.
>>> It sure seems to me that the time is ripe to start trying to  
>>> coalesce
>>> some of the ongoing efforts.
>>> Mind you, I don't think it is necessarily good to start with only  
>>> one
>>> version. Allowing a few different approaches initially has its
>>> benefits. You get to see more approaches and ideas in play and  
>>> having
>>> experience with them is very helpful in deciding which one is more
>>> productive. And sometimes there is room for more than one in the  
>>> long
>>> run. The different approaches may have their own niches. But it is
>>> hard to imagine any long-term need for more than two or three
>>> different approaches.
>>> Early on there are some pragmatic needs for different approaches.  
>>> For
>>> example, having a fairly "literal" translation of IDL tools into
>>> Python has its benefit. It is very useful for those that would  
>>> like to
>>> migrate IDL code, and given the existing IDL versions, make it much
>>> easier to test their correctness. But I don't see this as a  
>>> substitute
>>> for a good set of modular tools that have a better object design and
>>> consistent interfaces with other modules. Doing this is more work  
>>> and
>>> will take more time. So a need for both approaches is likely. Some
>>> could argue the same for replacing IRAF tasks.
>>> All this is much easier said than done of course.
>>> I wish STScI had more resources to devote to this than we've  
>>> actually
>>> had. We've been planning to do more on this front than we've  
>>> actually
>>> done. Things come up repeatedly that ruin these plans. But it may be
>>> worth saying where some of our current efforts are going that may
>>> overlap some of these other efforts.
>>> 1) We've been planning (along with Gemini, and particularly James
>>> Turner), to try to develop some Sage-like installation package that
>>> would make it easy to install all the basic tools for most
>>> astronomers. We had hoped to have a beta version out, but one of the
>>> people working on this left at the end of last year, and we've not
>>> been able to replace that person. We are going to continue this  
>>> effort
>>> with existing staff though. Hopefully in a few months we'll have
>>> something to try out.
>>> 2) There is a recognition that a more serious effort needs to be  
>>> made
>>> to replace IRAF functionality. Perhaps one of the benefits of a JWST
>>> delay is that it will allow us to do some of that work more
>>> explicitly. But we would not do it by replacing IRAF tasks one-for- 
>>> one
>>> but coming up with an entirely different approach which has to start
>>> from the bottom up (the end result could have applications that  
>>> mostly
>>> emulate IRAF tasks, but also provide much more modular tools).
>>> 3) More specifically, we are currently focussing on how to handle  
>>> WCS
>>> issues in a more general way than they are handled in FITS. If there
>>> is interest, perhaps we should say more about the intended approach.
>>> This is particularly important for replacing spectrographic tools in
>>> IRAF, and this is where we are starting our effort.
>>> 4) We need a way of saving these WCS models, and FITS is not the  
>>> way.
>>> We are looking at an alternate data format, not just for WCS models,
>>> but for data in general [gasp!]. Work has begun on this too.
>>> 5) A lot of our recent work has been on pysynphot and ETCs. We  
>>> plan on
>>> making the computational part of our ETCs a released tool. But I'm
>>> also wondering if we can generalize the pysynphot spectral models  
>>> for
>>> more general use in spectral tools.
>>> 6) We have been working on a framework for making pipelines easier  
>>> to
>>> build and configure. That won't be ready for at least a few months,
>>> but could well be of general interest and use.
>>> But besides these things, I would like to see if we can't begin the
>>> effort of narrowing some of the underlying libraries being used.  
>>> FITS
>>> WCS is one obvious area that seems ripe for that.
>>> But the community ought to identify one or two areas that are of the
>>> most interest in consolidating (let's start small). What should we
>>> start with? Focus is important in making any progress in this area.
>>> Perry
>>> _______________________________________________
>>> AstroPy mailing list
>>> AstroPy at scipy.org
>>> http://mail.scipy.org/mailman/listinfo/astropy
> -- 
> James E.H. Turner
> Gemini Observatory Southern Operations Centre,
> Casilla 603,		Tel. (+56) 51 205609
> La Serena, Chile.	Fax. (+56) 51 205650
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org
> http://mail.scipy.org/mailman/listinfo/astropy

More information about the AstroPy mailing list