[AstroPy] Proliferating py-astro-libs

Perry Greenfield perry at stsci.edu
Mon Jun 13 15:15:28 EDT 2011 

It may be too late. Those attending may have already made travel plans  
that preclude an extra day or two (I think it deserves at least a  
day). And it may be too late for those that weren't planning to go.

Perry

On Jun 13, 2011, at 3:07 PM, Tom Aldcroft wrote:

> What about a splinter meeting at SciPy2011?  I guess the question is
> how many interested parties will NOT be there this year.
>
> - Tom
>
> On Mon, Jun 13, 2011 at 2:59 PM, Kelle Cruz <kellecruz at gmail.com>  
> wrote:
>> I think a sit-down is desperately needed to resolve these issues,  
>> figure out
>> the mgmt structure (aka, pecking order), for the BDFL to emerge,  
>> and for
>> progress to occur.
>> I'd be happy to participate as a non-python/programming expert and  
>> maybe
>> provide the voice of the "users".
>> I propose there be a Splinter Meeting at AAS in Austin. (Splinter  
>> Meeting
>> deadline is Dec 1.) Or else someone will have to organize at CfA  
>> (Tom A?
>> Thom R?) or STScI (Marshall? Perry?) since, as far as I can tell,  
>> that seems
>> to be where most of the movers and shakers in this game are located.
>> kelle
>>
>> On Mon, Jun 13, 2011 at 2:00 PM, Perry Greenfield <perry at stsci.edu>  
>> wrote:
>>>
>>> That's a good idea.
>>>
>>> Perry
>>>
>>> On Jun 13, 2011, at 1:43 PM, James Turner wrote:
>>>
>>>> Maybe we should hold an AstroPy conference, where we can discuss
>>>> co-ordination, get to know each other better and even sit down and
>>>> work on libraries together (like at SciPy). That might help  
>>>> generate
>>>> a bit of momentum. Some of us have had meetings before that were
>>>> full of ideas that didn't go anywhere, but I don't think it has to
>>>> be that way if active people on the ground are talking to one  
>>>> another
>>>> rather than having institutions present their plans and try to
>>>> negotiate at a high level.
>>>>
>>>>
>>>> On 13/06/11 13:25, James Turner wrote:
>>>>> It seems that several of us would really like to improve
>>>>> collaboration on Python libraries but have been struggling to pull
>>>>> it off. I've raised the same issue on this list in past months,  
>>>>> but
>>>>> my
>>>>> focus has unavoidably been on other things and since I'm wary of
>>>>> shouting a lot without contributing much, I haven't really been  
>>>>> able
>>>>> to keep the discussion alive...
>>>>>
>>>>> I tend to agree with Mark and Stefan about the question of
>>>>> leadership.
>>>>> Perry & co. at Space Telescope deserve recognition for getting us
>>>>> this
>>>>> far with things like PyFITS and PyRAF. Others have taken the
>>>>> initiative
>>>>> with things like astronomical plotting and documentation sprints.
>>>>> We're
>>>>> still lacking a bit of coherence though, which (as Mark  
>>>>> suggests) is
>>>>> likely to involve one or a few dedicated, energetic,  
>>>>> knowledgeable,
>>>>> hands-on developer(s) who can glue things together. Those people  
>>>>> need
>>>>> to be employed by someone, though, to ensure stability &  
>>>>> continuity
>>>>> (fortunately there's already a bit of that going on at STScI, eg.
>>>>> with
>>>>> Mike and Matplotlib). Personally, I have the motivation but have  
>>>>> not
>>>>> had the time/independence (and might not be assertive enough).
>>>>> Apparently we do have several energetic authors in the community  
>>>>> now
>>>>> (like Thomas & Eli), but each with their own project.
>>>>>
>>>>> A couple of years ago, a number of us at the observatories
>>>>> submitted a
>>>>> white paper to the Decadal Survey, pointing out the need for more
>>>>> co-ordinated funding so that we can have people who focus on  
>>>>> cross-
>>>>> institutional platform development & support. The report from the
>>>>> committee did give a nod to our concerns and their importance, but
>>>>> stopped short of making any recommendation, which basically means
>>>>> "good
>>>>> luck with that". Meanwhile, at Gemini we have had our own  
>>>>> problems to
>>>>> deal with, which make it very difficult for me to propose  
>>>>> something
>>>>> internally beyond working with STScI on the distribution of
>>>>> dependencies that Perry mentioned. Perhaps someone obtaining a  
>>>>> grant
>>>>> for this is not out of the question though.
>>>>>
>>>>> I would like it if we could get together organically behind  
>>>>> Astrolib,
>>>>> but sometimes it's difficult to get people away from their  
>>>>> immediate
>>>>> priorities to focus on that, even within my own institution. If we
>>>>> could get people dedicated to it, though, it could become
>>>>> indispensable
>>>>> enough to attract and co-ordinate more effort. I'm just not sure
>>>>> how we
>>>>> get started at this point and my personal options for tackling the
>>>>> problem seem limited given the overarching funding transition at
>>>>> Gemini
>>>>> and the intense focus on projects that are needed to make that
>>>>> work...
>>>>>
>>>>> Cheers,
>>>>>
>>>>> James.
>>>>>
>>>>>
>>>>> On 10/06/11 09:48, Perry Greenfield wrote:
>>>>>>
>>>>>> On Jun 9, 2011, at 11:12 PM, Thomas Robitaille wrote:
>>>>>>
>>>>>>> I just wanted to also add that (in agreement with Marshall)  
>>>>>>> I'm all
>>>>>>> in favor of many small modules that accomplish a particular task
>>>>>>> well, rather than packages that aim for a 'do-it-all' approach  
>>>>>>> and
>>>>>>> fall short. It's always possible to bundle small packages  
>>>>>>> together
>>>>>>> afterwards, and I don't mean merge development, but instead just
>>>>>>> bundling the packages for installation (kind of like EPD). I  
>>>>>>> think
>>>>>>> that is the easiest approach for all of us.
>>>>>>>
>>>>>>> Maybe in the long run, a specific set of core packages will  
>>>>>>> emerge
>>>>>>> as essential and we can then talk about truly merging them  
>>>>>>> into a
>>>>>>> scipy-like package, but for now, I think the race is still on.  
>>>>>>> And
>>>>>>> after all, there's nothing to say we *have* to achieve the same
>>>>>>> setup as in IDL.
>>>>>>
>>>>>> It sure seems to me that the time is ripe to start trying to
>>>>>> coalesce
>>>>>> some of the ongoing efforts.
>>>>>>
>>>>>> Mind you, I don't think it is necessarily good to start with only
>>>>>> one
>>>>>> version. Allowing a few different approaches initially has its
>>>>>> benefits. You get to see more approaches and ideas in play and
>>>>>> having
>>>>>> experience with them is very helpful in deciding which one is  
>>>>>> more
>>>>>> productive. And sometimes there is room for more than one in the
>>>>>> long
>>>>>> run. The different approaches may have their own niches. But it  
>>>>>> is
>>>>>> hard to imagine any long-term need for more than two or three
>>>>>> different approaches.
>>>>>>
>>>>>> Early on there are some pragmatic needs for different approaches.
>>>>>> For
>>>>>> example, having a fairly "literal" translation of IDL tools into
>>>>>> Python has its benefit. It is very useful for those that would
>>>>>> like to
>>>>>> migrate IDL code, and given the existing IDL versions, make it  
>>>>>> much
>>>>>> easier to test their correctness. But I don't see this as a
>>>>>> substitute
>>>>>> for a good set of modular tools that have a better object  
>>>>>> design and
>>>>>> consistent interfaces with other modules. Doing this is more work
>>>>>> and
>>>>>> will take more time. So a need for both approaches is likely.  
>>>>>> Some
>>>>>> could argue the same for replacing IRAF tasks.
>>>>>>
>>>>>> All this is much easier said than done of course.
>>>>>>
>>>>>> I wish STScI had more resources to devote to this than we've
>>>>>> actually
>>>>>> had. We've been planning to do more on this front than we've
>>>>>> actually
>>>>>> done. Things come up repeatedly that ruin these plans. But it  
>>>>>> may be
>>>>>> worth saying where some of our current efforts are going that may
>>>>>> overlap some of these other efforts.
>>>>>>
>>>>>> 1) We've been planning (along with Gemini, and particularly James
>>>>>> Turner), to try to develop some Sage-like installation package  
>>>>>> that
>>>>>> would make it easy to install all the basic tools for most
>>>>>> astronomers. We had hoped to have a beta version out, but one  
>>>>>> of the
>>>>>> people working on this left at the end of last year, and we've  
>>>>>> not
>>>>>> been able to replace that person. We are going to continue this
>>>>>> effort
>>>>>> with existing staff though. Hopefully in a few months we'll have
>>>>>> something to try out.
>>>>>>
>>>>>> 2) There is a recognition that a more serious effort needs to be
>>>>>> made
>>>>>> to replace IRAF functionality. Perhaps one of the benefits of a  
>>>>>> JWST
>>>>>> delay is that it will allow us to do some of that work more
>>>>>> explicitly. But we would not do it by replacing IRAF tasks one- 
>>>>>> for-
>>>>>> one
>>>>>> but coming up with an entirely different approach which has to  
>>>>>> start
>>>>>> from the bottom up (the end result could have applications that
>>>>>> mostly
>>>>>> emulate IRAF tasks, but also provide much more modular tools).
>>>>>>
>>>>>> 3) More specifically, we are currently focussing on how to handle
>>>>>> WCS
>>>>>> issues in a more general way than they are handled in FITS. If  
>>>>>> there
>>>>>> is interest, perhaps we should say more about the intended  
>>>>>> approach.
>>>>>> This is particularly important for replacing spectrographic  
>>>>>> tools in
>>>>>> IRAF, and this is where we are starting our effort.
>>>>>>
>>>>>> 4) We need a way of saving these WCS models, and FITS is not the
>>>>>> way.
>>>>>> We are looking at an alternate data format, not just for WCS  
>>>>>> models,
>>>>>> but for data in general [gasp!]. Work has begun on this too.
>>>>>>
>>>>>> 5) A lot of our recent work has been on pysynphot and ETCs. We
>>>>>> plan on
>>>>>> making the computational part of our ETCs a released tool. But  
>>>>>> I'm
>>>>>> also wondering if we can generalize the pysynphot spectral models
>>>>>> for
>>>>>> more general use in spectral tools.
>>>>>>
>>>>>> 6) We have been working on a framework for making pipelines  
>>>>>> easier
>>>>>> to
>>>>>> build and configure. That won't be ready for at least a few  
>>>>>> months,
>>>>>> but could well be of general interest and use.
>>>>>>
>>>>>> But besides these things, I would like to see if we can't begin  
>>>>>> the
>>>>>> effort of narrowing some of the underlying libraries being used.
>>>>>> FITS
>>>>>> WCS is one obvious area that seems ripe for that.
>>>>>>
>>>>>> But the community ought to identify one or two areas that are  
>>>>>> of the
>>>>>> most interest in consolidating (let's start small). What should  
>>>>>> we
>>>>>> start with? Focus is important in making any progress in this  
>>>>>> area.
>>>>>>
>>>>>> Perry
>>>>>>
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> AstroPy mailing list
>>>>>> AstroPy at scipy.org
>>>>>> http://mail.scipy.org/mailman/listinfo/astropy
>>>>
>>>>
>>>> --
>>>> James E.H. Turner
>>>> Gemini Observatory Southern Operations Centre,
>>>> Casilla 603,          Tel. (+56) 51 205609
>>>> La Serena, Chile.     Fax. (+56) 51 205650
>>>> _______________________________________________
>>>> AstroPy mailing list
>>>> AstroPy at scipy.org
>>>> http://mail.scipy.org/mailman/listinfo/astropy
>>>
>>> _______________________________________________
>>> AstroPy mailing list
>>> AstroPy at scipy.org
>>> http://mail.scipy.org/mailman/listinfo/astropy
>>
>>
>>
>> --
>> Kelle Cruz, PhD — http://kellecruz.com/
>> 917.725.1334 — Hunter ext: 16486 — AMNH ext: 3404
>>
>> _______________________________________________
>> AstroPy mailing list
>> AstroPy at scipy.org
>> http://mail.scipy.org/mailman/listinfo/astropy
>>
>>
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org
> http://mail.scipy.org/mailman/listinfo/astropy

More information about the AstroPy mailing list