[AstroPy] AstroPy Digest, Vol 58, Issue 16

[Eli Bressert]

Organizing a splinter meeting at the winter AAS would be a great idea (as Kelle suggested) to get the AstroPy movement going. I'll be at the meeting this year and would be happy to help out on this front.

It seems to me that the strength of Python and modules is that everyone has access to the language (unlike IDL) and multiple modules are developed all the time. If we look around the astronomy community there is already a large number of Python modules for astronomy (e.g. PyFITS, astropysics, asciitables, and many more - see http://www.astropython.org/resources). One of the main difficulties that we are having is integration between the modules. 

Most of the astronomy modules for Python use Numpy and Scipy (frequently Matplotlib too), but things diverge fairly quickly from that point on. Should we agree on a list of the core astronomy modules? Is this possible and is it a good idea? All of the things I have mentioned has shown up on this mailing list before, but it would be useful to move this forward. 

One thought that came up (this is *only* a suggestion) is should we have a basic python package called astropy (when we have agreed on the basic packages for it)? If we do this we could use a similar scheme to Scipy's Scikits (call it astrokits?). This would ensure that the modules would use core astronomy modules. Again, this is just a exploratory idea and I would be happy to hear what others have to say. 

Cheers,
Eli





On Monday, June 13, 2011 at 1:23 PM, astropy-request at scipy.org wrote:

> Send AstroPy mailing list submissions to
> astropy at scipy.org (mailto:astropy at scipy.org)
> 
> To subscribe or unsubscribe via the World Wide Web, visit
> http://mail.scipy.org/mailman/listinfo/astropy
> or, via email, send a message with subject or body 'help' to
> astropy-request at scipy.org (mailto:astropy-request at scipy.org)
> 
> You can reach the person managing the list at
> astropy-owner at scipy.org (mailto:astropy-owner at scipy.org)
> 
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of AstroPy digest..."
> 
> 
> Today's Topics:
> 
>  1. Re: Proliferating py-astro-libs (Matthew Turk)
>  2. Re: Proliferating py-astro-libs (Perry Greenfield)
>  3. Re: Proliferating py-astro-libs (Kelle Cruz)
> 
> 
> ----------------------------------------------------------------------
> 
> Message: 1
> Date: Mon, 13 Jun 2011 12:11:13 -0700
> From: Matthew Turk <matthewturk at gmail.com (mailto:matthewturk at gmail.com)>
> Subject: Re: [AstroPy] Proliferating py-astro-libs
> To: Tom Aldcroft <aldcroft at head.cfa.harvard.edu (mailto:aldcroft at head.cfa.harvard.edu)>
> Cc: astropy <astropy at scipy.org (mailto:astropy at scipy.org)>
> Message-ID: <BANLkTinvOyspdCdzB_2+=tjv1XNcPSkSUA at mail.gmail.com (mailto:tjv1XNcPSkSUA at mail.gmail.com)>
> Content-Type: text/plain; charset=windows-1252
> 
> Hi all,
> 
> I'd be very interested in participating in a discussion of these
> issues. I am not an observer, but I am a member of the astrophysical
> simulations community concerned with issues of interop, open science,
> and developing community-focused packages for scientific analysis, and
> I believe that increasing cross-talk between the observation
> community and the simulations community would be of great benefit to
> all concerned.
> 
> I won't be at SciPy (haven't been able to make it since 2009,
> unfortunately!) but I think this kind of discussion might warrant its
> own mini-workshop.
> 
> -Matt
> 
> On Mon, Jun 13, 2011 at 12:07 PM, Tom Aldcroft
> <aldcroft at head.cfa.harvard.edu (mailto:aldcroft at head.cfa.harvard.edu)> wrote:
> > What about a splinter meeting at SciPy2011? ?I guess the question is
> > how many interested parties will NOT be there this year.
> > 
> > - Tom
> > 
> > On Mon, Jun 13, 2011 at 2:59 PM, Kelle Cruz <kellecruz at gmail.com (mailto:kellecruz at gmail.com)> wrote:
> > > I think a sit-down is desperately needed to resolve these issues, figure out
> > > the mgmt structure (aka, pecking order), for the BDFL to emerge, and for
> > > progress to occur.
> > > I'd be happy to participate as a non-python/programming expert and maybe
> > > provide the voice of the "users".
> > > I propose there be a Splinter Meeting at AAS in Austin. (Splinter Meeting
> > > deadline is Dec 1.) Or else someone will have to organize at CfA (Tom A?
> > > Thom R?) or STScI (Marshall? Perry?) since, as far as I can tell, that seems
> > > to be where most of the movers and shakers in this game are located.
> > > kelle
> > > 
> > > On Mon, Jun 13, 2011 at 2:00 PM, Perry Greenfield <perry at stsci.edu (mailto:perry at stsci.edu)> wrote:
> > > > 
> > > > That's a good idea.
> > > > 
> > > > Perry
> > > > 
> > > > On Jun 13, 2011, at 1:43 PM, James Turner wrote:
> > > > 
> > > > > Maybe we should hold an AstroPy conference, where we can discuss
> > > > > co-ordination, get to know each other better and even sit down and
> > > > > work on libraries together (like at SciPy). That might help generate
> > > > > a bit of momentum. Some of us have had meetings before that were
> > > > > full of ideas that didn't go anywhere, but I don't think it has to
> > > > > be that way if active people on the ground are talking to one another
> > > > > rather than having institutions present their plans and try to
> > > > > negotiate at a high level.
> > > > > 
> > > > > 
> > > > > On 13/06/11 13:25, James Turner wrote:
> > > > > > It seems that several of us would really like to improve
> > > > > > collaboration on Python libraries but have been struggling to pull
> > > > > > it off. I've raised the same issue on this list in past months, but
> > > > > > my
> > > > > > focus has unavoidably been on other things and since I'm wary of
> > > > > > shouting a lot without contributing much, I haven't really been able
> > > > > > to keep the discussion alive...
> > > > > > 
> > > > > > I tend to agree with Mark and Stefan about the question of
> > > > > > leadership.
> > > > > > Perry & co. at Space Telescope deserve recognition for getting us
> > > > > > this
> > > > > > far with things like PyFITS and PyRAF. Others have taken the
> > > > > > initiative
> > > > > > with things like astronomical plotting and documentation sprints.
> > > > > > We're
> > > > > > still lacking a bit of coherence though, which (as Mark suggests) is
> > > > > > likely to involve one or a few dedicated, energetic, knowledgeable,
> > > > > > hands-on developer(s) who can glue things together. Those people need
> > > > > > to be employed by someone, though, to ensure stability & continuity
> > > > > > (fortunately there's already a bit of that going on at STScI, eg.
> > > > > > with
> > > > > > Mike and Matplotlib). Personally, I have the motivation but have not
> > > > > > had the time/independence (and might not be assertive enough).
> > > > > > Apparently we do have several energetic authors in the community now
> > > > > > (like Thomas & Eli), but each with their own project.
> > > > > > 
> > > > > > A couple of years ago, a number of us at the observatories
> > > > > > submitted a
> > > > > > white paper to the Decadal Survey, pointing out the need for more
> > > > > > co-ordinated funding so that we can have people who focus on cross-
> > > > > > institutional platform development & support. The report from the
> > > > > > committee did give a nod to our concerns and their importance, but
> > > > > > stopped short of making any recommendation, which basically means
> > > > > > "good
> > > > > > luck with that". Meanwhile, at Gemini we have had our own problems to
> > > > > > deal with, which make it very difficult for me to propose something
> > > > > > internally beyond working with STScI on the distribution of
> > > > > > dependencies that Perry mentioned. Perhaps someone obtaining a grant
> > > > > > for this is not out of the question though.
> > > > > > 
> > > > > > I would like it if we could get together organically behind Astrolib,
> > > > > > but sometimes it's difficult to get people away from their immediate
> > > > > > priorities to focus on that, even within my own institution. If we
> > > > > > could get people dedicated to it, though, it could become
> > > > > > indispensable
> > > > > > enough to attract and co-ordinate more effort. I'm just not sure
> > > > > > how we
> > > > > > get started at this point and my personal options for tackling the
> > > > > > problem seem limited given the overarching funding transition at
> > > > > > Gemini
> > > > > > and the intense focus on projects that are needed to make that
> > > > > > work...
> > > > > > 
> > > > > > Cheers,
> > > > > > 
> > > > > > James.
> > > > > > 
> > > > > > 
> > > > > > On 10/06/11 09:48, Perry Greenfield wrote:
> > > > > > > 
> > > > > > > On Jun 9, 2011, at 11:12 PM, Thomas Robitaille wrote:
> > > > > > > 
> > > > > > > > I just wanted to also add that (in agreement with Marshall) I'm all
> > > > > > > > in favor of many small modules that accomplish a particular task
> > > > > > > > well, rather than packages that aim for a 'do-it-all' approach and
> > > > > > > > fall short. It's always possible to bundle small packages together
> > > > > > > > afterwards, and I don't mean merge development, but instead just
> > > > > > > > bundling the packages for installation (kind of like EPD). I think
> > > > > > > > that is the easiest approach for all of us.
> > > > > > > > 
> > > > > > > > Maybe in the long run, a specific set of core packages will emerge
> > > > > > > > as essential and we can then talk about truly merging them into a
> > > > > > > > scipy-like package, but for now, I think the race is still on. And
> > > > > > > > after all, there's nothing to say we *have* to achieve the same
> > > > > > > > setup as in IDL.
> > > > > > > 
> > > > > > > It sure seems to me that the time is ripe to start trying to
> > > > > > > coalesce
> > > > > > > some of the ongoing efforts.
> > > > > > > 
> > > > > > > Mind you, I don't think it is necessarily good to start with only
> > > > > > > one
> > > > > > > version. Allowing a few different approaches initially has its
> > > > > > > benefits. You get to see more approaches and ideas in play and
> > > > > > > having
> > > > > > > experience with them is very helpful in deciding which one is more
> > > > > > > productive. And sometimes there is room for more than one in the
> > > > > > > long
> > > > > > > run. The different approaches may have their own niches. But it is
> > > > > > > hard to imagine any long-term need for more than two or three
> > > > > > > different approaches.
> > > > > > > 
> > > > > > > Early on there are some pragmatic needs for different approaches.
> > > > > > > For
> > > > > > > example, having a fairly "literal" translation of IDL tools into
> > > > > > > Python has its benefit. It is very useful for those that would
> > > > > > > like to
> > > > > > > migrate IDL code, and given the existing IDL versions, make it much
> > > > > > > easier to test their correctness. But I don't see this as a
> > > > > > > substitute
> > > > > > > for a good set of modular tools that have a better object design and
> > > > > > > consistent interfaces with other modules. Doing this is more work
> > > > > > > and
> > > > > > > will take more time. So a need for both approaches is likely. Some
> > > > > > > could argue the same for replacing IRAF tasks.
> > > > > > > 
> > > > > > > All this is much easier said than done of course.
> > > > > > > 
> > > > > > > I wish STScI had more resources to devote to this than we've
> > > > > > > actually
> > > > > > > had. We've been planning to do more on this front than we've
> > > > > > > actually
> > > > > > > done. Things come up repeatedly that ruin these plans. But it may be
> > > > > > > worth saying where some of our current efforts are going that may
> > > > > > > overlap some of these other efforts.
> > > > > > > 
> > > > > > > 1) We've been planning (along with Gemini, and particularly James
> > > > > > > Turner), to try to develop some Sage-like installation package that
> > > > > > > would make it easy to install all the basic tools for most
> > > > > > > astronomers. We had hoped to have a beta version out, but one of the
> > > > > > > people working on this left at the end of last year, and we've not
> > > > > > > been able to replace that person. We are going to continue this
> > > > > > > effort
> > > > > > > with existing staff though. Hopefully in a few months we'll have
> > > > > > > something to try out.
> > > > > > > 
> > > > > > > 2) There is a recognition that a more serious effort needs to be
> > > > > > > made
> > > > > > > to replace IRAF functionality. Perhaps one of the benefits of a JWST
> > > > > > > delay is that it will allow us to do some of that work more
> > > > > > > explicitly. But we would not do it by replacing IRAF tasks one-for-
> > > > > > > one
> > > > > > > but coming up with an entirely different approach which has to start
> > > > > > > from the bottom up (the end result could have applications that
> > > > > > > mostly
> > > > > > > emulate IRAF tasks, but also provide much more modular tools).
> > > > > > > 
> > > > > > > 3) More specifically, we are currently focussing on how to handle
> > > > > > > WCS
> > > > > > > issues in a more general way than they are handled in FITS. If there
> > > > > > > is interest, perhaps we should say more about the intended approach.
> > > > > > > This is particularly important for replacing spectrographic tools in
> > > > > > > IRAF, and this is where we are starting our effort.
> > > > > > > 
> > > > > > > 4) We need a way of saving these WCS models, and FITS is not the
> > > > > > > way.
> > > > > > > We are looking at an alternate data format, not just for WCS models,
> > > > > > > but for data in general [gasp!]. Work has begun on this too.
> > > > > > > 
> > > > > > > 5) A lot of our recent work has been on pysynphot and ETCs. We
> > > > > > > plan on
> > > > > > > making the computational part of our ETCs a released tool. But I'm
> > > > > > > also wondering if we can generalize the pysynphot spectral models
> > > > > > > for
> > > > > > > more general use in spectral tools.
> > > > > > > 
> > > > > > > 6) We have been working on a framework for making pipelines easier
> > > > > > > to
> > > > > > > build and configure. That won't be ready for at least a few months,
> > > > > > > but could well be of general interest and use.
> > > > > > > 
> > > > > > > But besides these things, I would like to see if we can't begin the
> > > > > > > effort of narrowing some of the underlying libraries being used.
> > > > > > > FITS
> > > > > > > WCS is one obvious area that seems ripe for that.
> > > > > > > 
> > > > > > > But the community ought to identify one or two areas that are of the
> > > > > > > most interest in consolidating (let's start small). What should we
> > > > > > > start with? Focus is important in making any progress in this area.
> > > > > > > 
> > > > > > > Perry
> > > > > > > 
> > > > > > > 
> > > > > > > 
> > > > > > > _______________________________________________
> > > > > > > AstroPy mailing list
> > > > > > > AstroPy at scipy.org (mailto:AstroPy at scipy.org)
> > > > > > > http://mail.scipy.org/mailman/listinfo/astropy
> > > > > 
> > > > > 
> > > > > --
> > > > > James E.H. Turner
> > > > > Gemini Observatory Southern Operations Centre,
> > > > > Casilla 603, ? ? ? ? ?Tel. (+56) 51 205609
> > > > > La Serena, Chile. ? ? Fax. (+56) 51 205650
> > > > > _______________________________________________
> > > > > AstroPy mailing list
> > > > > AstroPy at scipy.org (mailto:AstroPy at scipy.org)
> > > > > http://mail.scipy.org/mailman/listinfo/astropy
> > > > 
> > > > _______________________________________________
> > > > AstroPy mailing list
> > > > AstroPy at scipy.org (mailto:AstroPy at scipy.org)
> > > > http://mail.scipy.org/mailman/listinfo/astropy
> > > 
> > > 
> > > 
> > > --
> > > Kelle Cruz, PhD ??http://kellecruz.com/
> > > 917.725.1334 ??Hunter ext: 16486 ? AMNH ext: 3404
> > > 
> > > _______________________________________________
> > > AstroPy mailing list
> > > AstroPy at scipy.org (mailto:AstroPy at scipy.org)
> > > http://mail.scipy.org/mailman/listinfo/astropy
> > _______________________________________________
> > AstroPy mailing list
> > AstroPy at scipy.org (mailto:AstroPy at scipy.org)
> > http://mail.scipy.org/mailman/listinfo/astropy
> 
> 
> ------------------------------
> 
> Message: 2
> Date: Mon, 13 Jun 2011 15:15:28 -0400
> From: Perry Greenfield <perry at stsci.edu (mailto:perry at stsci.edu)>
> Subject: Re: [AstroPy] Proliferating py-astro-libs
> To: Tom Aldcroft <aldcroft at head.cfa.harvard.edu (mailto:aldcroft at head.cfa.harvard.edu)>
> Cc: astropy <astropy at scipy.org (mailto:astropy at scipy.org)>
> Message-ID: <D3C94402-208C-4FEF-B158-0EE12D495E5F at stsci.edu (mailto:D3C94402-208C-4FEF-B158-0EE12D495E5F at stsci.edu)>
> Content-Type: text/plain; charset=WINDOWS-1252; format=flowed;
>  delsp=yes
> 
> It may be too late. Those attending may have already made travel plans 
> that preclude an extra day or two (I think it deserves at least a 
> day). And it may be too late for those that weren't planning to go.
> 
> Perry
> 
> On Jun 13, 2011, at 3:07 PM, Tom Aldcroft wrote:
> 
> > What about a splinter meeting at SciPy2011? I guess the question is
> > how many interested parties will NOT be there this year.
> > 
> > - Tom
> > 
> > On Mon, Jun 13, 2011 at 2:59 PM, Kelle Cruz <kellecruz at gmail.com (mailto:kellecruz at gmail.com)> 
> > wrote:
> > > I think a sit-down is desperately needed to resolve these issues, 
> > > figure out
> > > the mgmt structure (aka, pecking order), for the BDFL to emerge, 
> > > and for
> > > progress to occur.
> > > I'd be happy to participate as a non-python/programming expert and 
> > > maybe
> > > provide the voice of the "users".
> > > I propose there be a Splinter Meeting at AAS in Austin. (Splinter 
> > > Meeting
> > > deadline is Dec 1.) Or else someone will have to organize at CfA 
> > > (Tom A?
> > > Thom R?) or STScI (Marshall? Perry?) since, as far as I can tell, 
> > > that seems
> > > to be where most of the movers and shakers in this game are located.
> > > kelle
> > > 
> > > On Mon, Jun 13, 2011 at 2:00 PM, Perry Greenfield <perry at stsci.edu (mailto:perry at stsci.edu)> 
> > > wrote:
> > > > 
> > > > That's a good idea.
> > > > 
> > > > Perry
> > > > 
> > > > On Jun 13, 2011, at 1:43 PM, James Turner wrote:
> > > > 
> > > > > Maybe we should hold an AstroPy conference, where we can discuss
> > > > > co-ordination, get to know each other better and even sit down and
> > > > > work on libraries together (like at SciPy). That might help 
> > > > > generate
> > > > > a bit of momentum. Some of us have had meetings before that were
> > > > > full of ideas that didn't go anywhere, but I don't think it has to
> > > > > be that way if active people on the ground are talking to one 
> > > > > another
> > > > > rather than having institutions present their plans and try to
> > > > > negotiate at a high level.
> > > > > 
> > > > > 
> > > > > On 13/06/11 13:25, James Turner wrote:
> > > > > > It seems that several of us would really like to improve
> > > > > > collaboration on Python libraries but have been struggling to pull
> > > > > > it off. I've raised the same issue on this list in past months, 
> > > > > > but
> > > > > > my
> > > > > > focus has unavoidably been on other things and since I'm wary of
> > > > > > shouting a lot without contributing much, I haven't really been 
> > > > > > able
> > > > > > to keep the discussion alive...
> > > > > > 
> > > > > > I tend to agree with Mark and Stefan about the question of
> > > > > > leadership.
> > > > > > Perry & co. at Space Telescope deserve recognition for getting us
> > > > > > this
> > > > > > far with things like PyFITS and PyRAF. Others have taken the
> > > > > > initiative
> > > > > > with things like astronomical plotting and documentation sprints.
> > > > > > We're
> > > > > > still lacking a bit of coherence though, which (as Mark 
> > > > > > suggests) is
> > > > > > likely to involve one or a few dedicated, energetic, 
> > > > > > knowledgeable,
> > > > > > hands-on developer(s) who can glue things together. Those people 
> > > > > > need
> > > > > > to be employed by someone, though, to ensure stability & 
> > > > > > continuity
> > > > > > (fortunately there's already a bit of that going on at STScI, eg.
> > > > > > with
> > > > > > Mike and Matplotlib). Personally, I have the motivation but have 
> > > > > > not
> > > > > > had the time/independence (and might not be assertive enough).
> > > > > > Apparently we do have several energetic authors in the community 
> > > > > > now
> > > > > > (like Thomas & Eli), but each with their own project.
> > > > > > 
> > > > > > A couple of years ago, a number of us at the observatories
> > > > > > submitted a
> > > > > > white paper to the Decadal Survey, pointing out the need for more
> > > > > > co-ordinated funding so that we can have people who focus on 
> > > > > > cross-
> > > > > > institutional platform development & support. The report from the
> > > > > > committee did give a nod to our concerns and their importance, but
> > > > > > stopped short of making any recommendation, which basically means
> > > > > > "good
> > > > > > luck with that". Meanwhile, at Gemini we have had our own 
> > > > > > problems to
> > > > > > deal with, which make it very difficult for me to propose 
> > > > > > something
> > > > > > internally beyond working with STScI on the distribution of
> > > > > > dependencies that Perry mentioned. Perhaps someone obtaining a 
> > > > > > grant
> > > > > > for this is not out of the question though.
> > > > > > 
> > > > > > I would like it if we could get together organically behind 
> > > > > > Astrolib,
> > > > > > but sometimes it's difficult to get people away from their 
> > > > > > immediate
> > > > > > priorities to focus on that, even within my own institution. If we
> > > > > > could get people dedicated to it, though, it could become
> > > > > > indispensable
> > > > > > enough to attract and co-ordinate more effort. I'm just not sure
> > > > > > how we
> > > > > > get started at this point and my personal options for tackling the
> > > > > > problem seem limited given the overarching funding transition at
> > > > > > Gemini
> > > > > > and the intense focus on projects that are needed to make that
> > > > > > work...
> > > > > > 
> > > > > > Cheers,
> > > > > > 
> > > > > > James.
> > > > > > 
> > > > > > 
> > > > > > On 10/06/11 09:48, Perry Greenfield wrote:
> > > > > > > 
> > > > > > > On Jun 9, 2011, at 11:12 PM, Thomas Robitaille wrote:
> > > > > > > 
> > > > > > > > I just wanted to also add that (in agreement with Marshall) 
> > > > > > > > I'm all
> > > > > > > > in favor of many small modules that accomplish a particular task
> > > > > > > > well, rather than packages that aim for a 'do-it-all' approach 
> > > > > > > > and
> > > > > > > > fall short. It's always possible to bundle small packages 
> > > > > > > > together
> > > > > > > > afterwards, and I don't mean merge development, but instead just
> > > > > > > > bundling the packages for installation (kind of like EPD). I 
> > > > > > > > think
> > > > > > > > that is the easiest approach for all of us.
> > > > > > > > 
> > > > > > > > Maybe in the long run, a specific set of core packages will 
> > > > > > > > emerge
> > > > > > > > as essential and we can then talk about truly merging them 
> > > > > > > > into a
> > > > > > > > scipy-like package, but for now, I think the race is still on. 
> > > > > > > > And
> > > > > > > > after all, there's nothing to say we *have* to achieve the same
> > > > > > > > setup as in IDL.
> > > > > > > 
> > > > > > > It sure seems to me that the time is ripe to start trying to
> > > > > > > coalesce
> > > > > > > some of the ongoing efforts.
> > > > > > > 
> > > > > > > Mind you, I don't think it is necessarily good to start with only
> > > > > > > one
> > > > > > > version. Allowing a few different approaches initially has its
> > > > > > > benefits. You get to see more approaches and ideas in play and
> > > > > > > having
> > > > > > > experience with them is very helpful in deciding which one is 
> > > > > > > more
> > > > > > > productive. And sometimes there is room for more than one in the
> > > > > > > long
> > > > > > > run. The different approaches may have their own niches. But it 
> > > > > > > is
> > > > > > > hard to imagine any long-term need for more than two or three
> > > > > > > different approaches.
> > > > > > > 
> > > > > > > Early on there are some pragmatic needs for different approaches.
> > > > > > > For
> > > > > > > example, having a fairly "literal" translation of IDL tools into
> > > > > > > Python has its benefit. It is very useful for those that would
> > > > > > > like to
> > > > > > > migrate IDL code, and given the existing IDL versions, make it 
> > > > > > > much
> > > > > > > easier to test their correctness. But I don't see this as a
> > > > > > > substitute
> > > > > > > for a good set of modular tools that have a better object 
> > > > > > > design and
> > > > > > > consistent interfaces with other modules. Doing this is more work
> > > > > > > and
> > > > > > > will take more time. So a need for both approaches is likely. 
> > > > > > > Some
> > > > > > > could argue the same for replacing IRAF tasks.
> > > > > > > 
> > > > > > > All this is much easier said than done of course.
> > > > > > > 
> > > > > > > I wish STScI had more resources to devote to this than we've
> > > > > > > actually
> > > > > > > had. We've been planning to do more on this front than we've
> > > > > > > actually
> > > > > > > done. Things come up repeatedly that ruin these plans. But it 
> > > > > > > may be
> > > > > > > worth saying where some of our current efforts are going that may
> > > > > > > overlap some of these other efforts.
> > > > > > > 
> > > > > > > 1) We've been planning (along with Gemini, and particularly James
> > > > > > > Turner), to try to develop some Sage-like installation package 
> > > > > > > that
> > > > > > > would make it easy to install all the basic tools for most
> > > > > > > astronomers. We had hoped to have a beta version out, but one 
> > > > > > > of the
> > > > > > > people working on this left at the end of last year, and we've 
> > > > > > > not
> > > > > > > been able to replace that person. We are going to continue this
> > > > > > > effort
> > > > > > > with existing staff though. Hopefully in a few months we'll have
> > > > > > > something to try out.
> > > > > > > 
> > > > > > > 2) There is a recognition that a more serious effort needs to be
> > > > > > > made
> > > > > > > to replace IRAF functionality. Perhaps one of the benefits of a 
> > > > > > > JWST
> > > > > > > delay is that it will allow us to do some of that work more
> > > > > > > explicitly. But we would not do it by replacing IRAF tasks one- 
> > > > > > > for-
> > > > > > > one
> > > > > > > but coming up with an entirely different approach which has to 
> > > > > > > start
> > > > > > > from the bottom up (the end result could have applications that
> > > > > > > mostly
> > > > > > > emulate IRAF tasks, but also provide much more modular tools).
> > > > > > > 
> > > > > > > 3) More specifically, we are currently focussing on how to handle
> > > > > > > WCS
> > > > > > > issues in a more general way than they are handled in FITS. If 
> > > > > > > there
> > > > > > > is interest, perhaps we should say more about the intended 
> > > > > > > approach.
> > > > > > > This is particularly important for replacing spectrographic 
> > > > > > > tools in
> > > > > > > IRAF, and this is where we are starting our effort.
> > > > > > > 
> > > > > > > 4) We need a way of saving these WCS models, and FITS is not the
> > > > > > > way.
> > > > > > > We are looking at an alternate data format, not just for WCS 
> > > > > > > models,
> > > > > > > but for data in general [gasp!]. Work has begun on this too.
> > > > > > > 
> > > > > > > 5) A lot of our recent work has been on pysynphot and ETCs. We
> > > > > > > plan on
> > > > > > > making the computational part of our ETCs a released tool. But 
> > > > > > > I'm
> > > > > > > also wondering if we can generalize the pysynphot spectral models
> > > > > > > for
> > > > > > > more general use in spectral tools.
> > > > > > > 
> > > > > > > 6) We have been working on a framework for making pipelines 
> > > > > > > easier
> > > > > > > to
> > > > > > > build and configure. That won't be ready for at least a few 
> > > > > > > months,
> > > > > > > but could well be of general interest and use.
> > > > > > > 
> > > > > > > But besides these things, I would like to see if we can't begin 
> > > > > > > the
> > > > > > > effort of narrowing some of the underlying libraries being used.
> > > > > > > FITS
> > > > > > > WCS is one obvious area that seems ripe for that.
> > > > > > > 
> > > > > > > But the community ought to identify one or two areas that are 
> > > > > > > of the
> > > > > > > most interest in consolidating (let's start small). What should 
> > > > > > > we
> > > > > > > start with? Focus is important in making any progress in this 
> > > > > > > area.
> > > > > > > 
> > > > > > > Perry
> > > > > > > 
> > > > > > > 
> > > > > > > 
> > > > > > > _______________________________________________
> > > > > > > AstroPy mailing list
> > > > > > > AstroPy at scipy.org (mailto:AstroPy at scipy.org)
> > > > > > > http://mail.scipy.org/mailman/listinfo/astropy
> > > > > 
> > > > > 
> > > > > --
> > > > > James E.H. Turner
> > > > > Gemini Observatory Southern Operations Centre,
> > > > > Casilla 603, Tel. (+56) 51 205609
> > > > > La Serena, Chile. Fax. (+56) 51 205650
> > > > > _______________________________________________
> > > > > AstroPy mailing list
> > > > > AstroPy at scipy.org (mailto:AstroPy at scipy.org)
> > > > > http://mail.scipy.org/mailman/listinfo/astropy
> > > > 
> > > > _______________________________________________
> > > > AstroPy mailing list
> > > > AstroPy at scipy.org (mailto:AstroPy at scipy.org)
> > > > http://mail.scipy.org/mailman/listinfo/astropy
> > > 
> > > 
> > > 
> > > --
> > > Kelle Cruz, PhD ? http://kellecruz.com/
> > > 917.725.1334 ? Hunter ext: 16486 ? AMNH ext: 3404
> > > 
> > > _______________________________________________
> > > AstroPy mailing list
> > > AstroPy at scipy.org (mailto:AstroPy at scipy.org)
> > > http://mail.scipy.org/mailman/listinfo/astropy
> > _______________________________________________
> > AstroPy mailing list
> > AstroPy at scipy.org (mailto:AstroPy at scipy.org)
> > http://mail.scipy.org/mailman/listinfo/astropy
> 
> 
> 
> ------------------------------
> 
> Message: 3
> Date: Mon, 13 Jun 2011 15:23:05 -0400
> From: Kelle Cruz <kellecruz at gmail.com (mailto:kellecruz at gmail.com)>
> Subject: Re: [AstroPy] Proliferating py-astro-libs
> To: Tom Aldcroft <aldcroft at head.cfa.harvard.edu (mailto:aldcroft at head.cfa.harvard.edu)>
> Cc: astropy <astropy at scipy.org (mailto:astropy at scipy.org)>
> Message-ID: <BANLkTinLndKjGCt7OWWXrSuZ+X51t2LiZg at mail.gmail.com (mailto:BANLkTinLndKjGCt7OWWXrSuZ+X51t2LiZg at mail.gmail.com)>
> Content-Type: text/plain; charset="utf-8"
> 
> For those not in the know, like I was, SciPy 2011 is July 11-16 in Austin.
> Registration is $200 for Academics and $150 for students.
> 
> The one thing I can say about this is that it is CRAZY hot in the Texas Hill
> Country right now and it's only going to get worse. (My parents live halfway
> between San Antonio and Austin.) If you decided to go with this option, I
> would likely not attend.
> 
> That said, if there are > 3 ppl on this list already planning on going, then
> you guys should definitely get together to have something a bit more formal
> than just a chat. You could plan the next meeting, evaluate the existing
> packages, start to sketch out a roadmap, maybe even assign some tasks, or
> create some leadership roles (e.g., Lead of Phot packages, Lead of Spectra
> packages, Lead of Xray), etc.
> 
> kelle
> 
> On Mon, Jun 13, 2011 at 3:07 PM, Tom Aldcroft <aldcroft at head.cfa.harvard.edu (mailto:aldcroft at head.cfa.harvard.edu)
> > wrote:
> 
> > What about a splinter meeting at SciPy2011? I guess the question is
> > how many interested parties will NOT be there this year.
> > 
> > - Tom
> > 
> > On Mon, Jun 13, 2011 at 2:59 PM, Kelle Cruz <kellecruz at gmail.com (mailto:kellecruz at gmail.com)> wrote:
> > > I think a sit-down is desperately needed to resolve these issues, figure
> > out
> > > the mgmt structure (aka, pecking order), for the BDFL to emerge, and for
> > > progress to occur.
> > > I'd be happy to participate as a non-python/programming expert and maybe
> > > provide the voice of the "users".
> > > I propose there be a Splinter Meeting at AAS in Austin. (Splinter Meeting
> > > deadline is Dec 1.) Or else someone will have to organize at CfA (Tom A?
> > > Thom R?) or STScI (Marshall? Perry?) since, as far as I can tell, that
> > seems
> > > to be where most of the movers and shakers in this game are located.
> > > kelle
> > > 
> > > On Mon, Jun 13, 2011 at 2:00 PM, Perry Greenfield <perry at stsci.edu (mailto:perry at stsci.edu)>
> > wrote:
> > > > 
> > > > That's a good idea.
> > > > 
> > > > Perry
> > > > 
> > > > On Jun 13, 2011, at 1:43 PM, James Turner wrote:
> > > > 
> > > > > Maybe we should hold an AstroPy conference, where we can discuss
> > > > > co-ordination, get to know each other better and even sit down and
> > > > > work on libraries together (like at SciPy). That might help generate
> > > > > a bit of momentum. Some of us have had meetings before that were
> > > > > full of ideas that didn't go anywhere, but I don't think it has to
> > > > > be that way if active people on the ground are talking to one another
> > > > > rather than having institutions present their plans and try to
> > > > > negotiate at a high level.
> > > > > 
> > > > > 
> > > > > On 13/06/11 13:25, James Turner wrote:
> > > > > > It seems that several of us would really like to improve
> > > > > > collaboration on Python libraries but have been struggling to pull
> > > > > > it off. I've raised the same issue on this list in past months, but
> > > > > > my
> > > > > > focus has unavoidably been on other things and since I'm wary of
> > > > > > shouting a lot without contributing much, I haven't really been able
> > > > > > to keep the discussion alive...
> > > > > > 
> > > > > > I tend to agree with Mark and Stefan about the question of
> > > > > > leadership.
> > > > > > Perry & co. at Space Telescope deserve recognition for getting us
> > > > > > this
> > > > > > far with things like PyFITS and PyRAF. Others have taken the
> > > > > > initiative
> > > > > > with things like astronomical plotting and documentation sprints.
> > > > > > We're
> > > > > > still lacking a bit of coherence though, which (as Mark suggests) is
> > > > > > likely to involve one or a few dedicated, energetic, knowledgeable,
> > > > > > hands-on developer(s) who can glue things together. Those people need
> > > > > > to be employed by someone, though, to ensure stability & continuity
> > > > > > (fortunately there's already a bit of that going on at STScI, eg.
> > > > > > with
> > > > > > Mike and Matplotlib). Personally, I have the motivation but have not
> > > > > > had the time/independence (and might not be assertive enough).
> > > > > > Apparently we do have several energetic authors in the community now
> > > > > > (like Thomas & Eli), but each with their own project.
> > > > > > 
> > > > > > A couple of years ago, a number of us at the observatories
> > > > > > submitted a
> > > > > > white paper to the Decadal Survey, pointing out the need for more
> > > > > > co-ordinated funding so that we can have people who focus on cross-
> > > > > > institutional platform development & support. The report from the
> > > > > > committee did give a nod to our concerns and their importance, but
> > > > > > stopped short of making any recommendation, which basically means
> > > > > > "good
> > > > > > luck with that". Meanwhile, at Gemini we have had our own problems to
> > > > > > deal with, which make it very difficult for me to propose something
> > > > > > internally beyond working with STScI on the distribution of
> > > > > > dependencies that Perry mentioned. Perhaps someone obtaining a grant
> > > > > > for this is not out of the question though.
> > > > > > 
> > > > > > I would like it if we could get together organically behind Astrolib,
> > > > > > but sometimes it's difficult to get people away from their immediate
> > > > > > priorities to focus on that, even within my own institution. If we
> > > > > > could get people dedicated to it, though, it could become
> > > > > > indispensable
> > > > > > enough to attract and co-ordinate more effort. I'm just not sure
> > > > > > how we
> > > > > > get started at this point and my personal options for tackling the
> > > > > > problem seem limited given the overarching funding transition at
> > > > > > Gemini
> > > > > > and the intense focus on projects that are needed to make that
> > > > > > work...
> > > > > > 
> > > > > > Cheers,
> > > > > > 
> > > > > > James.
> > > > > > 
> > > > > > 
> > > > > > On 10/06/11 09:48, Perry Greenfield wrote:
> > > > > > > 
> > > > > > > On Jun 9, 2011, at 11:12 PM, Thomas Robitaille wrote:
> > > > > > > 
> > > > > > > > I just wanted to also add that (in agreement with Marshall) I'm all
> > > > > > > > in favor of many small modules that accomplish a particular task
> > > > > > > > well, rather than packages that aim for a 'do-it-all' approach and
> > > > > > > > fall short. It's always possible to bundle small packages together
> > > > > > > > afterwards, and I don't mean merge development, but instead just
> > > > > > > > bundling the packages for installation (kind of like EPD). I think
> > > > > > > > that is the easiest approach for all of us.
> > > > > > > > 
> > > > > > > > Maybe in the long run, a specific set of core packages will emerge
> > > > > > > > as essential and we can then talk about truly merging them into a
> > > > > > > > scipy-like package, but for now, I think the race is still on. And
> > > > > > > > after all, there's nothing to say we *have* to achieve the same
> > > > > > > > setup as in IDL.
> > > > > > > 
> > > > > > > It sure seems to me that the time is ripe to start trying to
> > > > > > > coalesce
> > > > > > > some of the ongoing efforts.
> > > > > > > 
> > > > > > > Mind you, I don't think it is necessarily good to start with only
> > > > > > > one
> > > > > > > version. Allowing a few different approaches initially has its
> > > > > > > benefits. You get to see more approaches and ideas in play and
> > > > > > > having
> > > > > > > experience with them is very helpful in deciding which one is more
> > > > > > > productive. And sometimes there is room for more than one in the
> > > > > > > long
> > > > > > > run. The different approaches may have their own niches. But it is
> > > > > > > hard to imagine any long-term need for more than two or three
> > > > > > > different approaches.
> > > > > > > 
> > > > > > > Early on there are some pragmatic needs for different approaches.
> > > > > > > For
> > > > > > > example, having a fairly "literal" translation of IDL tools into
> > > > > > > Python has its benefit. It is very useful for those that would
> > > > > > > like to
> > > > > > > migrate IDL code, and given the existing IDL versions, make it much
> > > > > > > easier to test their correctness. But I don't see this as a
> > > > > > > substitute
> > > > > > > for a good set of modular tools that have a better object design and
> > > > > > > consistent interfaces with other modules. Doing this is more work
> > > > > > > and
> > > > > > > will take more time. So a need for both approaches is likely. Some
> > > > > > > could argue the same for replacing IRAF tasks.
> > > > > > > 
> > > > > > > All this is much easier said than done of course.
> > > > > > > 
> > > > > > > I wish STScI had more resources to devote to this than we've
> > > > > > > actually
> > > > > > > had. We've been planning to do more on this front than we've
> > > > > > > actually
> > > > > > > done. Things come up repeatedly that ruin these plans. But it may be
> > > > > > > worth saying where some of our current efforts are going that may
> > > > > > > overlap some of these other efforts.
> > > > > > > 
> > > > > > > 1) We've been planning (along with Gemini, and particularly James
> > > > > > > Turner), to try to develop some Sage-like installation package that
> > > > > > > would make it easy to install all the basic tools for most
> > > > > > > astronomers. We had hoped to have a beta version out, but one of the
> > > > > > > people working on this left at the end of last year, and we've not
> > > > > > > been able to replace that person. We are going to continue this
> > > > > > > effort
> > > > > > > with existing staff though. Hopefully in a few months we'll have
> > > > > > > something to try out.
> > > > > > > 
> > > > > > > 2) There is a recognition that a more serious effort needs to be
> > > > > > > made
> > > > > > > to replace IRAF functionality. Perhaps one of the benefits of a JWST
> > > > > > > delay is that it will allow us to do some of that work more
> > > > > > > explicitly. But we would not do it by replacing IRAF tasks one-for-
> > > > > > > one
> > > > > > > but coming up with an entirely different approach which has to start
> > > > > > > from the bottom up (the end result could have applications that
> > > > > > > mostly
> > > > > > > emulate IRAF tasks, but also provide much more modular tools).
> > > > > > > 
> > > > > > > 3) More specifically, we are currently focussing on how to handle
> > > > > > > WCS
> > > > > > > issues in a more general way than they are handled in FITS. If there
> > > > > > > is interest, perhaps we should say more about the intended approach.
> > > > > > > This is particularly important for replacing spectrographic tools in
> > > > > > > IRAF, and this is where we are starting our effort.
> > > > > > > 
> > > > > > > 4) We need a way of saving these WCS models, and FITS is not the
> > > > > > > way.
> > > > > > > We are looking at an alternate data format, not just for WCS models,
> > > > > > > but for data in general [gasp!]. Work has begun on this too.
> > > > > > > 
> > > > > > > 5) A lot of our recent work has been on pysynphot and ETCs. We
> > > > > > > plan on
> > > > > > > making the computational part of our ETCs a released tool. But I'm
> > > > > > > also wondering if we can generalize the pysynphot spectral models
> > > > > > > for
> > > > > > > more general use in spectral tools.
> > > > > > > 
> > > > > > > 6) We have been working on a framework for making pipelines easier
> > > > > > > to
> > > > > > > build and configure. That won't be ready for at least a few months,
> > > > > > > but could well be of general interest and use.
> > > > > > > 
> > > > > > > But besides these things, I would like to see if we can't begin the
> > > > > > > effort of narrowing some of the underlying libraries being used.
> > > > > > > FITS
> > > > > > > WCS is one obvious area that seems ripe for that.
> > > > > > > 
> > > > > > > But the community ought to identify one or two areas that are of the
> > > > > > > most interest in consolidating (let's start small). What should we
> > > > > > > start with? Focus is important in making any progress in this area.
> > > > > > > 
> > > > > > > Perry
> > > > > > > 
> > > > > > > 
> > > > > > > 
> > > > > > > _______________________________________________
> > > > > > > AstroPy mailing list
> > > > > > > AstroPy at scipy.org (mailto:AstroPy at scipy.org)
> > > > > > > http://mail.scipy.org/mailman/listinfo/astropy
> > > > > 
> > > > > 
> > > > > --
> > > > > James E.H. Turner
> > > > > Gemini Observatory Southern Operations Centre,
> > > > > Casilla 603, Tel. (+56) 51 205609
> > > > > La Serena, Chile. Fax. (+56) 51 205650
> > > > > _______________________________________________
> > > > > AstroPy mailing list
> > > > > AstroPy at scipy.org (mailto:AstroPy at scipy.org)
> > > > > http://mail.scipy.org/mailman/listinfo/astropy
> > > > 
> > > > _______________________________________________
> > > > AstroPy mailing list
> > > > AstroPy at scipy.org (mailto:AstroPy at scipy.org)
> > > > http://mail.scipy.org/mailman/listinfo/astropy
> > > 
> > > 
> > > 
> > > --
> > > Kelle Cruz, PhD ? http://kellecruz.com/
> > > 917.725.1334 ? Hunter ext: 16486 ? AMNH ext: 3404
> > > 
> > > _______________________________________________
> > > AstroPy mailing list
> > > AstroPy at scipy.org (mailto:AstroPy at scipy.org)
> > > http://mail.scipy.org/mailman/listinfo/astropy
> 
> 
> 
> -- 
> Kelle Cruz, PhD ? http://kellecruz.com/
> 917.725.1334 ? Hunter ext: 16486 ? AMNH ext: 3404
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: http://mail.scipy.org/pipermail/astropy/attachments/20110613/ad224ff8/attachment.html 
> 
> ------------------------------
> 
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org (mailto:AstroPy at scipy.org)
> http://mail.scipy.org/mailman/listinfo/astropy
> 
> 
> End of AstroPy Digest, Vol 58, Issue 16
> ***************************************