[AstroPy] Co-ordinating Python astronomy libraries?

Erik Tollerud erik.tollerud at gmail.com
Sun Jul 11 22:09:50 EDT 2010


I think the "astrokits" idea is a good one - (definitely not melding
into scikits, though - that's too big of an umbrella community for me
to be comfortable with).  I'm somewhat concerned about resulting long
import statements("from astrokits.apackage.amodule.submodule import
SomeReallyLongClassOrFunctionName"), but I guess that might be the
necessary price to pay.

I still think, though, that using that same sort of infrastructure,
while relaxing the namespace requirement, is a better solution - that
is, you can have an organizational structure like Thomas suggests
without requiring that the packages start with "astrokits" - just
strongly recommend the guidelines of consistent documentation
(ideally, I think, using sphinx) and consistent use of the standard
setup.py schemes (most important here is consistency in compiling any
Fortran or C codes... or better yet have everyone use Cython for any
interface layers), and you get the same advantages.  As James rightly
pointed out, if you go the astrokits route, you still don't really
solve the installation problem because everything is installed
separately - all that's really important is a central listing of the
"sanctioned" astrokits.

Another thing that would probably be a very good thing to do if
possible would be to allow for a "proprietary time" - that is, allow
people to develop something only they have access to until they
publish the first paper or whatever using/describing that tool, and
then afterwards it goes public as an astrokit for everyone to see.

So in my mind it seems reasonable to set up a shared repository/web
site with the option (but not requirement) of using an "astrokits"
namespace, with the understanding that when possible, the core
"astropy" modules would be used to avoid duplicating effort (and the
code must follow the aforementioned rather loose standards).



On Wed, Jul 7, 2010 at 6:01 PM, Kelle Cruz <kellecruz at gmail.com> wrote:
> Well, Eli and Tom have already done the listing on AstroPython:
> http://www.astropython.org/resources
> or
> http://www.astropython.org/resources?sort=tag

I hadn't realized how complete this listing is - it's a great start,
but the one thing that's missing for what I had in mind is an access
API or rigorous entry standard.  That is, it has to be possible to
write scripts that work like pip to grab download links and use them
to automatically build and install packages.  This isn't at all hard
to do, it just requires either a secondary database with an API, or a
slightly more standardized entry format with source download links
(which don't have to be hosted on the same page), and some
standardized description of the non-python requirements.  With that,
it'd be pretty easy to write something like an "astropip" that could
be used to automate installation/upgrade of any of the listed
packages.

An "astropip" has the advantage of allowing us to just roll our own
sorts of metapackages and get around the setuptools/distribute
shortcomings.  The more I think about it, that might be the way to go.


> - I think a AAS Splinter is a great idea...looks like the deadline isn't
> until Dec 1 for Seattle, but we should get on it since they are assigned on
> a first-come, first-served basis based on room availability.  It might make
> more sense to do Boston in May 2011 because people won't be as busy with
> other meeting things and it should be nice there in spring

I would think Seattle might be better because a lot more people will
be there... and anyway, a dreary Seattle winter day is much more
likely to keep people in a room talking about python libraries, isn't
it?  (Also, for full disclosure, I'm on the west coast, so I
personally would prefer it given that I'm not sure I can make it to
Boston...)



On Thu, Jul 8, 2010 at 11:03 AM, Perry Greenfield <perry at stsci.edu> wrote:
> Well, more the former, but also to enable something along the latter (though
> not necessarily part of a single install). When things are here and there,
> it is more difficult to package those together in an automatic way. Not
> impossible, just more work and more ways for things to go wrong.

You are right about this, but as I've described above, I think with
better consistency in how all the packages are installed, it'll be far
easier to deal with these problems.  And in my mind, the gains are
much greater because it give access to a much larger brain-trust of
people who want a fair amount of freedom (at least initially) in how
they're doing their project.



> But trac is fairly portable since it is widely used. Migrating the info from
> some of the others can be a  more difficult problem (lock-in issues). As for
> svn, git, hg, bazaar issues, nothing is going to satisfy everyone, I agree

The Trac thing is a personal preference - I admit it might well be the
best all-around compromise, although I still think allowing the option
of external hosting might be a good idea, as long as they have
standardized entries in whatever listing is used.  I want to
reiterate, though, that I think it's important to *not* use svn over
the other options I mentioned - distributed version control systems
(e.g. bzr, hg, or git) are far more suited to the kind of development
I think most astronomers typically are used to, as they can make their
own local copy, do their own thing without any access to the central
repository, and later merge it back in.



> That works to a certain level. But before long, there are n flavors of
> representing this and that, and combining 10 packages like this to use in
> your own  code can get to be a real chore. You'll spend a lot of time moving
> from one convention to another (and not catching some bugs). I don't think
> making everyone conform is a good solution, but eventually standardizing on
> core libraries is a very good thing in the long run. I think there is a
> reasonable middle ground on this kind of thing.

You're absolutely right about the problems of fragmentation and the
virtues of later standardization. But right now, given that those core
libraries don't exist, the freedom to experiment is also very
important.  Of course, I'm biased in that I'm working for a rather
more pythonic/object-oriented flavor than the traditional IDL-like
libraries so I'm leveraged a bit towards the experiment side... but I
think the point is still valid, so as you say, a general middle ground
is important.


-- 
Erik Tollerud



More information about the AstroPy mailing list