<html>
<head>
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
[*Snip*]<br>
<br>
<blockquote type="cite">Group-by and related functionality is top on
my list of priorities for astropy.table (in fact I see it every
day on my google keep app...). Join and merging are in master
now. In my tests the astropy table join is within a factor of 2
to 3 in speed relative to pandas, so in most use cases it should
be good enough.</blockquote>
Join and merge would follow the pandas behaviour? Because that is
one of its major assets, I believe - its elegant handling of missing
data and misaligned indices etc.<br>
Speed is a minor issue in my world, we're not often working with the
data set sizes the quants are.<br>
<br>
<blockquote
cite="mid:CAMtEP6ypJOoJoneaX3qeiqDXctVmcpDVmEKx3xD=-+Vm1Bqwgg@mail.gmail.com"
type="cite">
<div dir="ltr">
<div class="gmail_extra">
<div class="gmail_quote">
<div>It's probably worth pointing out to the community that
it was not a lightly-taken decision to reject pandas for
use as the base data storage container. For the case of
tables there is one show-stopper which is that pandas
DataFrame does not support arbitrary multi-dimensional
columns, i.e. column where each element is itself an N-d
array. These occur enough in astronomy and are supported
by FITS and VO standards, so the astropy Table must be
able to represent that. The lack of support for table and
column metadata is a smaller but still important issue.</div>
<div><br>
</div>
</div>
</div>
</div>
</blockquote>
That is very interesting to hear. I had, in fact, been wondering
what was the rationale, but it certainly makes sense.<br>
I was under the impression, though, that pandas pretty much
supported arbitrary objects as entries in its data structures. But I
was apparently mistaken?<br>
<br>
<blockquote
cite="mid:CAMtEP6ypJOoJoneaX3qeiqDXctVmcpDVmEKx3xD=-+Vm1Bqwgg@mail.gmail.com"
type="cite">
<div dir="ltr">
<div class="gmail_extra">
<div class="gmail_quote">
<div>Having said that, there is no question pandas has a ton
of highly-efficient and useful machinery and we are
working on ways to improve inter-operability. This
includes being able convert between Table and DataFrame
easily. Suggestions and (especially) pull requests
welcome.</div>
</div>
</div>
</div>
</blockquote>
<br>
An easy and near-seamless conversion between a DataFrame and an
astropy table (as long as data type support allows for it, of
course) would definitely be a great thing.<br>
<br>
<blockquote
cite="mid:CAMtEP6ypJOoJoneaX3qeiqDXctVmcpDVmEKx3xD=-+Vm1Bqwgg@mail.gmail.com"
type="cite">
<div dir="ltr">
<div class="gmail_extra">
<div class="gmail_quote">
<div> </div>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr">
<div><br>
</div>
<div>I won't speculate about whether that's enough an
asset to warrant a dependency in astropy. I do agree
that lots of other pandas features don't translate as
well into astronomy use.</div>
<div><br>
</div>
</div>
<div class="gmail_extra">
<div>
<div class="h5"><br>
<br>
<div class="gmail_quote">On Thu, Jun 20, 2013 at
12:34 PM, Erik Tollerud <span dir="ltr"><<a
moz-do-not-send="true"
href="mailto:erik.tollerud@gmail.com"
target="_blank">erik.tollerud@gmail.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0
0 .8ex;border-left:1px #ccc
solid;padding-left:1ex">
<div dir="ltr">
<div>I'm of mixed minds about traits UI
because once you know it you can make great
GUIs with it, but I've spent a lot of time
troubleshooting people's python
installations to get traits to work. That
is, in general it can be tricky to get
installed because of all the dependencies.
Maybe this has improved recently with
Enthought's Canopy (or other new python
distros), but that's been my past
experience. </div>
<div><br>
</div>
<div>More generally, the view in the astropy
core package is that we don't want to put
GUIs in the core because GUIs always carry
lots of dependencies, which we don't want to
be forced to deal with. But part of the
whole reason for affiliated packages was to
get around this, so we're happy to see
GUI-based affiliated packages.</div>
<div><br>
</div>
<div><br>
</div>
<div>As for Pandas, to be totally honest, I
don't see a huge amount to be gained from
adding a Pandas dependency Astropy. It's
honestly not clear what it gives the
astronomy community that numpy does not
already have. The following quote from the
Pandas web site has guided me to that
conclusion: "<em
style="line-height:22.390625px;text-align:justify;font-size:medium;font-family:Arial,sans-serif;margin:0px;padding:0px">pandas</em><span
style="line-height:22.390625px;text-align:justify;font-size:medium;font-family:Arial,sans-serif"> helps
fill this gap, enabling you to carry out
your entire data analysis workflow in
Python without having to switch to a more
domain specific language like R."</span></div>
<div><span
style="line-height:22.390625px;text-align:justify;font-size:medium;font-family:Arial,sans-serif"><br>
</span></div>
<div>I have been carrying out my entire data
analysis workflow for some time now in
python without using Pandas. It looks to me
like Pandas is a tool that was written by
and for statisticians who use R. While we
can take lessons from this, it's not clear
we get much out of it in an astronomy
context. For example, how would it make
astropy's NDData, Quantity, or Table better
to use a Pandas DataFrame vs. a numpy array?
Most of what we are doing is
building astronomy-convenient interfaces,
and I'm not sure what Pandas adds there, at
the cost of a pretty heavy-weight
dependency.</div>
<div><br>
</div>
<div>It could just be that I don't know enough
about Pandas, though. So if someone who
knows Pandas better can speak to this, I'm
all ears.</div>
<div> </div>
<div><br>
</div>
</div>
<div class="gmail_extra">
<div>
<div><br>
<br>
<div class="gmail_quote">On Tue, Jun 18,
2013 at 3:35 PM, Thøger Rivera-Thorsen <span
dir="ltr"><<a
moz-do-not-send="true"
href="mailto:trive@astro.su.se"
target="_blank">trive@astro.su.se</a>></span>
wrote:<br>
<blockquote class="gmail_quote"
style="margin:0 0 0
.8ex;border-left:1px #ccc
solid;padding-left:1ex">
<div text="#000000" bgcolor="#FFFFFF">
<div>Pandas is a part of the
newly-defined SciPy stack, after
all, so that would be part of any
science-oriented distribution
worth its salt. In fact, I think
it could be a good idea for
astropy in general to use under
the hood, but again, could clash
with the philosophy of the project
and possibly also
maintainabillity.<br>
<br>
As for offering my code or just my
experience, I'll have to square it
with my supervisor first, and I
also think it depends on what
direction the project in question
will take. I'm positive about the
idea (which is why I wrote in the
first place), but supervisor might
think it is a better idea to
actually get my paper in the
project wrapped up before sending
the code out there. Will get back
about that one!<br>
<br>
/Emil
<div>
<div><br>
<br>
<br>
<br>
<br>
On 2013-06-18 20:53, Slavin,
Jonathan wrote:<br>
</div>
</div>
</div>
<blockquote type="cite">
<div>
<div><font
face="verdana,sans-serif">Hi
Emil,</font>
<div><font
face="verdana,sans-serif"><br>
</font></div>
<div><font
face="verdana,sans-serif">That
looks very nice! I don't
see Pandas as a big issue
in terms of dependencies.
I don't know that much
about traits, etc. My
thought about the gui was
just based on my
experience with
matplotlib, and the fact
that it is widely used --
though I would agree that
too many dependencies can
be a deterrent to people
using something. Are you
offering your code as a
starting point for the
project? It strikes me
that many have gotten some
sort of fitting package to
a point of personal
usability but no one has
the
time/interest/motivation
to make a more generally
usable package.</font></div>
<div><font
face="verdana,sans-serif"><br>
</font></div>
<div><font
face="verdana,sans-serif">Jon</font></div>
<div><font
face="verdana,sans-serif"><br>
</font>
<div class="gmail_quote">On
Tue, Jun 18, 2013 at 2:34
PM, <span dir="ltr"><<a
moz-do-not-send="true"
href="mailto:astropy-request@scipy.org" target="_blank">astropy-request@scipy.org</a>></span>
wrote:<br>
<blockquote
class="gmail_quote"
style="margin:0 0 0
.8ex;border-left:1px
#ccc
solid;padding-left:1ex">
<div
style="overflow:hidden">Date:
<span><span>Tue, 18
Jun 2013 20:39:55</span></span>
+0200<br>
From: Th?ger
Rivera-Thorsen <<a
moz-do-not-send="true" href="mailto:thoger.emil@gmail.com"
target="_blank">thoger.emil@gmail.com</a>><br>
Subject: Re: [AstroPy]
ESA Summer of Code in
Space 2013<br>
To: <a
moz-do-not-send="true"
href="mailto:astropy@scipy.org" target="_blank">astropy@scipy.org</a><br>
Message-ID: <<a
moz-do-not-send="true"
href="mailto:51C0A97B.8090703@gmail.com" target="_blank">51C0A97B.8090703@gmail.com</a>><br>
Content-Type:
text/plain;
charset="iso-8859-1"<br>
<br>
I have been working on
a fitting GUI for a
while, although it is
made<br>
with a specific task
in mind.<br>
However, it is not
based on Matplotlib
but on
Traits/Traitsui/Chaco
and<br>
Pandas. It is made for
a specific projhect
I'm working and as
such not<br>
yet usable for more
general cases, but it
could be a starting
point, if<br>
the dependencies don't
conflict with astropy
politics.<br>
<br>
Especially, I am happy
about the choice of
Pandas for managing a
quite<br>
complex data structure
(the fitted and/or
guessed values of an
arbitrary<br>
number of transitions
for an arbitrary
number of rows or
collapsed rows<br>
of a spatially
resolved spectrum) of
a), but also with the
Traits-based<br>
interactive interface
to build complex line
profiles from single<br>
gaussians, good for
fitting-by-eye and
giving good initial
guesses for<br>
fitting of complex
line profiles. It
hooks directly up to a
wrapper I've<br>
made for lmfit, but
given the modularity,
it should be
relatively easy<br>
to change to other
backends.<br>
<br>
It's still a
work-in-progress, but
there are some
screenshots here:<br>
<a
moz-do-not-send="true"
href="http://flic.kr/s/aHsjGaEMGg" target="_blank">http://flic.kr/s/aHsjGaEMGg</a>
.<br>
I know the choice and
number of dependencies
may be prohibitive but
it<br>
saved a lot of work on
the GUI, and Pandas
means the difference
between<br>
sanity and madness
when it comes to
keeping track of so
many parameters.<br>
<br>
Cheers,<br>
Emil<br>
</div>
</blockquote>
</div>
<br>
<br>
<br clear="all">
<div>
<div dir="ltr">________________________________________________________<br>
Jonathan D.
Slavin
Harvard-Smithsonian CfA<br>
<a
moz-do-not-send="true"
href="mailto:jslavin@cfa.harvard.edu" target="_blank">jslavin@cfa.harvard.edu</a>
60 Garden Street, MS
83<br>
phone: <a
moz-do-not-send="true"
href="tel:%28617%29%20496-7981" value="+16174967981" target="_blank">(617)
496-7981</a>
Cambridge, MA 02138-1516<br>
fax: <a
moz-do-not-send="true"
href="tel:%28617%29%20496-7577" value="+16174967577" target="_blank">(617)
496-7577</a>
USA<br>
________________________________________________________<br>
<br>
</div>
</div>
</div>
<br>
<fieldset></fieldset>
<br>
</div>
</div>
<pre>_______________________________________________
AstroPy mailing list
<a moz-do-not-send="true" href="mailto:AstroPy@scipy.org" target="_blank">AstroPy@scipy.org</a>
<a moz-do-not-send="true" href="http://mail.scipy.org/mailman/listinfo/astropy" target="_blank">http://mail.scipy.org/mailman/listinfo/astropy</a>
</pre>
</blockquote>
<br>
</div>
<br>
_______________________________________________<br>
AstroPy mailing list<br>
<a moz-do-not-send="true"
href="mailto:AstroPy@scipy.org"
target="_blank">AstroPy@scipy.org</a><br>
<a moz-do-not-send="true"
href="http://mail.scipy.org/mailman/listinfo/astropy"
target="_blank">http://mail.scipy.org/mailman/listinfo/astropy</a><br>
<br>
</blockquote>
</div>
<br>
<br clear="all">
<div><br>
</div>
</div>
</div>
<span><font color="#888888">-- <br>
Erik
</font></span></div>
<br>
_______________________________________________<br>
AstroPy mailing list<br>
<a moz-do-not-send="true"
href="mailto:AstroPy@scipy.org"
target="_blank">AstroPy@scipy.org</a><br>
<a moz-do-not-send="true"
href="http://mail.scipy.org/mailman/listinfo/astropy"
target="_blank">http://mail.scipy.org/mailman/listinfo/astropy</a><br>
<br>
</blockquote>
</div>
<br>
<br clear="all">
<div><br>
</div>
-- <br>
</div>
</div>
************************************<span class="HOEnZb"><font
color="#888888"><br>
Chris Beaumont<br>
Graduate Student<br>
Institute for Astronomy<br>
University of Hawaii at Manoa<br>
2680 Woodlawn Drive<br>
Honolulu, HI 96822<br>
<a moz-do-not-send="true"
href="http://www.ifa.hawaii.edu/%7Ebeaumont"
target="_blank">www.ifa.hawaii.edu/~beaumont</a><br>
************************************
</font></span></div>
<br>
_______________________________________________<br>
AstroPy mailing list<br>
<a moz-do-not-send="true" href="mailto:AstroPy@scipy.org">AstroPy@scipy.org</a><br>
<a moz-do-not-send="true"
href="http://mail.scipy.org/mailman/listinfo/astropy"
target="_blank">http://mail.scipy.org/mailman/listinfo/astropy</a><br>
<br>
</blockquote>
</div>
<br>
</div>
</div>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">_______________________________________________
AstroPy mailing list
<a class="moz-txt-link-abbreviated" href="mailto:AstroPy@scipy.org">AstroPy@scipy.org</a>
<a class="moz-txt-link-freetext" href="http://mail.scipy.org/mailman/listinfo/astropy">http://mail.scipy.org/mailman/listinfo/astropy</a>
</pre>
</blockquote>
<br>
</body>
</html>