From mdroe at stsci.edu  Wed Apr  1 08:46:47 2015
From: mdroe at stsci.edu (Michael Droettboom)
Date: Wed, 1 Apr 2015 08:46:47 -0400
Subject: [AstroPy] getting id value from xml files
In-Reply-To: <551ABCA0.50101@stsci.edu>
References: <55149A76.5050504@gmail.com>
	<55155E70.8070904@stsci.edu>	<55186CE0.6070106@gmail.com>
	<551ABCA0.50101@stsci.edu>
Message-ID: <551BE8B7.4030805@stsci.edu>

In the specific case of your file, the INFO elements are all at the top 
level, so they can be obtained through the |.infos| member, e.g.:

|from astropy.io import votable
vot = votable.parse('file.xml')

def find_by_ID(vot, ID):
     for info in vot.infos:
         if info.ID == ID:
             return info.value

find_by_ID(vot, 'sfr')
|

But, as I said, this functionality really should be built-in to astropy, 
and PR https://github.com/astropy/astropy/pull/3633 adds this to master 
(and it?s more general because it will search the entire file, not just 
the INFO elements at the top-level).

Mike

On 03/31/2015 11:26 AM, Michael Droettboom wrote:

> On 03/29/2015 05:21 PM, Grigoris Maravelias wrote:
>> Hi Mike! I just found time to check it...
>>
>> On 03/27/2015 02:43 PM, Michael Droettboom wrote:
>>> The you could do:
>>> |from astropy.io import votable
>>> vot = votable.parse("file.xml")
>>> value = vot.resources[0].infos[0].value|
>> |I tried that but returns an error:
>> |
>> |ERROR: IndexError: list index out of range [__main__]
>> Traceback (most recent call last):
>>   File "./xml_extractor.py", line 21, in <module>
>>     value = vot.resources[0].infos[0].value
>> IndexError: list index out of range|
>
> That was just an example if your file happened to have the same layout 
> as what I suggested -- since I hadn't seen your file, I just had to 
> guess.  Chances are your INFO element in question is in a different 
> location, so you'll need to adjust accordingly.
>
>>> I assume you are asking how to do this with astropy.io.votable?
>>>
>>> INFO tags can appear in a number of places in a VOTable file, so 
>>> without seeing the whole file, it?s hard to say.
>>>
>>> For example, if you had a file like:
>>>
>>> |<VOTABLE>
>>>    <RESOURCE>
>>>      <INFO ID="sfr" name="sfr" value="1.34267351483e-40"/>
>>>    </RESOURCE>
>>> </VOTABLE>
>>> |
>> You are almost correct. I provide an original xml file (see [1]) 
>
> I don't see the target of the citation (maybe just forgot?).
>
> Cheers,
> Mike
>>
>>
>>
>>
>> _______________________________________________
>> AstroPy mailing list
>> AstroPy at scipy.org
>> http://mail.scipy.org/mailman/listinfo/astropy
>
>
>
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org
> http://mail.scipy.org/mailman/listinfo/astropy

?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/astropy/attachments/20150401/e3ff99d2/attachment.html>

From mdroe at stsci.edu  Wed Apr  1 08:51:08 2015
From: mdroe at stsci.edu (Michael Droettboom)
Date: Wed, 1 Apr 2015 08:51:08 -0400
Subject: [AstroPy] Fast ASCII reader with Fortran notation
In-Reply-To: <E201A6C7-0BAA-422B-88E0-7DE45F6D26BF@astro.physik.uni-goettingen.de>
References: <E201A6C7-0BAA-422B-88E0-7DE45F6D26BF@astro.physik.uni-goettingen.de>
Message-ID: <551BE9BC.6090808@stsci.edu>

Derek,

This seems like a great addition.  I'd say go ahead and file a pull 
request from your branch (with your description below in the initial 
comment), as it's probably easiest for us to comment on it in that 
interface.  It's not unusual for some back-and-forth on pull requests, 
so it doesn't need to be "perfect" when you create it.

Mike

On 03/31/2015 05:33 PM, Derek Homeier wrote:
> Hi all,
>
> first thanks to everyone who contributed to the fast C parser framework in 1.0?s io.ascii,
> having decent speed when dealing with cumbersome text files has been on my wish list
> for some time!
> My next highest priority was always being able to read in Fortran-written ascii files with
> its standard double-precision notation (1.495978707D+13?) without further processing,
> which required setting up converters for each column with np.loadtxt or np.genfromtxt.
> I decided to take a shot at the fast_reader module to enable the c parser to directly
> recognise the ?D? as exponential character, a branch from the stable tree is here
>
> https://github.com/dhomeier/astropy/tree/fortranio
>
> which adds ?fortran_dexp? as new option to the fast_reader arguments.
> Is there general interest for a pull request against stable or master? I?d also like to put
> some questions on refinements for discussion:
>
> 1. This patch automatically activates the fast_converter, as changing the call to xstrtod was
> by far the simplest implementation. I guess it would be possible to add it to the more
> precise parser as well, but don?t know how big the demand for this is. I have yet to see
> a case where use_fast_converter gave a different result.
>
> 2. Currently ?fortran_dexp?=True changes the recognised exponential character to ?d? or ?D? only,
> it?s easy to add any other character or an ?automatic? option that will both recognise ?e? and ?d?
> in the same file, at a seemingly small performance penalty of 5-10 %.
>
> If there is interest I could definitely add the second option.
>
> Cheers,
> 					Derek
>
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org
> http://mail.scipy.org/mailman/listinfo/astropy



From gr.maravelias at gmail.com  Wed Apr  1 09:37:58 2015
From: gr.maravelias at gmail.com (Grigoris Maravelias)
Date: Wed, 01 Apr 2015 15:37:58 +0200
Subject: [AstroPy] getting id value from xml files
In-Reply-To: <551BE8B7.4030805@stsci.edu>
References: <55149A76.5050504@gmail.com>	<55155E70.8070904@stsci.edu>	<55186CE0.6070106@gmail.com>	<551ABCA0.50101@stsci.edu>
	<551BE8B7.4030805@stsci.edu>
Message-ID: <551BF4B6.6050009@gmail.com>

Thanks Mike! That works great.
Thanks also for the pull request.

Best
Grigoris
 
On 04/01/2015 02:46 PM, Michael Droettboom wrote:
>
> In the specific case of your file, the INFO elements are all at the
> top level, so they can be obtained through the |.infos| member, e.g.:
>
> |from astropy.io import votable
> vot = votable.parse('file.xml')
>
> def find_by_ID(vot, ID):
>     for info in vot.infos:
>         if info.ID == ID:
>             return info.value
>
> find_by_ID(vot, 'sfr')
> |
>
> But, as I said, this functionality really should be built-in to
> astropy, and PR https://github.com/astropy/astropy/pull/3633 adds this
> to master (and it?s more general because it will search the entire
> file, not just the INFO elements at the top-level).
>
> Mike
>
> On 03/31/2015 11:26 AM, Michael Droettboom wrote:
>
>> On 03/29/2015 05:21 PM, Grigoris Maravelias wrote:
>>> Hi Mike! I just found time to check it...
>>>
>>> On 03/27/2015 02:43 PM, Michael Droettboom wrote:
>>>> The you could do:
>>>> |from astropy.io import votable
>>>> vot = votable.parse("file.xml")
>>>> value = vot.resources[0].infos[0].value|
>>> |I tried that but returns an error:
>>> |
>>> |ERROR: IndexError: list index out of range [__main__]
>>> Traceback (most recent call last):
>>>   File "./xml_extractor.py", line 21, in <module>
>>>     value = vot.resources[0].infos[0].value
>>> IndexError: list index out of range|
>>
>> That was just an example if your file happened to have the same
>> layout as what I suggested -- since I hadn't seen your file, I just
>> had to guess.  Chances are your INFO element in question is in a
>> different location, so you'll need to adjust accordingly.
>>
>>>> I assume you are asking how to do this with astropy.io.votable?
>>>>
>>>> INFO tags can appear in a number of places in a VOTable file, so
>>>> without seeing the whole file, it?s hard to say.
>>>>
>>>> For example, if you had a file like:
>>>>
>>>> |<VOTABLE>
>>>>   <RESOURCE>
>>>>     <INFO ID="sfr" name="sfr" value="1.34267351483e-40"/>
>>>>   </RESOURCE>
>>>> </VOTABLE>
>>>> |
>>> You are almost correct. I provide an original xml file (see [1]) 
>>
>> I don't see the target of the citation (maybe just forgot?). 
>>
>> Cheers,
>> Mike
>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> AstroPy mailing list
>>> AstroPy at scipy.org
>>> http://mail.scipy.org/mailman/listinfo/astropy
>>
>>
>>
>> _______________________________________________
>> AstroPy mailing list
>> AstroPy at scipy.org
>> http://mail.scipy.org/mailman/listinfo/astropy
> ?
>
>
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org
> http://mail.scipy.org/mailman/listinfo/astropy

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/astropy/attachments/20150401/5a586b7a/attachment.html>

From derek at astro.physik.uni-goettingen.de  Wed Apr  1 10:59:11 2015
From: derek at astro.physik.uni-goettingen.de (Derek Homeier)
Date: Wed, 1 Apr 2015 16:59:11 +0200
Subject: [AstroPy] Fast ASCII reader with Fortran notation
In-Reply-To: <551BE9BC.6090808@stsci.edu>
References: <E201A6C7-0BAA-422B-88E0-7DE45F6D26BF@astro.physik.uni-goettingen.de>
	<551BE9BC.6090808@stsci.edu>
Message-ID: <28FB5A6C-51AF-4553-BCFF-5369C6E3AB71@astro.physik.uni-goettingen.de>

Mike,

> This seems like a great addition.  I'd say go ahead and file a pull 
> request from your branch (with your description below in the initial 
> comment), as it's probably easiest for us to comment on it in that 
> interface.  It's not unusual for some back-and-forth on pull requests, 
> so it doesn't need to be "perfect" when you create it.

done (#3655) with some follow-up described there. Yes, some points
remain open for discussion re. performance vs. versatility.
I forked from master now since I could not get cython to rebuild cparser.c
in the stable tree, and switching back and forth from the last tagged release
turned out to be really messy.

Cheers,
					Derek



From mhabibi at mpe.mpg.de  Thu Apr  2 10:22:53 2015
From: mhabibi at mpe.mpg.de (mhabibi at mpe.mpg.de)
Date: Thu, 2 Apr 2015 16:22:53 +0200
Subject: [AstroPy] a question  about python3-astropy installation
Message-ID: <0fcda67b84ccb87392565640105bc402.squirrel@mpemail.mpe.mpg.de>

Hi all,

I try to install latest version of python3-astropy.

When I install it with the pip command on my Ubuntu 14.04 machine it
installs the 0.3 version. Can anyone please help me how can I install a
newer version of python3-astropy?

Many thanks,
Maryam





From dklaes at astro.uni-bonn.de  Thu Apr  2 10:25:02 2015
From: dklaes at astro.uni-bonn.de (Dominik Klaes)
Date: Thu, 02 Apr 2015 16:25:02 +0200
Subject: [AstroPy] a question  about python3-astropy installation
In-Reply-To: <0fcda67b84ccb87392565640105bc402.squirrel@mpemail.mpe.mpg.de>
References: <0fcda67b84ccb87392565640105bc402.squirrel@mpemail.mpe.mpg.de>
Message-ID: <551D513E.2030600@astro.uni-bonn.de>

Hi Maryam,

I suggest you to install and use the Anaconda package. With that, it's
quite easy to install older and newer version with a simple command line.

Cheers,

Dominik

Am 04/02/2015 um 04:22 PM schrieb mhabibi at mpe.mpg.de:
> Hi all,
> 
> I try to install latest version of python3-astropy.
> 
> When I install it with the pip command on my Ubuntu 14.04 machine it
> installs the 0.3 version. Can anyone please help me how can I install a
> newer version of python3-astropy?
> 
> Many thanks,
> Maryam
> 
> 
> 
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org
> http://mail.scipy.org/mailman/listinfo/astropy
> 

-- 
Dominik Klaes
Argelander-Institut f?r Astronomie
Room 2.027a
Auf dem H?gel 71
53121 Bonn

Telefon: 0228/73-5773
E-Mail: dklaes at astro.uni-bonn.de
Homepage: https://www.astro.uni-bonn.de/m/dklaes/


From evert.rol at gmail.com  Thu Apr  2 10:44:36 2015
From: evert.rol at gmail.com (Evert Rol)
Date: Thu, 2 Apr 2015 16:44:36 +0200
Subject: [AstroPy] a question  about python3-astropy installation
In-Reply-To: <0fcda67b84ccb87392565640105bc402.squirrel@mpemail.mpe.mpg.de>
References: <0fcda67b84ccb87392565640105bc402.squirrel@mpemail.mpe.mpg.de>
Message-ID: <29ECE0DA-FA7F-440B-9021-0970987BEC47@gmail.com>

  Hi Maryam,


> I try to install latest version of python3-astropy.
> 
> When I install it with the pip command on my Ubuntu 14.04 machine it
> installs the 0.3 version. Can anyone please help me how can I install a
> newer version of python3-astropy?

Firstly, try updating pip; it may use an old database (not sure if it uses one online or locally):
 
  > pip3 update

You could also explicitly try and state the version:

> pip3 install astropy==1.0.1

(note: double =-sign)

If you want the most recent development version from github, you could try:

> pip3 install git+https://github.com/astropy/astropy


I notice that you mention "python3-astropy"; that sounds more like an Ubuntu package than something pip will use (to pip, it's just called "astropy"); if you're instead trying to install that Ubuntu package, then it's more likely it's using an older version of astropy (essentially the one that was active when the packages for your Ubuntu version were finalized).


Cheers,

  Evert


> 
> Many thanks,
> Maryam
> 
> 
> 
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org
> http://mail.scipy.org/mailman/listinfo/astropy



From pweilbacher at aip.de  Thu Apr  2 12:11:40 2015
From: pweilbacher at aip.de (Peter Weilbacher)
Date: Thu, 2 Apr 2015 18:11:40 +0200 (CEST)
Subject: [AstroPy] DS9 color tables in Python
In-Reply-To: <alpine.LSU.2.00.1503311345480.1856@durham.aip.de>
References: <alpine.LSU.2.00.1503311345480.1856@durham.aip.de>
Message-ID: <alpine.LSU.2.00.1504021749260.27139@durham.aip.de>

Thanks, everybody, for the comments in this thread. It would be great to
get these color tables into astropy.

In the short term, the hint that I can get very similar versions of all
these tables from CIAO has given me exactly what I need. I just did

    import numpy
    import matplotlib.colors as colors
    import matplotlib.cm as cm
    import aplpy

    hsv = numpy.loadtxt('hsv.lut')
    ciao_hsv = colors.ListedColormap(hsv, 'ciao_hsv')
    cm.register_cmap(cmap=ciao_hsv)

    halpha = aplpy.FITSFigure('2015_data/halpha.fits', 3)
    halpha.show_colorscale(cmap='ciao_hsv')
    halpha.show_colorbar()
    halpha.save('halpha_hsv_ciao.png')

and got a very nice plot -- exactly what I need!

The tables from CIAO are numerically a bit different from the ones I
managed to export from DS9 as three-column tables (like hsv). But in
practise, I can hardly notice the difference in the plots.

Cheers,
   Peter.
-- 
Dr Peter Weilbacher                 http://www.aip.de/People/PWeilbacher
Phone +49 331 74 99-667                       encryption key ID 7D6B4AA0
------------------------------------------------------------------------
Leibniz-Institut f?r Astrophysik Potsdam (AIP)
An der Sternwarte 16, D-14482 Potsdam

Vorstand: Prof. Dr. Matthias Steinmetz
Stiftung b?rgerlichen Rechts, Stiftungsverz. Brandenburg: 26 742-00/7026


From amara at konteur.com  Thu Apr  2 14:47:43 2015
From: amara at konteur.com (Amara Graps)
Date: Thu, 02 Apr 2015 21:47:43 +0300
Subject: [AstroPy] Interested in a wavelet toolbox?
Message-ID: <551D8ECF.4050508@konteur.com>

Greetings astropy-ers,

I'm proposing the development of a wavelet library in Python to a
(European Commission) funding agency and I'll need beta testing
of some parts. I think it would be especially great to have the
library be incorporated into AstroPy after it's written and tested
because the funds for maintenance would end when the project ends..
yet astronomers continue to love wavelets.

Some large part of my Python wavelet library would be built upon this
well-established (general purpose) Matlab toolbox:

http://statweb.stanford.edu/~wavelab/Wavelab_850/index_wavelab850.html

Does this sound useful and interesting to this group? If it does, please
reply to graps at psi.edu. I'm gathering documentation of the AstroPy user
community's interest and my testing and maintenance methodology in order
to help the project get funded.

Thanks very much in advance!

Sincerely,
Amara

Amara Graps, PhD
Lead Researcher,  University of Latvia
Astronomijas instit?ts, Rai?a bulv?ris 19, R?ga, LV-1050, Latvia
amara at lu.lv  +371 / 28853907
and
Senior Scientist, Planetary Science Institute (PSI, USA),
Aleksandra Caka iela, 96-31,
Riga, Latvia LV-1011
http://www.psi.edu/about/staff/graps/graps.html
graps at psi.edu  +1 / 644-6250


From aldcroft at head.cfa.harvard.edu  Thu Apr  2 15:43:47 2015
From: aldcroft at head.cfa.harvard.edu (Aldcroft, Thomas)
Date: Thu, 2 Apr 2015 15:43:47 -0400
Subject: [AstroPy] Interested in a wavelet toolbox?
In-Reply-To: <551D8ECF.4050508@konteur.com>
References: <551D8ECF.4050508@konteur.com>
Message-ID: <CAMtEP6x=odiy0R=fnC9CnibuKnu4cq--_V3spzXmR0ERSsr+ow@mail.gmail.com>

Hi Amara,

Though I'm not officially speaking for the Astropy project, I would say
that a wavelets package like you are proposing would be a highly valued
addition to the Astropy analysis environment.  From personal experience the
primary source detection algorithm for Chandra X-ray sources is based on
wavelets and this has been quite successful.

The normal starting point for a project like this would be as an Astropy
affiliated package [1, 2].  An affiliated package is an astronomy-related
Python package that is not part of the astropy core package, but has
requested to be included as part of the Astropy project?s community. These
packages are expressing an interest in Astropy?s goals of improving reuse,
interoperability, and interface standards for python astronomy and
astrophysics packages.  As a package matures it could be considered for
inclusion in the core astropy package.

The question of continued maintenance after the primary development funding
dries up also points to becoming part of the Astropy and open-source
development community from the start.  A key element here is doing
development on Github right away rather than waiting until it is fully
written and tested.  While there are no guarantees, if a project is useful
enough to engage other members of community then that provides hope for
on-going maintenance.

Cheers,
Tom

[1]: http://docs.astropy.org/en/stable/overview.html#affiliated-packages
[2]: http://www.astropy.org/affiliated/index.html


On Thu, Apr 2, 2015 at 2:47 PM, Amara Graps <amara at konteur.com> wrote:

> Greetings astropy-ers,
>
> I'm proposing the development of a wavelet library in Python to a
> (European Commission) funding agency and I'll need beta testing
> of some parts. I think it would be especially great to have the
> library be incorporated into AstroPy after it's written and tested
> because the funds for maintenance would end when the project ends..
> yet astronomers continue to love wavelets.
>
> Some large part of my Python wavelet library would be built upon this
> well-established (general purpose) Matlab toolbox:
>
> http://statweb.stanford.edu/~wavelab/Wavelab_850/index_wavelab850.html
>
> Does this sound useful and interesting to this group? If it does, please
> reply to graps at psi.edu. I'm gathering documentation of the AstroPy user
> community's interest and my testing and maintenance methodology in order
> to help the project get funded.
>
> Thanks very much in advance!
>
> Sincerely,
> Amara
>
> Amara Graps, PhD
> Lead Researcher,  University of Latvia
> Astronomijas instit?ts, Rai?a bulv?ris 19, R?ga, LV-1050, Latvia
> amara at lu.lv  +371 / 28853907
> and
> Senior Scientist, Planetary Science Institute (PSI, USA),
> Aleksandra Caka iela, 96-31,
> Riga, Latvia LV-1011
> http://www.psi.edu/about/staff/graps/graps.html
> graps at psi.edu  +1 / 644-6250
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org
> http://mail.scipy.org/mailman/listinfo/astropy
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/astropy/attachments/20150402/6e40d281/attachment.html>

From ricky.egeland at gmail.com  Thu Apr  2 17:40:41 2015
From: ricky.egeland at gmail.com (Ricky Egeland)
Date: Thu, 2 Apr 2015 15:40:41 -0600
Subject: [AstroPy] Interested in a wavelet toolbox?
In-Reply-To: <551D8ECF.4050508@konteur.com>
References: <551D8ECF.4050508@konteur.com>
Message-ID: <493058D7-1419-4836-AABC-A596B140F8E4@gmail.com>

I was searching for a well-tested wavelet library in Python a few months ago, and did not find anything fully satisfactory.  I wanted a continuous wavelet transform using the Morlet mother wavelet, with significance contours.  I feel like such a package belongs in scipy more than it does astropy, as wavelets are a general signal processing technique not restricted to astronomy.  There are a few wavelet-related functions in scipy.signal already, but they do not work together (e.g. scipy.signal.cwt is unrelated to scipy.signal.morlet) and it is not clear how they are intended to be used.  I have seen scipy discussions that propose throwing all that away and starting over.

In case they are useful, here are my notes from the search:

===
 - scipy.signal.wavelet : cwt(), only Mexican hat wavelet?  Poor documentation

 - kPyWavelet : https://github.com/Cadair/kPyWavelet/
   Linked from http://ocgweb.marine.usf.edu/~liu/wavelet.html
   Few users... real interest?

 - A Practical Guide to Wavelet Analysis
   http://paos.colorado.edu/research/wavelets/
   Highly cited ... no python

 - pycwt
   https://github.com/regeirk/pycwt
   Explicitly based on T&C code above
   # Tested.  Example code has a few minor bugs, but appears to be doing the right thing

 - PyCWT
   https://github.com/Unidata/pyCWT
   Simple, 1 file
   Based on T&C
   # Tested. No example code, but docstrings have examples (that don't work as-is)  Source code is cleaner than pycwt.  scaleogram() has some issues.  Need to input scales, not frequencies...

 - ObsPy
   http://docs.obspy.org/master/tutorial/code_snippets/continuous_wavelet_transform.html
   
 - PyWavelets
   http://www.pybytes.com/pywavelets/
   Part of MacPorts
   Good documentation effort
   Only one developer, but 5+ years effort

 - https://github.com/aaren/wavelets

 - https://pypi.python.org/pypi/waipy
   Based on T&C code

 - scipy wavelet discussions: https://github.com/scipy/scipy/issues?utf8=?&q=is%3Aissue+is%3Aopen+wavelet
===

In the end I forked Unidata/pyCWT, fixed a few things and used that (https://github.com/rickyegeland/pyCWT).  On a second look I decided regeirk/pycwt might have been the better implementation, but the interface and documentation were not the best for getting started.

Hope this helps.

Regards,
Ricky Egeland


On Apr 2, 2015, at 12:47 PM, Amara Graps <amara at konteur.com> wrote:

> Greetings astropy-ers,
> 
> I'm proposing the development of a wavelet library in Python to a
> (European Commission) funding agency and I'll need beta testing
> of some parts. I think it would be especially great to have the
> library be incorporated into AstroPy after it's written and tested
> because the funds for maintenance would end when the project ends..
> yet astronomers continue to love wavelets.
> 
> Some large part of my Python wavelet library would be built upon this
> well-established (general purpose) Matlab toolbox:
> 
> http://statweb.stanford.edu/~wavelab/Wavelab_850/index_wavelab850.html
> 
> Does this sound useful and interesting to this group? If it does, please
> reply to graps at psi.edu. I'm gathering documentation of the AstroPy user
> community's interest and my testing and maintenance methodology in order
> to help the project get funded.
> 
> Thanks very much in advance!
> 
> Sincerely,
> Amara
> 
> Amara Graps, PhD
> Lead Researcher,  University of Latvia
> Astronomijas instit?ts, Rai?a bulv?ris 19, R?ga, LV-1050, Latvia
> amara at lu.lv  +371 / 28853907
> and
> Senior Scientist, Planetary Science Institute (PSI, USA),
> Aleksandra Caka iela, 96-31,
> Riga, Latvia LV-1011
> http://www.psi.edu/about/staff/graps/graps.html
> graps at psi.edu  +1 / 644-6250
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org
> http://mail.scipy.org/mailman/listinfo/astropy



From deil.christoph at googlemail.com  Thu Apr  2 18:33:38 2015
From: deil.christoph at googlemail.com (Christoph Deil)
Date: Fri, 3 Apr 2015 00:33:38 +0200
Subject: [AstroPy] Interested in a wavelet toolbox?
In-Reply-To: <493058D7-1419-4836-AABC-A596B140F8E4@gmail.com>
References: <551D8ECF.4050508@konteur.com>
	<493058D7-1419-4836-AABC-A596B140F8E4@gmail.com>
Message-ID: <D0473F1E-51E6-41CC-BC96-7A5F801CC045@gmail.com>


> On 02 Apr 2015, at 23:40, Ricky Egeland <ricky.egeland at gmail.com> wrote:
> 
> I was searching for a well-tested wavelet library in Python a few months ago, and did not find anything fully satisfactory.  I wanted a continuous wavelet transform using the Morlet mother wavelet, with significance contours.  I feel like such a package belongs in scipy more than it does astropy, as wavelets are a general signal processing technique not restricted to astronomy.

Hi Amara,

I agree with Ricky here ? a good wavelets toolbox belongs in Scipy because it?s useful to many scientist and engineers.
If you can contribute there, here?s a starting point: https://github.com/scipy/scipy/issues/3931 <https://github.com/scipy/scipy/issues/3931> (and I?ve put Ralf Gommers as lead on this effort in CC).

Then there?s probably going to be astronomy-specific wavelets and wavelet methods and these could go in Astropy or an Astropy-affiliated package.
But I don?t think we should duplicate the functionality that?s already implemented in the packages listed below and that will hopefully soon become available in Scipy.

Christoph

>  There are a few wavelet-related functions in scipy.signal already, but they do not work together (e.g. scipy.signal.cwt is unrelated to scipy.signal.morlet) and it is not clear how they are intended to be used.  I have seen scipy discussions that propose throwing all that away and starting over.
> 
> In case they are useful, here are my notes from the search:
> 
> ===
> - scipy.signal.wavelet : cwt(), only Mexican hat wavelet?  Poor documentation
> 
> - kPyWavelet : https://github.com/Cadair/kPyWavelet/
>   Linked from http://ocgweb.marine.usf.edu/~liu/wavelet.html
>   Few users... real interest?
> 
> - A Practical Guide to Wavelet Analysis
>   http://paos.colorado.edu/research/wavelets/
>   Highly cited ... no python
> 
> - pycwt
>   https://github.com/regeirk/pycwt
>   Explicitly based on T&C code above
>   # Tested.  Example code has a few minor bugs, but appears to be doing the right thing
> 
> - PyCWT
>   https://github.com/Unidata/pyCWT
>   Simple, 1 file
>   Based on T&C
>   # Tested. No example code, but docstrings have examples (that don't work as-is)  Source code is cleaner than pycwt.  scaleogram() has some issues.  Need to input scales, not frequencies...
> 
> - ObsPy
>   http://docs.obspy.org/master/tutorial/code_snippets/continuous_wavelet_transform.html
> 
> - PyWavelets
>   http://www.pybytes.com/pywavelets/
>   Part of MacPorts
>   Good documentation effort
>   Only one developer, but 5+ years effort
> 
> - https://github.com/aaren/wavelets
> 
> - https://pypi.python.org/pypi/waipy
>   Based on T&C code
> 
> - scipy wavelet discussions: https://github.com/scipy/scipy/issues?utf8=?&q=is%3Aissue+is%3Aopen+wavelet
> ===
> 
> In the end I forked Unidata/pyCWT, fixed a few things and used that (https://github.com/rickyegeland/pyCWT).  On a second look I decided regeirk/pycwt might have been the better implementation, but the interface and documentation were not the best for getting started.
> 
> Hope this helps.
> 
> Regards,
> Ricky Egeland
> 
> 
> On Apr 2, 2015, at 12:47 PM, Amara Graps <amara at konteur.com> wrote:
> 
>> Greetings astropy-ers,
>> 
>> I'm proposing the development of a wavelet library in Python to a
>> (European Commission) funding agency and I'll need beta testing
>> of some parts. I think it would be especially great to have the
>> library be incorporated into AstroPy after it's written and tested
>> because the funds for maintenance would end when the project ends..
>> yet astronomers continue to love wavelets.
>> 
>> Some large part of my Python wavelet library would be built upon this
>> well-established (general purpose) Matlab toolbox:
>> 
>> http://statweb.stanford.edu/~wavelab/Wavelab_850/index_wavelab850.html
>> 
>> Does this sound useful and interesting to this group? If it does, please
>> reply to graps at psi.edu. I'm gathering documentation of the AstroPy user
>> community's interest and my testing and maintenance methodology in order
>> to help the project get funded.
>> 
>> Thanks very much in advance!
>> 
>> Sincerely,
>> Amara
>> 
>> Amara Graps, PhD
>> Lead Researcher,  University of Latvia
>> Astronomijas instit?ts, Rai?a bulv?ris 19, R?ga, LV-1050, Latvia
>> amara at lu.lv  +371 / 28853907
>> and
>> Senior Scientist, Planetary Science Institute (PSI, USA),
>> Aleksandra Caka iela, 96-31,
>> Riga, Latvia LV-1011
>> http://www.psi.edu/about/staff/graps/graps.html
>> graps at psi.edu  +1 / 644-6250
>> _______________________________________________
>> AstroPy mailing list
>> AstroPy at scipy.org
>> http://mail.scipy.org/mailman/listinfo/astropy
> 
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org
> http://mail.scipy.org/mailman/listinfo/astropy

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/astropy/attachments/20150403/e244b565/attachment.html>

From jturner at gemini.edu  Thu Apr  2 19:03:47 2015
From: jturner at gemini.edu (James Turner)
Date: Thu, 2 Apr 2015 20:03:47 -0300
Subject: [AstroPy] Interested in a wavelet toolbox?
In-Reply-To: <D0473F1E-51E6-41CC-BC96-7A5F801CC045@gmail.com>
References: <551D8ECF.4050508@konteur.com>	<493058D7-1419-4836-AABC-A596B140F8E4@gmail.com>
	<D0473F1E-51E6-41CC-BC96-7A5F801CC045@gmail.com>
Message-ID: <551DCAD3.9040003@gemini.edu>

> I agree with Ricky here ? a good wavelets toolbox belongs in Scipy because it?s
> useful to many scientist and engineers.

Should it be a scikit? Last I heard there was quite a lot of
resistance to adding new stuff to the SciPy core and the
expectation was that code in development would at least start
off as a scikit (but I haven't really been following the list
of late).

   https://scikits.appspot.com/about
   https://scikits.appspot.com/scikits

It seems like the only requirements are to name it something
like scikit-wavelets, put it on PyPI and use an OSI-approved
licence, so a package could *probably* be both a scikit and an
AstroPy affiliated package at the same time (whether that
makes sense if it eventually gets folded into scipy is another
matter)...

There's also bound to be some interest in this from the rather
active scikit-image community, who could probably help with
maintenance or even incorporate it there. There was some
discussion here recently about sharing code between AstroPy &
scikit-image but I can't remember the conclusion regarding
what should depend on what.

It definitely seems like a good thing to have available.

Cheers,

James.


>>  There are a few wavelet-related functions in scipy.signal already, but they
>> do not work together (e.g. scipy.signal.cwt is unrelated to
>> scipy.signal.morlet) and it is not clear how they are intended to be used.  I
>> have seen scipy discussions that propose throwing all that away and starting over.
>>
>> In case they are useful, here are my notes from the search:
>>
>> ===
>> - scipy.signal.wavelet : cwt(), only Mexican hat wavelet?  Poor documentation
>>
>> - kPyWavelet : https://github.com/Cadair/kPyWavelet/
>>   Linked from http://ocgweb.marine.usf.edu/~liu/wavelet.html
>>   Few users... real interest?
>>
>> - A Practical Guide to Wavelet Analysis
>> http://paos.colorado.edu/research/wavelets/
>>   Highly cited ... no python
>>
>> - pycwt
>> https://github.com/regeirk/pycwt
>>   Explicitly based on T&C code above
>>   # Tested.  Example code has a few minor bugs, but appears to be doing the
>> right thing
>>
>> - PyCWT
>> https://github.com/Unidata/pyCWT
>>   Simple, 1 file
>>   Based on T&C
>>   # Tested. No example code, but docstrings have examples (that don't work
>> as-is)  Source code is cleaner than pycwt.  scaleogram() has some issues.
>>  Need to input scales, not frequencies...
>>
>> - ObsPy
>> http://docs.obspy.org/master/tutorial/code_snippets/continuous_wavelet_transform.html
>>
>> - PyWavelets
>> http://www.pybytes.com/pywavelets/
>>   Part of MacPorts
>>   Good documentation effort
>>   Only one developer, but 5+ years effort
>>
>> - https://github.com/aaren/wavelets
>>
>> - https://pypi.python.org/pypi/waipy
>>   Based on T&C code
>>
>> - scipy wavelet discussions:
>> https://github.com/scipy/scipy/issues?utf8=?&q=is%3Aissue+is%3Aopen+wavelet
>> ===
>>
>> In the end I forked Unidata/pyCWT, fixed a few things and used that
>> (https://github.com/rickyegeland/pyCWT).  On a second look I decided
>> regeirk/pycwt might have been the better implementation, but the interface and
>> documentation were not the best for getting started.
>>
>> Hope this helps.
>>
>> Regards,
>> Ricky Egeland
>>
>>
>> On Apr 2, 2015, at 12:47 PM, Amara Graps <amara at konteur.com
>> <mailto:amara at konteur.com>> wrote:
>>
>>> Greetings astropy-ers,
>>>
>>> I'm proposing the development of a wavelet library in Python to a
>>> (European Commission) funding agency and I'll need beta testing
>>> of some parts. I think it would be especially great to have the
>>> library be incorporated into AstroPy after it's written and tested
>>> because the funds for maintenance would end when the project ends..
>>> yet astronomers continue to love wavelets.
>>>
>>> Some large part of my Python wavelet library would be built upon this
>>> well-established (general purpose) Matlab toolbox:
>>>
>>> http://statweb.stanford.edu/~wavelab/Wavelab_850/index_wavelab850.html
>>>
>>> Does this sound useful and interesting to this group? If it does, please
>>> reply to graps at psi.edu. I'm gathering documentation of the AstroPy user
>>> community's interest and my testing and maintenance methodology in order
>>> to help the project get funded.
>>>
>>> Thanks very much in advance!
>>>
>>> Sincerely,
>>> Amara
>>>
>>> Amara Graps, PhD
>>> Lead Researcher,  University of Latvia
>>> Astronomijas instit?ts, Rai?a bulv?ris 19, R?ga, LV-1050, Latvia
>>> amara at lu.lv  +371 / 28853907
>>> and
>>> Senior Scientist, Planetary Science Institute (PSI, USA),
>>> Aleksandra Caka iela, 96-31,
>>> Riga, Latvia LV-1011
>>> http://www.psi.edu/about/staff/graps/graps.html
>>> graps at psi.edu  +1 / 644-6250
>>> _______________________________________________
>>> AstroPy mailing list
>>> AstroPy at scipy.org
>>> http://mail.scipy.org/mailman/listinfo/astropy
>>
>> _______________________________________________
>> AstroPy mailing list
>> AstroPy at scipy.org <mailto:AstroPy at scipy.org>
>> http://mail.scipy.org/mailman/listinfo/astropy
>
>
>
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org
> http://mail.scipy.org/mailman/listinfo/astropy


From thomas.robitaille at gmail.com  Fri Apr  3 13:48:23 2015
From: thomas.robitaille at gmail.com (Thomas Robitaille)
Date: Fri, 3 Apr 2015 19:48:23 +0200
Subject: [AstroPy] Fwd: Last call for Talk Submissions (Extended to 4/10),
	Keynotes Announced
In-Reply-To: <946.1428082294.237729825551ece76b2d2c7.52451822@etouches.com>
References: <946.1428082294.237729825551ece76b2d2c7.52451822@etouches.com>
Message-ID: <CAGMHX_3EX2u1C4P6hm=NOuz1Ar9BB9qx=2M17ou_ZOzHn5nL5w@mail.gmail.com>

Hi everyone,

Just a reminder that the deadline for submitting talks for SciPy 2015 is
April 10th. As usual, there will be an Astronomy & Astrophysics
mini-symposium, so feel free to submit talks for that!

Note that there is also a plotting contest in memory of John Hunter (who
originally developed Matplotlib). I strongly encourage you to submit your
best plots/visualizations to this! When I asked last year, I was told that
one does not need to attend the conference in order to be able to submit an
entry.

Cheers,
Tom

---------- Forwarded message ----------
From: SciPy 2015 Organizers <scipy-organizers at scipy.org>
Date: 3 April 2015 at 19:31
Subject: Last call for Talk Submissions (Extended to 4/10), Keynotes
Announced
To: thomas.robitaille at gmail.com


 [image: SciPy 2015 Logo]
<https://www.eiseverywhere.com/emarketing/go.php?i=188033&e=dGhvbWFzLnJvYml0YWlsbGVAZ21haWwuY29t&l=http://scipy2015.scipy.org/ehome/index.php%7CQ%7Ceventid%7CE%7C115969%7CA%7C>
  LAST CALL! FINAL Talk Submission Extension: Due April 10th

There's always something new and exciting going on in the world of Science
+ Python, this is your chance to share!

*Visit the SciPy 2015 website
<https://www.eiseverywhere.com/emarketing/go.php?i=188033&e=dGhvbWFzLnJvYml0YWlsbGVAZ21haWwuY29t&l=http://scipy2015.scipy.org/ehome/index.php%7CQ%7Ceventid%7CE%7C115969%7CA%7C>
for full details or click here to submit a proposal
<https://www.eiseverywhere.com/emarketing/go.php?i=188033&e=dGhvbWFzLnJvYml0YWlsbGVAZ21haWwuY29t&l=http://www.scipy2015.scipy.org/eselectv2/frontend/index/115969>.*
Choose a topic in one of the 3 main conference tracks:

   - Scientific Computing in Python (General track), Python in Data
   Science, Quantitative and Computational Social Sciences

*And/or submit for one of the 7 domain-specific mini-symposia:
<https://www.eiseverywhere.com/emarketing/go.php?i=188033&e=dGhvbWFzLnJvYml0YWlsbGVAZ21haWwuY29t&l=http://www.scipy2015.scipy.org/eselectv2/frontend/index/115969>*

   - Astronomy and astrophysics, Computational life and medical sciences,
   Engineering, Geographic information systems (GIS), Geophysics, Oceanography
   and meteorology, Visualization, vision and imaging

   Submit a Talk or Poster Proposal Here
<https://www.eiseverywhere.com/emarketing/go.php?i=188033&e=dGhvbWFzLnJvYml0YWlsbGVAZ21haWwuY29t&l=http://www.scipy2015.scipy.org/eselectv2/frontend/index/115969>
------------------------------
   Keynote Speakers Announced!

We have a fabulous lineup of keynote speakers this year, including:

   - * Jake VanderPlas*, Director of Research in the Physical Sciences at
   the University of Washington's eScience Institute
   <https://www.eiseverywhere.com/emarketing/go.php?i=188033&e=dGhvbWFzLnJvYml0YWlsbGVAZ21haWwuY29t&l=http://www.astro.washington.edu/users/vanderplas/>.
   His own research is in astronomy, astrophysics, machine learning
   <https://www.eiseverywhere.com/emarketing/go.php?i=188033&e=dGhvbWFzLnJvYml0YWlsbGVAZ21haWwuY29t&l=http://scikit-learn.org/>,
   and scalable computation. He is a also a frequent contributor to many open
   source Python projects, including scikit-learn
   <https://www.eiseverywhere.com/emarketing/go.php?i=188033&e=dGhvbWFzLnJvYml0YWlsbGVAZ21haWwuY29t&l=http://scikit-learn.org/>,
   scipy
   <https://www.eiseverywhere.com/emarketing/go.php?i=188033&e=dGhvbWFzLnJvYml0YWlsbGVAZ21haWwuY29t&l=http://scipy.org/>,
   AstroML
   <https://www.eiseverywhere.com/emarketing/go.php?i=188033&e=dGhvbWFzLnJvYml0YWlsbGVAZ21haWwuY29t&l=http://www.astroml.org/>
   and others.
   - *Wes McKinney,* Software Engineer at Cloudera
   <https://www.eiseverywhere.com/emarketing/go.php?i=188033&e=dGhvbWFzLnJvYml0YWlsbGVAZ21haWwuY29t&l=http://www.cloudera.com/>
   and author of Pandas (Python Data Analysis Library)
   <https://www.eiseverywhere.com/emarketing/go.php?i=188033&e=dGhvbWFzLnJvYml0YWlsbGVAZ21haWwuY29t&l=http://pandas.pydata.org/>.
   From 2010 to 2012, he served as a Python consultant to hedge funds and
   banks. From 2007 to 2010, he researched global macro and credit trading
   strategies at AQR Capital Management.
   - *Chris Wiggins,* Chief Data Scientist at *The New York Times*
   <https://www.eiseverywhere.com/emarketing/go.php?i=188033&e=dGhvbWFzLnJvYml0YWlsbGVAZ21haWwuY29t&l=http://www.fastcompany.com/3026162/most-creative-people/why-the-new-york-times-hired-a-biology-researcher-as-its-chief-data-sci>,
   an associate professor of applied mathematics at Columbia University, the
   co-founder and co-organizer of hackNY,
   <https://www.eiseverywhere.com/emarketing/go.php?i=188033&e=dGhvbWFzLnJvYml0YWlsbGVAZ21haWwuY29t&l=http://hackny.org/>
   and a founding member of the Data Science Institute.
   <https://www.eiseverywhere.com/emarketing/go.php?i=188033&e=dGhvbWFzLnJvYml0YWlsbGVAZ21haWwuY29t&l=http://datascience.columbia.edu/>
   His research focuses on applications of machine learning to real-world
   data, particularly biology.

 *The State of the Scientific Python Stack*
*presented by Jake VanderPlas*
The scientific Python community relies not just on the Python language, but
on an entire ecosystem of open-source tools built on top of it. A key
strength of the community is that these tools are not static: they are
constantly evolving and improving through the volunteer effort of thousands
of community members around the world. I'll provide a high-level view of
the current state of the scientific Python software ecosystem, discuss its
strengths and weaknesses relative to other available tools, and highlight
some specific areas where I see room for growth and improvement.

Look for abstracts for Wes and Chris's presentations in future updates!
------------------------------
   SciPy 2015 Diversity Events

Want to get involved with the diversity committee to plan events and
outreach to broaden community participation in SciPy 2015? Know of a group
we can contact to publicize opportunities? Contact us
<scipy-organizers at scipy.org>.

We're continuing some of the successful new diversity focus areas from last
year, and would love additional suggestions or ideas
<scipy-organizers at scipy.org> as well. Current plans include:

   - Offsite diversity luncheon during the general conference
   - Five diversity scholarships, funded by NumFOCUS
   <https://www.eiseverywhere.com/emarketing/go.php?i=188033&e=dGhvbWFzLnJvYml0YWlsbGVAZ21haWwuY29t&l=https://numfocus.org/>
    (apply here before April 15th
   <https://www.eiseverywhere.com/emarketing/go.php?i=188033&e=dGhvbWFzLnJvYml0YWlsbGVAZ21haWwuY29t&l=http://scipy2015.scipy.org/ehome/115969/259279/%7CQ%7C%7CA%7C>
   )
   - Partnering with media sponsors and community organizations to
   advertise conference opportunities and activities to their memberships and
   audiences

------------------------------
  Birds of a Feather (BoF) Submissions Open

Birds-of-a-Feather sessions
<https://www.eiseverywhere.com/emarketing/go.php?i=188033&e=dGhvbWFzLnJvYml0YWlsbGVAZ21haWwuY29t&l=http://www.scipy2015.scipy.org/ehome/115969/259291/%7CQ%7C%7CA%7C>
are self-organized discussions that run parallel to the main conference.
The BOFs sessions cover primary, tangential, or unrelated topics in an
interactive, discussion setting. This year, some of the BOF sessions will
be scheduled and announced ahead of the conference. *By scheduling in
advance we can help promote attendance at the sessions, so please consider
submitting your ideas now
<https://www.eiseverywhere.com/emarketing/go.php?i=188033&e=dGhvbWFzLnJvYml0YWlsbGVAZ21haWwuY29t&l=http://www.scipy2015.scipy.org/ehome/115969/259291/%7CQ%7C%7CA%7C>!*

We would like to solicit the community for ideas and organizers for other
BoF topics. Please include a small description of the BoF, possible
panelists, and whether you would be willing to moderate.
   Submit your BOF Proposal here
<https://www.eiseverywhere.com/emarketing/go.php?i=188033&e=dGhvbWFzLnJvYml0YWlsbGVAZ21haWwuY29t&l=http://www.scipy2015.scipy.org/ehome/115969/259291/%7CQ%7C%7CA%7C>
------------------------------
   Registration Open - Please Register ASAP

New this year: *Register before May 15th*
<https://www.eiseverywhere.com/emarketing/go.php?i=188033&e=dGhvbWFzLnJvYml0YWlsbGVAZ21haWwuY29t&l=http://scipy2015.scipy.org/ehome/index.php%7CQ%7Ceventid%7CE%7C115969%7CA%7C>
and not only get early bird rates, but *two winners will be drawn to get
their choice of conference or tutorial registration free *(either as a
refund of paid fees or give it to a friend)!

*Please register ASAP*
<https://www.eiseverywhere.com/emarketing/go.php?i=188033&e=dGhvbWFzLnJvYml0YWlsbGVAZ21haWwuY29t&l=http://scipy2015.scipy.org>
if
you can to help us get a good head count and open the conference to as many
people as we can. Tutorials will also fill quickly, so don't miss out!
------------------------------
  Plotting Contest: Entries Due 4/13

In memory of John Hunter
<https://www.eiseverywhere.com/emarketing/go.php?i=188033&e=dGhvbWFzLnJvYml0YWlsbGVAZ21haWwuY29t&l=https://groups.google.com/forum/#!msg/pydata/FpwXp3sX6N8/mxopkZ1PkBQJ>,
creator of matplotlib
<https://www.eiseverywhere.com/emarketing/go.php?i=188033&e=dGhvbWFzLnJvYml0YWlsbGVAZ21haWwuY29t&l=http://matplotlib.org/>,
we are pleased to announce the Third Annual SciPy John Hunter Excellence in
Plotting Competition. This open competition aims to highlight the
importance of quality plotting to scientific progress and showcase
<https://www.eiseverywhere.com/emarketing/go.php?i=188033&e=dGhvbWFzLnJvYml0YWlsbGVAZ21haWwuY29t&l=http://www.scipy2015.scipy.org/ehome/115969/276538/%7CQ%7C%7CA%7C>the
capabilities of the current generation of plotting software.

Participants are invited to submit scientific plots to be judged by a
panel. John Hunter?s family is graciously sponsoring cash prizes for the
contest and the winning entries will be announced and displayed at the
conference. Full requirements and submission details here
<https://www.eiseverywhere.com/emarketing/go.php?i=188033&e=dGhvbWFzLnJvYml0YWlsbGVAZ21haWwuY29t&l=http://www.scipy2015.scipy.org/ehome/115969/276538/%7CQ%7C%7CA%7C>
.

Need a little inspiration? Check out the 2014 entries
<https://www.eiseverywhere.com/emarketing/go.php?i=188033&e=dGhvbWFzLnJvYml0YWlsbGVAZ21haWwuY29t&l=http://stsdas.stsci.edu/download/mdroe/plotting/>
!
------------------------------
  Calendar and Important Deadlines

   - *Currently Open: *Sprint submissions, BoF submissions, financial
   scholarship applications, talk submissions, registation
   - *Apr 10, 2015*: Talk and Poster
   <https://www.eiseverywhere.com/emarketing/go.php?i=188033&e=dGhvbWFzLnJvYml0YWlsbGVAZ21haWwuY29t&l=http://www.scipy2015.scipy.org/ehome/115969/259288/%7CQ%7C%7CA%7C>
   submission extended deadline
    - *Apr 13, 2015: *Plotting contest
   <https://www.eiseverywhere.com/emarketing/go.php?i=188033&e=dGhvbWFzLnJvYml0YWlsbGVAZ21haWwuY29t&l=http://www.scipy2015.scipy.org/ehome/115969/276538/%7CQ%7C%7CA%7C>
   submissions due
   - *Apr 15, 2015:* Financial scholarship
   <https://www.eiseverywhere.com/emarketing/go.php?i=188033&e=dGhvbWFzLnJvYml0YWlsbGVAZ21haWwuY29t&l=http://www.scipy2015.scipy.org/ehome/115969/259279/%7CQ%7C%7CA%7C>
   application deadline
   - *Apr 17, 2015:*Tutorial speakers and schedule announced
    - *May 1, 2015:* General conference speakers and schedule announced
   - *May 15, 2015* (or 150 registrants): Early-bird registration
   <https://www.eiseverywhere.com/emarketing/go.php?i=188033&e=dGhvbWFzLnJvYml0YWlsbGVAZ21haWwuY29t&l=http://www.scipy2015.scipy.org/ehome/115969/259272/%7CQ%7C%7CA%7C>
   ends
   - *Jun 1, 2015:* BoF submission deadline (for pre-scheduling and
   inclusion in program)

Follow @SciPy on Twitter
<https://www.eiseverywhere.com/emarketing/go.php?i=188033&e=dGhvbWFzLnJvYml0YWlsbGVAZ21haWwuY29t&l=https://twitter.com/scipyconf>,
Facebook
<https://www.eiseverywhere.com/emarketing/go.php?i=188033&e=dGhvbWFzLnJvYml0YWlsbGVAZ21haWwuY29t&l=https://www.facebook.com/scipyconf>
& Google+
<https://www.eiseverywhere.com/emarketing/go.php?i=188033&e=dGhvbWFzLnJvYml0YWlsbGVAZ21haWwuY29t&l=https://plus.google.com/100948873231627513165/>
for highlights from previous SciPy conferences and all of the latest 2015
updates.
------------------------------
  Sponsor Thank Yous We gratefully recognize our SciPy conference sponsors,
whose contributions ensure the accessibility of the conference and growth
of the important work being done by the scientific Python community.
[image: Square Root Logo]
<https://www.eiseverywhere.com/emarketing/go.php?i=188033&e=dGhvbWFzLnJvYml0YWlsbGVAZ21haWwuY29t&l=https://www.plaidcloud.com/>
*This week's sponsor highlight is silver sponsor, Tartan Solutions:*

*We'd like to thank Tartan Solutions
<https://www.eiseverywhere.com/emarketing/go.php?i=188033&e=dGhvbWFzLnJvYml0YWlsbGVAZ21haWwuY29t&l=https://www.tartansolutions.com/>,
developer of PlaidCloud
<https://www.eiseverywhere.com/emarketing/go.php?i=188033&e=dGhvbWFzLnJvYml0YWlsbGVAZ21haWwuY29t&l=https://plaidcloud.com/>
business analytics solutions for their support of SciPy 2015. PlaidCloud
provides people with impactful analysis tools on demand. Our tools provide
the ability to use Python for user defined processes as well as reporting
and exploring information using the IPython Notebook, all in a secure and
private environment.*

*PlaidCloud embraces the reality that insightful data lives on-premise
behind firewalls as well as being freely available in the cloud. We know
good quality data is a journey, not a given. Our tools allow the
integration of information from multiple sources and formats with the
ability to handle extensive data cleansing and business rule application
leading to rapid insight.*
------------------------------

Know a company that might want to sponsor SciPy 2015 or provide scholarship
funding? Share the sponsorship prospectus
<https://www.eiseverywhere.com/emarketing/go.php?i=188033&e=dGhvbWFzLnJvYml0YWlsbGVAZ21haWwuY29t&l=http://scipy2015.scipy.org/ehome/115969/259285/?&>
  --------------------------------------
To unsubscribe from this mailing list, please click here
<https://www.eiseverywhere.com/emarketing/profile.php?id=JMx413mdCkSJZEiCLOwc0uSnu%2FhNtx6bg4AT1d%2FKrkpGBG%2Bga7Ehgw%3D%3D>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/astropy/attachments/20150403/b901e4f1/attachment.html>

From ralf.gommers at gmail.com  Sat Apr  4 08:57:44 2015
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Sat, 4 Apr 2015 14:57:44 +0200
Subject: [AstroPy] Interested in a wavelet toolbox?
In-Reply-To: <D0473F1E-51E6-41CC-BC96-7A5F801CC045@gmail.com>
References: <551D8ECF.4050508@konteur.com>
	<493058D7-1419-4836-AABC-A596B140F8E4@gmail.com>
	<D0473F1E-51E6-41CC-BC96-7A5F801CC045@gmail.com>
Message-ID: <CABL7CQgkJy_w-=74b-MHS8snGRsumZDegcJ2V1fYgs2FmqvQSg@mail.gmail.com>

On Fri, Apr 3, 2015 at 12:33 AM, Christoph Deil <
deil.christoph at googlemail.com> wrote:

>
> On 02 Apr 2015, at 23:40, Ricky Egeland <ricky.egeland at gmail.com> wrote:
>
> I was searching for a well-tested wavelet library in Python a few months
> ago, and did not find anything fully satisfactory.  I wanted a continuous
> wavelet transform using the Morlet mother wavelet, with significance
> contours.  I feel like such a package belongs in scipy more than it does
> astropy, as wavelets are a general signal processing technique not
> restricted to astronomy.
>
>
Hi,

Hi Amara,
>
> I agree with Ricky here ? a good wavelets toolbox belongs in Scipy because
> it?s useful to many scientist and engineers.
> If you can contribute there, here?s a starting point:
> https://github.com/scipy/scipy/issues/3931 (and I?ve put Ralf Gommers as
> lead on this effort in CC).
>

A few comments:
- The main repo for that is at https://github.com/rgommers/pywt
- That code is a PyWavelets fork, so only DWT
- Quite a lot of effort went into it with contributions from 4 people. It's
close to being mergeable into Scipy, but it's currently stalled mainly
because I have very little bandwidth for it.
- Help is very much welcome. In particular, issues 46, 51 and 54 need
attention, plus review of and critical feedback on PR 55 would be very
useful.
- There's also interest from scikit-image for DWT:
https://github.com/scikit-image/scikit-image/pull/760

For continuous wavelets there has not been much recent work that I'm aware
of, and there is no decision or consensus over what code could be adapted
for use in Scipy. It is clear though that there's room in Scipy for
continuous wavelets, and that when a good feature-complete implementation
gets proposed it can go into scipy.wavelets. The current
scipy.signal.wavelets can then be deprecated.

Cheers,
Ralf



Then there?s probably going to be astronomy-specific wavelets and wavelet
> methods and these could go in Astropy or an Astropy-affiliated package.
> But I don?t think we should duplicate the functionality that?s already
> implemented in the packages listed below and that will hopefully soon
> become available in Scipy.
>
> Christoph
>
>  There are a few wavelet-related functions in scipy.signal already, but
> they do not work together (e.g. scipy.signal.cwt is unrelated to
> scipy.signal.morlet) and it is not clear how they are intended to be used.
> I have seen scipy discussions that propose throwing all that away and
> starting over.
>
> In case they are useful, here are my notes from the search:
>
> ===
> - scipy.signal.wavelet : cwt(), only Mexican hat wavelet?  Poor
> documentation
>
> - kPyWavelet : https://github.com/Cadair/kPyWavelet/
>   Linked from http://ocgweb.marine.usf.edu/~liu/wavelet.html
>   Few users... real interest?
>
> - A Practical Guide to Wavelet Analysis
>   http://paos.colorado.edu/research/wavelets/
>   Highly cited ... no python
>
> - pycwt
>   https://github.com/regeirk/pycwt
>   Explicitly based on T&C code above
>   # Tested.  Example code has a few minor bugs, but appears to be doing
> the right thing
>
> - PyCWT
>   https://github.com/Unidata/pyCWT
>   Simple, 1 file
>   Based on T&C
>   # Tested. No example code, but docstrings have examples (that don't work
> as-is)  Source code is cleaner than pycwt.  scaleogram() has some issues.
> Need to input scales, not frequencies...
>
> - ObsPy
>
> http://docs.obspy.org/master/tutorial/code_snippets/continuous_wavelet_transform.html
>
> - PyWavelets
>   http://www.pybytes.com/pywavelets/
>   Part of MacPorts
>   Good documentation effort
>   Only one developer, but 5+ years effort
>
> - https://github.com/aaren/wavelets
>
> - https://pypi.python.org/pypi/waipy
>   Based on T&C code
>
> - scipy wavelet discussions:
> https://github.com/scipy/scipy/issues?utf8=?&q=is%3Aissue+is%3Aopen+wavelet
> ===
>
> In the end I forked Unidata/pyCWT, fixed a few things and used that (
> https://github.com/rickyegeland/pyCWT).  On a second look I decided
> regeirk/pycwt might have been the better implementation, but the interface
> and documentation were not the best for getting started.
>
> Hope this helps.
>
> Regards,
> Ricky Egeland
>
>
> On Apr 2, 2015, at 12:47 PM, Amara Graps <amara at konteur.com> wrote:
>
> Greetings astropy-ers,
>
> I'm proposing the development of a wavelet library in Python to a
> (European Commission) funding agency and I'll need beta testing
> of some parts. I think it would be especially great to have the
> library be incorporated into AstroPy after it's written and tested
> because the funds for maintenance would end when the project ends..
> yet astronomers continue to love wavelets.
>
> Some large part of my Python wavelet library would be built upon this
> well-established (general purpose) Matlab toolbox:
>
> http://statweb.stanford.edu/~wavelab/Wavelab_850/index_wavelab850.html
>
> Does this sound useful and interesting to this group? If it does, please
> reply to graps at psi.edu. I'm gathering documentation of the AstroPy user
> community's interest and my testing and maintenance methodology in order
> to help the project get funded.
>
> Thanks very much in advance!
>
> Sincerely,
> Amara
>
> Amara Graps, PhD
> Lead Researcher,  University of Latvia
> Astronomijas instit?ts, Rai?a bulv?ris 19, R?ga, LV-1050, Latvia
> amara at lu.lv  +371 / 28853907
> and
> Senior Scientist, Planetary Science Institute (PSI, USA),
> Aleksandra Caka iela, 96-31,
> Riga, Latvia LV-1011
> http://www.psi.edu/about/staff/graps/graps.html
> graps at psi.edu  +1 / 644-6250
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org
> http://mail.scipy.org/mailman/listinfo/astropy
>
>
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org
> http://mail.scipy.org/mailman/listinfo/astropy
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/astropy/attachments/20150404/282b37a4/attachment.html>

From abhijithrajan at asu.edu  Tue Apr 14 16:26:55 2015
From: abhijithrajan at asu.edu (Abhijith Rajan)
Date: Tue, 14 Apr 2015 13:26:55 -0700
Subject: [AstroPy] boolean header keyword
Message-ID: <CAHfbfxE5GpFdUjDcHWb+ic65TNs4B_j+ZP+p8+pxi8KrFwQp1A@mail.gmail.com>

Hello,

This is most likely a question for Eric, but I thought I'd see if someone
knew the answer. I'm using the setval command in astropy.io.fits to update
a header keyword in an HST image (SUBARRAY). The keyword originally
contains either a T or F if the image is a subarray or not.

How do I use setval to change the keyword from T to F or vice versa. Right
now its updating the header with a string and essentially crashing the code
checking to make sure that the image is not a subarray.

Thanks
Abhi

-------------------------------------------------------------------------------------------
Abhijith Rajan
PhD Candidate
Arizona State University
School of Earth & Space Exploration
Interdisciplinary Sci & Tech Bldg 4 Rm 542
Tempe, AZ 85287
Tel: (480) 727-2559
Fax: (480) 965-8102
www.abhijithrajan.com
-------------------------------------------------------------------------------------------
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/astropy/attachments/20150414/efa38d00/attachment.html>

From dencheva at stsci.edu  Tue Apr 14 16:42:53 2015
From: dencheva at stsci.edu (Nadezhda Dencheva)
Date: Tue, 14 Apr 2015 20:42:53 +0000
Subject: [AstroPy] boolean header keyword
In-Reply-To: <CAHfbfxE5GpFdUjDcHWb+ic65TNs4B_j+ZP+p8+pxi8KrFwQp1A@mail.gmail.com>
References: <CAHfbfxE5GpFdUjDcHWb+ic65TNs4B_j+ZP+p8+pxi8KrFwQp1A@mail.gmail.com>
Message-ID: <0153364C9F56D944B0A795780ECF88DF6778BE44@EXCHMAIL2.stsci.edu>

Hi Abhi,

Use python boolean values, i.e. pass value=True to setval.

Nadia


________________________________
From: astropy-bounces at scipy.org [astropy-bounces at scipy.org] on behalf of Abhijith Rajan [abhijithrajan at asu.edu]
Sent: Tuesday, April 14, 2015 4:26 PM
To: astropy
Subject: [AstroPy] boolean header keyword

Hello,

This is most likely a question for Eric, but I thought I'd see if someone knew the answer. I'm using the setval command in astropy.io.fits to update a header keyword in an HST image (SUBARRAY). The keyword originally contains either a T or F if the image is a subarray or not.

How do I use setval to change the keyword from T to F or vice versa. Right now its updating the header with a string and essentially crashing the code checking to make sure that the image is not a subarray.

Thanks
Abhi

-------------------------------------------------------------------------------------------
Abhijith Rajan
PhD Candidate
Arizona State University
School of Earth & Space Exploration
Interdisciplinary Sci & Tech Bldg 4 Rm 542
Tempe, AZ 85287
Tel: (480) 727-2559
Fax: (480) 965-8102
www.abhijithrajan.com<http://www.abhijithrajan.com>
-------------------------------------------------------------------------------------------
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/astropy/attachments/20150414/5348d6b7/attachment.html>

From embray at stsci.edu  Tue Apr 14 16:52:58 2015
From: embray at stsci.edu (Erik Bray)
Date: Tue, 14 Apr 2015 16:52:58 -0400
Subject: [AstroPy] boolean header keyword
In-Reply-To: <CAHfbfxE5GpFdUjDcHWb+ic65TNs4B_j+ZP+p8+pxi8KrFwQp1A@mail.gmail.com>
References: <CAHfbfxE5GpFdUjDcHWb+ic65TNs4B_j+ZP+p8+pxi8KrFwQp1A@mail.gmail.com>
Message-ID: <552D7E2A.4030407@stsci.edu>

T and F (without quotes) are bool literals in FITS.  Hence,

In [1]: from astropy.io import fits

In [2]: h = fits.Header()

In [3]: h['SIMPLE'] = True

In [4]: h
Out[4]: SIMPLE  =            T

In [5]: h['SIMPLE'] = False

In [6]: h
Out[6]: SIMPLE  =            F



On 04/14/2015 04:26 PM, Abhijith Rajan wrote:
> Hello,
>
> This is most likely a question for Eric, but I thought I'd see if someone knew
> the answer. I'm using the setval command in astropy.io.fits to update a header
> keyword in an HST image (SUBARRAY). The keyword originally contains either a T
> or F if the image is a subarray or not.
>
> How do I use setval to change the keyword from T to F or vice versa. Right now
> its updating the header with a string and essentially crashing the code checking
> to make sure that the image is not a subarray.
>
> Thanks
> Abhi
>
> -------------------------------------------------------------------------------------------
> Abhijith Rajan
> PhD Candidate
> Arizona State University
> School of Earth & Space Exploration
> Interdisciplinary Sci & Tech Bldg 4 Rm 542
> Tempe, AZ 85287
> Tel: (480) 727-2559
> Fax: (480) 965-8102
> www.abhijithrajan.com <http://www.abhijithrajan.com>
> -------------------------------------------------------------------------------------------
>
>
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org
> http://mail.scipy.org/mailman/listinfo/astropy
>



From mlai at ices.utexas.edu  Tue Apr 14 19:08:59 2015
From: mlai at ices.utexas.edu (Mark Lai)
Date: Tue, 14 Apr 2015 18:08:59 -0500
Subject: [AstroPy] seeking feedback on how to process seismic images from
	astronomers
Message-ID: <CAFu39xL=2-WQSUQNg=E7PTsLAX_BbiAdJ_6-aozmGw5mQXBDWA@mail.gmail.com>

Hi all!

My name is Mark Lai, and I am currently a 1st year post doc in ICES
(computational math) at UT Austin doing work in seismic imaging with Sergey
Fomel.

http://users.ices.utexas.edu/~mlai/

http://www.jsg.utexas.edu/researcher/sergey_fomel/

A senior math professor mentioned some interesting image processing work by
astronomers so I thought I would try to reach out!

One of the problems we face is to how to process a seismic image to make it
visually appealing for geologists.  Page 3 in the below linked PDF shows
different types of processing. The middle figure in the second column is
the preferred one.  The paper, essentially a blog post, is by a famous
retired geophysicist in our field and argues for histogram equalization.

http://sepwww.stanford.edu/sep/jon/softclips.pdf

I am curious if anyone might any suggestions or feedback on how to process
these seismic images.  Thanks much and please let me know if you have any
questions!

Mark Lai
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/astropy/attachments/20150414/d1f5690a/attachment.html>

From wkerzendorf at gmail.com  Wed Apr 15 03:56:23 2015
From: wkerzendorf at gmail.com (Wolfgang Kerzendorf)
Date: Wed, 15 Apr 2015 07:56:23 +0000
Subject: [AstroPy] seeking feedback on how to process seismic images
	from astronomers
In-Reply-To: <CAFu39xL=2-WQSUQNg=E7PTsLAX_BbiAdJ_6-aozmGw5mQXBDWA@mail.gmail.com>
References: <CAFu39xL=2-WQSUQNg=E7PTsLAX_BbiAdJ_6-aozmGw5mQXBDWA@mail.gmail.com>
Message-ID: <CAH33GGHBeRgBpAB0F_iSQE+OAqUgOzmQi1igA+HZBpa4k4YH3w@mail.gmail.com>

Dear Mark,

Maybe astronomy was once at the forefront of digital image processing, but
this not the case anymore. I suggest looking at scikit-image and opencv
both of them do these things pretty well. Let us know how you go!

Good luck,
    Wolfgang

On Wed, Apr 15, 2015 at 1:09 AM Mark Lai <mlai at ices.utexas.edu> wrote:

> Hi all!
>
> My name is Mark Lai, and I am currently a 1st year post doc in ICES
> (computational math) at UT Austin doing work in seismic imaging with Sergey
> Fomel.
>
> http://users.ices.utexas.edu/~mlai/
>
> http://www.jsg.utexas.edu/researcher/sergey_fomel/
>
> A senior math professor mentioned some interesting image processing work
> by astronomers so I thought I would try to reach out!
>
> One of the problems we face is to how to process a seismic image to make
> it visually appealing for geologists.  Page 3 in the below linked PDF shows
> different types of processing. The middle figure in the second column is
> the preferred one.  The paper, essentially a blog post, is by a famous
> retired geophysicist in our field and argues for histogram equalization.
>
> http://sepwww.stanford.edu/sep/jon/softclips.pdf
>
> I am curious if anyone might any suggestions or feedback on how to process
> these seismic images.  Thanks much and please let me know if you have any
> questions!
>
> Mark Lai
>
>
>
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org
> http://mail.scipy.org/mailman/listinfo/astropy
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/astropy/attachments/20150415/858dd2ad/attachment.html>

From rays at blue-cove.com  Wed Apr 15 11:58:54 2015
From: rays at blue-cove.com (R Schumacher)
Date: Wed, 15 Apr 2015 08:58:54 -0700
Subject: [AstroPy] seeking feedback on how to process seismic images
 from astronomers
In-Reply-To: <CAFu39xL=2-WQSUQNg=E7PTsLAX_BbiAdJ_6-aozmGw5mQXBDWA@mail.g
	mail.com>
References: <CAFu39xL=2-WQSUQNg=E7PTsLAX_BbiAdJ_6-aozmGw5mQXBDWA@mail.gmail.com>
Message-ID: <201504151558.t3FFwt20030492@blue-cove.com>

At 04:08 PM 4/14/2015, you wrote:
>One of the problems we face is to how to process a seismic image to 
>make it visually appealing for geologists.

In industrial vibration analysis we've moved mostly from waterfall 
plots to color intensity waterfalls, either Fourrier domain or time 
series ensembles.
Similar to what i do
http://jaganadhg.freeflux.net/blog/archive/2009/09/09/plotting-wave-form-and-spectrogram-the-pure-python-way.html
but using the sometimes hated jet map 
(http://cresspahl.blogspot.com/2012/03/expanded-control-of-octaves-colormap.html)

I now try to use Mayavi when it lends itself, and it appears to be 
catching on in geophysics:
http://agilegeoscience.squarespace.com/journal/tag/programming?currentPage=3

Ray Schumacher
Programmer/Engineer

Jan Medical, Inc. (SDRC) 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/astropy/attachments/20150415/5fa564ea/attachment.html>

From mr.alex.hagen at gmail.com  Thu Apr 16 10:31:22 2015
From: mr.alex.hagen at gmail.com (Alex Hagen)
Date: Thu, 16 Apr 2015 16:31:22 +0200
Subject: [AstroPy] combining columns when writing a table to latex
Message-ID: <CANS2AxKw-TYS77UVcojRtMESnqZkwh-XgMLke_OXWDj5wiDM6Q@mail.gmail.com>

Hi Astropy Folks,

I have a question regarding writing out a table to latex. One of my columns
is the value of a parameter, and another column is the uncertainty. I was
wondering if there's a nice way to have the writer combine these into one
column that has value \pm uncertainty? Or should I combine the columns
beforehand into exactly the string I want to be written out?

Thanks!

--alex

-- 
alex hagen
916.425.3581
https://astronomeralex.github.io/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/astropy/attachments/20150416/ef6131e1/attachment.html>

From moritz.guenther at gmx.de  Thu Apr 16 13:06:17 2015
From: moritz.guenther at gmx.de (Moritz Guenther)
Date: Thu, 16 Apr 2015 13:06:17 -0400
Subject: [AstroPy] combining columns when writing a table to latex
In-Reply-To: <mailman.1.1429203601.2445.astropy@scipy.org>
References: <mailman.1.1429203601.2445.astropy@scipy.org>
Message-ID: <552FEC09.5070902@gmx.de>

Hi Alex,

you will need to modify the table and add a new column before you call 
the latex writer. The LaTeX writer can only write the columns of your 
tables as they are.
However, personally, I would strongly prefer to have two columns (value 
and uncertainty) and not merge them into a single string. If anybody 
ever wants to read in the data from your LaTeX table (I sometimes pull 
the latex from astro-ph if I want to read data from a journal article 
that is not included as electronic table), astropy will read a tables 
where value and uncertainty are different columns easily, but it's some 
extra pain to parse a string column with values like "$10.5 \pm 1.1".

Yours,
Moritz


From pbarmby at uwo.ca  Thu Apr 16 15:45:10 2015
From: pbarmby at uwo.ca (Pauline Barmby)
Date: Thu, 16 Apr 2015 15:45:10 -0400
Subject: [AstroPy] AstroPy Digest, Vol 103, Issue 9
In-Reply-To: <mailman.1.1429203601.2445.astropy@scipy.org>
References: <mailman.1.1429203601.2445.astropy@scipy.org>
Message-ID: <29D0C092-FAD2-41D4-AA2C-7ADE2F878CA4@uwo.ca>

> 
> 
> I have a question regarding writing out a table to latex. One of my columns
> is the value of a parameter, and another column is the uncertainty. I was
> wondering if there's a nice way to have the writer combine these into one
> column that has value \pm uncertainty? Or should I combine the columns
> beforehand into exactly the string I want to be written out?
> 

Hi Alex -- 

I have some code that does what you want, admittedly not in
a super-elegant way:

https://gist.github.com/PBarmby/e4384a81cf2e1c53365c

Suggestions, PRs, etc welcome.

Dr. Pauline Barmby, Associate Professor
Department of Physics & Astronomy,  Western University
1151 Richmond St., London, ON N6A 3K7, Canada
Voice: (519) 661-2111 x81557 Fax: (519) 661-2033
pbarmby at uwo.ca







From crawford at saao.ac.za  Fri Apr 17 14:32:54 2015
From: crawford at saao.ac.za (Steve Crawford)
Date: Fri, 17 Apr 2015 20:32:54 +0200 (SAST)
Subject: [AstroPy] [ANN] ccdproc v0.3: CCD data reduction
In-Reply-To: <1272203121.193118.1429281904459.JavaMail.zimbra@saao.ac.za>
Message-ID: <1011740793.197519.1429295574863.JavaMail.zimbra@saao.ac.za>


Dear colleagues

We are pleased to announce the release of ccdproc v0.3.  Ccdproc is is an affiliated package 
for the AstroPy package for basic data reductions of CCD images. The ccdproc package provides 
many of the necessary tools for processing of ccd images built on a framework to provide 
error propagation and bad pixel tracking throughout the reduction process.  This release of
ccdproc does require astropy v1.0.   

More Information and repository
https://github.com/astropy/ccdproc

Documentation
http://ccdproc.readthedocs.org/en/latest/

Installation
http://ccdproc.readthedocs.org/en/latest/ccdproc/install.html

Feedback, contributions, and comments are welcome.  We would especially be interested in 
contributions of examples of using ccdproc and comparisons to other reduction methods.   
Thanks to everyone that has commented and contributed to the project so far.

Cheers
Steve Crawford and Matthew Craig


From skwok at keck.hawaii.edu  Sat Apr 18 23:25:24 2015
From: skwok at keck.hawaii.edu (Shui Kwok)
Date: Sun, 19 Apr 2015 03:25:24 +0000
Subject: [AstroPy] seeking feedback on how to process seismic images
 from	astronomers
In-Reply-To: <CAFu39xL=2-WQSUQNg=E7PTsLAX_BbiAdJ_6-aozmGw5mQXBDWA@mail.gmail.com>
References: <CAFu39xL=2-WQSUQNg=E7PTsLAX_BbiAdJ_6-aozmGw5mQXBDWA@mail.gmail.com>
Message-ID: <E1D5BAA761375F4F9E27DF9F7A3141384361AFB0@wsmail1.keck.hawaii.edu>

Hi Mark,

I used to write software for seismic data processing many years ago.
As I remember, the state of the seismic data processing software was quite advanced.  Most of the software used in the oil and gas exploration industry was (maybe still is) commercial, proprietary and expensive because there was a lot of money to be made. There are open source packages, but they are for educational proposes and not for production.

The commercial packages also offer excellent visualization features. I'm not going to mention any brands or products here, but they are easy to find. May be their product information can give you new ideas.

Shui


________________________________
From: astropy-bounces at scipy.org [astropy-bounces at scipy.org] on behalf of Mark Lai [mlai at ices.utexas.edu]
Sent: Tuesday, April 14, 2015 1:08 PM
To: astropy at scipy.org
Subject: [AstroPy] seeking feedback on how to process seismic images from astronomers

Hi all!

My name is Mark Lai, and I am currently a 1st year post doc in ICES (computational math) at UT Austin doing work in seismic imaging with Sergey Fomel.

http://users.ices.utexas.edu/~mlai/

http://www.jsg.utexas.edu/researcher/sergey_fomel/

A senior math professor mentioned some interesting image processing work by astronomers so I thought I would try to reach out!

One of the problems we face is to how to process a seismic image to make it visually appealing for geologists.  Page 3 in the below linked PDF shows different types of processing. The middle figure in the second column is the preferred one.  The paper, essentially a blog post, is by a famous retired geophysicist in our field and argues for histogram equalization.

http://sepwww.stanford.edu/sep/jon/softclips.pdf

I am curious if anyone might any suggestions or feedback on how to process these seismic images.  Thanks much and please let me know if you have any questions!

Mark Lai



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/astropy/attachments/20150419/249011a6/attachment.html>

From ivan.zolotukhin at gmail.com  Sun Apr 19 17:40:16 2015
From: ivan.zolotukhin at gmail.com (Ivan Zolotukhin)
Date: Sun, 19 Apr 2015 23:40:16 +0200
Subject: [AstroPy] writing FITS files fast
Message-ID: <CAD=Ky+eNBQ6KxmUv1jGGaK+N7nYb+9pjHxG2Cb8mF5rfQe3CwQ@mail.gmail.com>

Hi,

I need to write a FITS file from a web application in python and I
need to do it fast. When I write file using astropy.io.fits module
(v0.3.x), it takes several seconds for a table with 300+ columns and
only 20 rows. Here's the hotshot profiler output of the relevant piece
of code:

         8506345 function calls (8480504 primitive calls) in 18.801 seconds

   Ordered by: internal time, call count
   List reduced from 2343 to 20 due to restriction <20>

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
    10635    5.409    0.001   13.575    0.001
/usr/local/lib/python2.7/dist-packages/astropy/io/fits/column.py:953(__getattr__)
  3660493    4.787    0.000    8.172    0.000
/usr/local/lib/python2.7/dist-packages/astropy/io/fits/column.py:996(__getitem__)
  3650946    3.386    0.000    3.386    0.000
/usr/local/lib/python2.7/dist-packages/astropy/io/fits/util.py:589(_is_int)
     2099    0.393    0.000    0.393    0.000
/usr/local/lib/python2.7/dist-packages/astropy/io/fits/header.py:1779(_updateindices)

For the more recent astropy version 1.0.2 situation is somewhat better
but it's still significantly slower than I need:

         1153615 function calls (1138771 primitive calls) in 4.364 seconds

   Ordered by: internal time, call count
   List reduced from 736 to 20 due to restriction <20>

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
     2099    0.419    0.000    0.425    0.000
/usr/local/lib/python2.7/dist-packages/astropy/io/fits/header.py:1808(_updateindices)
      346    0.312    0.001    0.852    0.002
/usr/local/lib/python2.7/dist-packages/astropy/io/fits/column.py:1245(__getattr__)
   124176    0.275    0.000    0.406    0.000
/usr/local/lib/python2.7/dist-packages/astropy/io/fits/column.py:1285(__getitem__)
      327    0.215    0.001    0.222    0.001
/usr/local/lib/python2.7/dist-packages/django/db/backends/utils.py:58(execute)
      357    0.201    0.001    0.293    0.001
/usr/local/lib/python2.7/dist-packages/astropy/io/fits/header.py:1399(index)
   137557    0.183    0.000    0.183    0.000
/usr/local/lib/python2.7/dist-packages/astropy/io/fits/column.py:421(__get__)
   124226    0.132    0.000    0.132    0.000
/usr/local/lib/python2.7/dist-packages/astropy/io/fits/util.py:906(_is_int)

My code is very simple however, it just creates necessary set of
columns (astropy.table.MaskedColumn objects) casting data from python
lists coming from the database to numpy arrays. Are there ways to save
on checks like _is_int() in case the input array datatype has been
already enforced?

When the code is rewritten to use fitsio module
(https://pypi.python.org/pypi/fitsio/), the execution time is
significantly better:

         129409 function calls (128693 primitive calls) in 0.811 seconds

   Ordered by: internal time, call count
   List reduced from 289 to 20 due to restriction <20>

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
      327    0.218    0.001    0.221    0.001
/usr/local/lib/python2.7/dist-packages/django/db/backends/utils.py:58(execute)
        4    0.083    0.021    0.083    0.021
/usr/local/lib/python2.7/dist-packages/fitsio/fitslib.py:1311(_update_info)

...but fitsio does not seem to support 64 bit integers which is another
requirement I have.

What's the fastest solution to write FITS files that's available on
the python market? Am I missing something in the astropy
configuration?

--
With best regards,
 Ivan


From demitri.muna at gmail.com  Sun Apr 19 18:01:02 2015
From: demitri.muna at gmail.com (Demitri Muna)
Date: Sun, 19 Apr 2015 17:01:02 -0500
Subject: [AstroPy] writing FITS files fast
In-Reply-To: <CAD=Ky+eNBQ6KxmUv1jGGaK+N7nYb+9pjHxG2Cb8mF5rfQe3CwQ@mail.gmail.com>
References: <CAD=Ky+eNBQ6KxmUv1jGGaK+N7nYb+9pjHxG2Cb8mF5rfQe3CwQ@mail.gmail.com>
Message-ID: <C75CDC6E-2F00-4495-8267-80EC129B68F2@gmail.com>

Hi Ivan,

On Apr 19, 2015, at 4:40 PM, Ivan Zolotukhin <ivan.zolotukhin at gmail.com> wrote:

> ...but fitsio does not seem to support 64 bit integers which is another
> requirement I have.
> 
> What's the fastest solution to write FITS files that's available on
> the python market? Am I missing something in the astropy
> configuration?


Erin Sheldon?s fitsio is, as you've found, the fastest solution on the market. It wraps cfitsio under the hood. My guess is that adding the functionality to support 64 bit integers is probably not a lot of work, and very likely a fair bit less work than getting astropy.io.fits to be as fast.

Cheers,
Demitri

_________________________________________
Demitri Muna

Department of Astronomy
Il Ohio State University

http://trillianverse.org
http://scicoder.org



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/astropy/attachments/20150419/0cbd3d80/attachment.html>

From demitri.muna at gmail.com  Sun Apr 19 18:19:38 2015
From: demitri.muna at gmail.com (Demitri Muna)
Date: Sun, 19 Apr 2015 17:19:38 -0500
Subject: [AstroPy] writing FITS files fast
In-Reply-To: <CAD=Ky+eNBQ6KxmUv1jGGaK+N7nYb+9pjHxG2Cb8mF5rfQe3CwQ@mail.gmail.com>
References: <CAD=Ky+eNBQ6KxmUv1jGGaK+N7nYb+9pjHxG2Cb8mF5rfQe3CwQ@mail.gmail.com>
Message-ID: <8738C82A-736E-4CBC-8616-EF495267AF9B@gmail.com>


A quick glance at the source seems to suggest that fitsio does support int64...

_________________________________________
Demitri Muna

Department of Astronomy
An Ohio State University

http://trillianverse.org
http://scicoder.org



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/astropy/attachments/20150419/1d6f16e3/attachment.html>

From ivan.zolotukhin at gmail.com  Sun Apr 19 18:33:50 2015
From: ivan.zolotukhin at gmail.com (Ivan Zolotukhin)
Date: Mon, 20 Apr 2015 00:33:50 +0200
Subject: [AstroPy] writing FITS files fast
In-Reply-To: <8738C82A-736E-4CBC-8616-EF495267AF9B@gmail.com>
References: <CAD=Ky+eNBQ6KxmUv1jGGaK+N7nYb+9pjHxG2Cb8mF5rfQe3CwQ@mail.gmail.com>
	<8738C82A-736E-4CBC-8616-EF495267AF9B@gmail.com>
Message-ID: <CAD=Ky+dRTjnXxZpeMVPTQ+=eS2GCN90tO0fc1H9UB27qw7HZNg@mail.gmail.com>

On Mon, Apr 20, 2015 at 12:19 AM, Demitri Muna <demitri.muna at gmail.com> wrote:
>
> A quick glance at the source seems to suggest that fitsio does support
> int64...

Writing FITS file with columns having dtype='int64' causes this error
in fitsio 0.9.7:

File "/usr/local/lib/python2.7/dist-packages/fitsio/fitslib.py", line
448, in write
     table_type=table_type)
File "/usr/local/lib/python2.7/dist-packages/fitsio/fitslib.py", line
716, in write_table
     self[-1].write(data,names=names)
File "/usr/local/lib/python2.7/dist-packages/fitsio/fitslib.py", line
1508, in write
     firstrow=firstrow+1)
 TypeError: Unsupported numpy table datatype 9

--
With best regards,
 Ivan


From erin.sheldon at gmail.com  Sun Apr 19 20:12:45 2015
From: erin.sheldon at gmail.com (Erin Sheldon)
Date: Sun, 19 Apr 2015 20:12:45 -0400
Subject: [AstroPy] writing FITS files fast
In-Reply-To: <CAD=Ky+dRTjnXxZpeMVPTQ+=eS2GCN90tO0fc1H9UB27qw7HZNg@mail.gmail.com>
References: <CAD=Ky+eNBQ6KxmUv1jGGaK+N7nYb+9pjHxG2Cb8mF5rfQe3CwQ@mail.gmail.com>
	<8738C82A-736E-4CBC-8616-EF495267AF9B@gmail.com>
	<CAD=Ky+dRTjnXxZpeMVPTQ+=eS2GCN90tO0fc1H9UB27qw7HZNg@mail.gmail.com>
Message-ID: <CABOnLw2pY3z-4boWzZTbeXv6tZR4Xp02tiW7a1G0mc1oh9ncyw@mail.gmail.com>

Hi Ivan -

fitsio does support 64-bit.

Let's try to figure it out.  Can you please open an issue on the
fitsio page, with a full example (including the actual data you write,
etc)?

https://github.com/esheldon/fitsio

thanks,
-e

On 4/19/15, Ivan Zolotukhin <ivan.zolotukhin at gmail.com> wrote:
> On Mon, Apr 20, 2015 at 12:19 AM, Demitri Muna <demitri.muna at gmail.com>
> wrote:
>>
>> A quick glance at the source seems to suggest that fitsio does support
>> int64...
>
> Writing FITS file with columns having dtype='int64' causes this error
> in fitsio 0.9.7:
>
> File "/usr/local/lib/python2.7/dist-packages/fitsio/fitslib.py", line
> 448, in write
>      table_type=table_type)
> File "/usr/local/lib/python2.7/dist-packages/fitsio/fitslib.py", line
> 716, in write_table
>      self[-1].write(data,names=names)
> File "/usr/local/lib/python2.7/dist-packages/fitsio/fitslib.py", line
> 1508, in write
>      firstrow=firstrow+1)
>  TypeError: Unsupported numpy table datatype 9
>
> --
> With best regards,
>  Ivan
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org
> http://mail.scipy.org/mailman/listinfo/astropy
>


-- 
Erin Scott Sheldon
Brookhaven National Laboratory erin dot sheldon at gmail dot com


From august.fly at gmail.com  Sun Apr 19 21:00:34 2015
From: august.fly at gmail.com (August (Gus) Muench)
Date: Sun, 19 Apr 2015 21:00:34 -0400
Subject: [AstroPy] combining columns when writing a table to latex
In-Reply-To: <552FEC09.5070902@gmx.de>
References: <mailman.1.1429203601.2445.astropy@scipy.org>
	<552FEC09.5070902@gmx.de>
Message-ID: <CAMER+OvKotvHWTpat7OBBhPZ=3RRRE68dK6o+E5-9qy8410QSg@mail.gmail.com>

Hi Alex,

Just want to add to Moritz's point that \pm represents just one of many
ways that authors format their tables for a nice print view but without
thinking at all about reuse by others...

It is true that many large tables are "normalized" by scientists at the AAS
Journals (where I work) and CDS, but it is also true that breaking out of a
print view of one's article benefits everyone immensely.

Just sayin...

 - Gus




----------
August (Gus) Muench
august.fly at gmail.com
@augustmuench

On Thu, Apr 16, 2015 at 1:06 PM, Moritz Guenther <moritz.guenther at gmx.de>
wrote:

> Hi Alex,
>
> you will need to modify the table and add a new column before you call
> the latex writer. The LaTeX writer can only write the columns of your
> tables as they are.
> However, personally, I would strongly prefer to have two columns (value
> and uncertainty) and not merge them into a single string. If anybody
> ever wants to read in the data from your LaTeX table (I sometimes pull
> the latex from astro-ph if I want to read data from a journal article
> that is not included as electronic table), astropy will read a tables
> where value and uncertainty are different columns easily, but it's some
> extra pain to parse a string column with values like "$10.5 \pm 1.1".
>
> Yours,
> Moritz
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org
> http://mail.scipy.org/mailman/listinfo/astropy
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/astropy/attachments/20150419/c0d39e73/attachment.html>

From embray at stsci.edu  Mon Apr 20 00:29:27 2015
From: embray at stsci.edu (Erik Bray)
Date: Mon, 20 Apr 2015 06:29:27 +0200
Subject: [AstroPy] writing FITS files fast
In-Reply-To: <CAD=Ky+eNBQ6KxmUv1jGGaK+N7nYb+9pjHxG2Cb8mF5rfQe3CwQ@mail.gmail.com>
References: <CAD=Ky+eNBQ6KxmUv1jGGaK+N7nYb+9pjHxG2Cb8mF5rfQe3CwQ@mail.gmail.com>
Message-ID: <553480A7.1000207@stsci.edu>

On 4/19/2015 11:40 PM, Ivan Zolotukhin wrote:
> Hi,
>
> I need to write a FITS file from a web application in python and I
> need to do it fast. When I write file using astropy.io.fits module
> (v0.3.x), it takes several seconds for a table with 300+ columns and
> only 20 rows. Here's the hotshot profiler output of the relevant piece
> of code:

It *probably* need not be that slow, but it would be helpful to see your actual 
code.  For example, your output below shows a *lot* of calls to and a lot of 
time spent in code that can *probably* be avoided.  Though as you found using 
the most recent version should be significantly faster (v0.3.x is very old by 
comparison and there have been a lot of improvements since then).

Though I also agree fitsio should get you your best bang for your buck in most 
cases for raw I/O, and it should support 64-bit ints just fine.  That said, most 
of the time you're seeing in the astropy.io.fits code is nothing I/O releated, 
but just overhead related to creating the columns in the first place, and is 
probably mostly avoidable.

Erik

>           8506345 function calls (8480504 primitive calls) in 18.801 seconds
>
>     Ordered by: internal time, call count
>     List reduced from 2343 to 20 due to restriction <20>
>
>     ncalls  tottime  percall  cumtime  percall filename:lineno(function)
>      10635    5.409    0.001   13.575    0.001
> /usr/local/lib/python2.7/dist-packages/astropy/io/fits/column.py:953(__getattr__)
>    3660493    4.787    0.000    8.172    0.000
> /usr/local/lib/python2.7/dist-packages/astropy/io/fits/column.py:996(__getitem__)
>    3650946    3.386    0.000    3.386    0.000
> /usr/local/lib/python2.7/dist-packages/astropy/io/fits/util.py:589(_is_int)
>       2099    0.393    0.000    0.393    0.000
> /usr/local/lib/python2.7/dist-packages/astropy/io/fits/header.py:1779(_updateindices)
>
> For the more recent astropy version 1.0.2 situation is somewhat better
> but it's still significantly slower than I need:
>
>           1153615 function calls (1138771 primitive calls) in 4.364 seconds
>
>     Ordered by: internal time, call count
>     List reduced from 736 to 20 due to restriction <20>
>
>     ncalls  tottime  percall  cumtime  percall filename:lineno(function)
>       2099    0.419    0.000    0.425    0.000
> /usr/local/lib/python2.7/dist-packages/astropy/io/fits/header.py:1808(_updateindices)
>        346    0.312    0.001    0.852    0.002
> /usr/local/lib/python2.7/dist-packages/astropy/io/fits/column.py:1245(__getattr__)
>     124176    0.275    0.000    0.406    0.000
> /usr/local/lib/python2.7/dist-packages/astropy/io/fits/column.py:1285(__getitem__)
>        327    0.215    0.001    0.222    0.001
> /usr/local/lib/python2.7/dist-packages/django/db/backends/utils.py:58(execute)
>        357    0.201    0.001    0.293    0.001
> /usr/local/lib/python2.7/dist-packages/astropy/io/fits/header.py:1399(index)
>     137557    0.183    0.000    0.183    0.000
> /usr/local/lib/python2.7/dist-packages/astropy/io/fits/column.py:421(__get__)
>     124226    0.132    0.000    0.132    0.000
> /usr/local/lib/python2.7/dist-packages/astropy/io/fits/util.py:906(_is_int)
>
> My code is very simple however, it just creates necessary set of
> columns (astropy.table.MaskedColumn objects) casting data from python
> lists coming from the database to numpy arrays. Are there ways to save
> on checks like _is_int() in case the input array datatype has been
> already enforced?
>
> When the code is rewritten to use fitsio module
> (https://pypi.python.org/pypi/fitsio/), the execution time is
> significantly better:
>
>           129409 function calls (128693 primitive calls) in 0.811 seconds
>
>     Ordered by: internal time, call count
>     List reduced from 289 to 20 due to restriction <20>
>
>     ncalls  tottime  percall  cumtime  percall filename:lineno(function)
>        327    0.218    0.001    0.221    0.001
> /usr/local/lib/python2.7/dist-packages/django/db/backends/utils.py:58(execute)
>          4    0.083    0.021    0.083    0.021
> /usr/local/lib/python2.7/dist-packages/fitsio/fitslib.py:1311(_update_info)
>
> ...but fitsio does not seem to support 64 bit integers which is another
> requirement I have.
>
> What's the fastest solution to write FITS files that's available on
> the python market? Am I missing something in the astropy
> configuration?
>
> --
> With best regards,
>   Ivan
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org
> http://mail.scipy.org/mailman/listinfo/astropy
>



From derek at astro.physik.uni-goettingen.de  Mon Apr 20 07:32:00 2015
From: derek at astro.physik.uni-goettingen.de (Derek Homeier)
Date: Mon, 20 Apr 2015 13:32:00 +0200
Subject: [AstroPy] combining columns when writing a table to latex
In-Reply-To: <CAMER+OvKotvHWTpat7OBBhPZ=3RRRE68dK6o+E5-9qy8410QSg@mail.gmail.com>
References: <mailman.1.1429203601.2445.astropy@scipy.org>
	<552FEC09.5070902@gmx.de>
	<CAMER+OvKotvHWTpat7OBBhPZ=3RRRE68dK6o+E5-9qy8410QSg@mail.gmail.com>
Message-ID: <7D5B26D2-CBD6-4EC1-AA73-7E4CAA92EC72@astro.physik.uni-goettingen.de>


> On 20 Apr 2015, at 3:00 am, August (Gus) Muench <august.fly at gmail.com> wrote:
> 
> Just want to add to Moritz's point that \pm represents just one of many ways that authors format their tables for a nice print view but without thinking at all about reuse by others... 
> 
> It is true that many large tables are "normalized" by scientists at the AAS Journals (where I work) and CDS, but it is also true that breaking out of a print view of one's article benefits everyone immensely. 
> 
Wouldn?t this both be solved by keeping the errors in their own column and simply setting
\pm as a custom separator, e.g. \begin{tabular}{lccr@{$\pm$}lc}??
Disclaimer: I haven?t tested if this works with the col_align option of Astropy?s latex writer.

				Derek



From ivan.zolotukhin at gmail.com  Tue Apr 21 14:01:01 2015
From: ivan.zolotukhin at gmail.com (Ivan Zolotukhin)
Date: Tue, 21 Apr 2015 20:01:01 +0200
Subject: [AstroPy] writing FITS files fast
In-Reply-To: <CABOnLw2pY3z-4boWzZTbeXv6tZR4Xp02tiW7a1G0mc1oh9ncyw@mail.gmail.com>
References: <CAD=Ky+eNBQ6KxmUv1jGGaK+N7nYb+9pjHxG2Cb8mF5rfQe3CwQ@mail.gmail.com>
	<8738C82A-736E-4CBC-8616-EF495267AF9B@gmail.com>
	<CAD=Ky+dRTjnXxZpeMVPTQ+=eS2GCN90tO0fc1H9UB27qw7HZNg@mail.gmail.com>
	<CABOnLw2pY3z-4boWzZTbeXv6tZR4Xp02tiW7a1G0mc1oh9ncyw@mail.gmail.com>
Message-ID: <CAD=Ky+fnajeToWB+ibGakt4MBnZtRDaiLyN1UnA=XMoZ4CgkHQ@mail.gmail.com>

Hi,

> fitsio does support 64-bit.

Indeed, I was probably using it wrong, but maybe its error message can
be improved not to cause a confusion in similar cases, even though my
use case is somewhat weird. Here's the minimal test case I had
problems with (tested in numpy 1.8.1 and 1.9.2, fitsio 0.9.7):

```
import numpy as np
from fitsio import FITS

fits = FITS('test.fits', 'rw')
arr = [1L, 2L]
nparr = np.array(arr)
nulls = np.equal(arr, None)
nparr[nulls] = -1 # or whatever your FITS null value is
#arr = nparr.astype('int64') # uncomment this line and comment the
next one to make it work
arr = np.array(nparr, dtype='int64')
fits.write({'test': arr})
fits.close()
```

This results in:

```
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
/home/user/<ipython-input-92-b693c45fa56d> in <module>()
----> 1 fits.write({'test': arr})

/usr/local/lib/python2.7/dist-packages/fitsio/fitslib.pyc in
write(self, data, units, extname, extver, compress, tile_dims, header,
names, table_type, **keys)
    446                              extname=extname, extver=extver,
header=header,
    447                              names=names,
--> 448                              table_type=table_type)
    449
    450

/usr/local/lib/python2.7/dist-packages/fitsio/fitslib.pyc in
write_table(self, data, table_type, names, formats, units, extname,
extver, header)
    714             self[-1]._update_info()
    715
--> 716         self[-1].write(data,names=names)
    717
    718     def create_table_hdu(self, data=None, dtype=None,

/usr/local/lib/python2.7/dist-packages/fitsio/fitslib.pyc in
write(self, data, **keys)
   1506                 firstrow=keys.get('firstrow',0)
   1507                 self._FITS.write_columns(self._ext+1,
nonobj_colnums, nonobj_arrays,
-> 1508                                          firstrow=firstrow+1)
   1509
   1510         # writing the object arrays always occurs the same way


TypeError: Unsupported numpy table datatype 9
```

I'll explain why I needed this seemingly excessive numpy operations. I
have a list of long integers as an input and I need to check it for
None values (even though the test case does not have it), that's why I
make numpy array of objects `nparr` first and set Nones in it to a
dedicated integer value (let me know if there's a way to avoid this in
case your input has missing values). Then I naively expected that
doing `np.array(nparr, dtype='int64')` casts my array from numpy array
of objects to the array of long integers. I still do not get this
numpy magics, but if I change the type casting to the line which is
commented above `arr = nparr.astype('int64')` the fitsio error is
gone. I initially thought this weird problem has something to do with
int64 because changing data type in question to int32 also makes this
error disappear.

As for performance, with fitsio I'm now bottlenecked by my SQL
database which is how I expect things to happen with complex data
models. This is still not the case for astropy 1.0.2, which
bottlenecks my FITS writing procedure. I can prepare a test case if
someone is interested.

--
With best regards,
 Ivan


From erin.sheldon at gmail.com  Wed Apr 22 10:15:41 2015
From: erin.sheldon at gmail.com (Erin Sheldon)
Date: Wed, 22 Apr 2015 10:15:41 -0400
Subject: [AstroPy] writing FITS files fast
In-Reply-To: <CAD=Ky+fnajeToWB+ibGakt4MBnZtRDaiLyN1UnA=XMoZ4CgkHQ@mail.gmail.com>
References: <CAD=Ky+eNBQ6KxmUv1jGGaK+N7nYb+9pjHxG2Cb8mF5rfQe3CwQ@mail.gmail.com>
	<8738C82A-736E-4CBC-8616-EF495267AF9B@gmail.com>
	<CAD=Ky+dRTjnXxZpeMVPTQ+=eS2GCN90tO0fc1H9UB27qw7HZNg@mail.gmail.com>
	<CABOnLw2pY3z-4boWzZTbeXv6tZR4Xp02tiW7a1G0mc1oh9ncyw@mail.gmail.com>
	<CAD=Ky+fnajeToWB+ibGakt4MBnZtRDaiLyN1UnA=XMoZ4CgkHQ@mail.gmail.com>
Message-ID: <CABOnLw2n_KB8cQoToz=PJiqNANfMgJOYfdt5Bb_oaqqSGw_ewg@mail.gmail.com>

Ivan, I've opened an issue for you here

https://github.com/esheldon/fitsio/issues/41

I encourage you to get a github account so you can follow the progress.
-e

On 4/21/15, Ivan Zolotukhin <ivan.zolotukhin at gmail.com> wrote:
> Hi,
>
>> fitsio does support 64-bit.
>
> Indeed, I was probably using it wrong, but maybe its error message can
> be improved not to cause a confusion in similar cases, even though my
> use case is somewhat weird. Here's the minimal test case I had
> problems with (tested in numpy 1.8.1 and 1.9.2, fitsio 0.9.7):
>
> ```
> import numpy as np
> from fitsio import FITS
>
> fits = FITS('test.fits', 'rw')
> arr = [1L, 2L]
> nparr = np.array(arr)
> nulls = np.equal(arr, None)
> nparr[nulls] = -1 # or whatever your FITS null value is
> #arr = nparr.astype('int64') # uncomment this line and comment the
> next one to make it work
> arr = np.array(nparr, dtype='int64')
> fits.write({'test': arr})
> fits.close()
> ```
>
> This results in:
>
> ```
> ---------------------------------------------------------------------------
> TypeError                                 Traceback (most recent call last)
> /home/user/<ipython-input-92-b693c45fa56d> in <module>()
> ----> 1 fits.write({'test': arr})
>
> /usr/local/lib/python2.7/dist-packages/fitsio/fitslib.pyc in
> write(self, data, units, extname, extver, compress, tile_dims, header,
> names, table_type, **keys)
>     446                              extname=extname, extver=extver,
> header=header,
>     447                              names=names,
> --> 448                              table_type=table_type)
>     449
>     450
>
> /usr/local/lib/python2.7/dist-packages/fitsio/fitslib.pyc in
> write_table(self, data, table_type, names, formats, units, extname,
> extver, header)
>     714             self[-1]._update_info()
>     715
> --> 716         self[-1].write(data,names=names)
>     717
>     718     def create_table_hdu(self, data=None, dtype=None,
>
> /usr/local/lib/python2.7/dist-packages/fitsio/fitslib.pyc in
> write(self, data, **keys)
>    1506                 firstrow=keys.get('firstrow',0)
>    1507                 self._FITS.write_columns(self._ext+1,
> nonobj_colnums, nonobj_arrays,
> -> 1508                                          firstrow=firstrow+1)
>    1509
>    1510         # writing the object arrays always occurs the same way
>
>
> TypeError: Unsupported numpy table datatype 9
> ```
>
> I'll explain why I needed this seemingly excessive numpy operations. I
> have a list of long integers as an input and I need to check it for
> None values (even though the test case does not have it), that's why I
> make numpy array of objects `nparr` first and set Nones in it to a
> dedicated integer value (let me know if there's a way to avoid this in
> case your input has missing values). Then I naively expected that
> doing `np.array(nparr, dtype='int64')` casts my array from numpy array
> of objects to the array of long integers. I still do not get this
> numpy magics, but if I change the type casting to the line which is
> commented above `arr = nparr.astype('int64')` the fitsio error is
> gone. I initially thought this weird problem has something to do with
> int64 because changing data type in question to int32 also makes this
> error disappear.
>
> As for performance, with fitsio I'm now bottlenecked by my SQL
> database which is how I expect things to happen with complex data
> models. This is still not the case for astropy 1.0.2, which
> bottlenecks my FITS writing procedure. I can prepare a test case if
> someone is interested.
>
> --
> With best regards,
>  Ivan
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org
> http://mail.scipy.org/mailman/listinfo/astropy
>


-- 
Erin Scott Sheldon
Brookhaven National Laboratory erin dot sheldon at gmail dot com


From akshat.singhal014 at gmail.com  Wed Apr 22 19:57:40 2015
From: akshat.singhal014 at gmail.com (Akshat Singhal)
Date: Thu, 23 Apr 2015 05:27:40 +0530
Subject: [AstroPy] Montage_wrapper
Message-ID: <CAMYg6Pr7ckjxdR91780+6gkrOkmGskkJ0OD0wcCzKLyTK6M1vg@mail.gmail.com>

Sir/Ma'am,

I am using Montage and Montage_wrapper in astropy to co-add large set of
CRTS (Catalina Real-Time Transient Survey) images of different area of sky
for my project.
We wished to co-add all the images which overlap with certain region in the
sky (which is roughly 100s of images). We realized that to create a mosaic
(co-adding) of all those 100s of images combining all at once is a slower
process compare to combining few at once independently and then again
combining the resulted mosaic and so on.
What we noticed that when we co-add images which have no overlap with each
other (although have partial overlap with the region in sky), the montage
either returns a blank image or goes into infinite loop and eat up the
entire RAM. The likelihood of this event increases if the region of sky is
large and/or the number of images co-added at a time are less.
We couldn't figure out a way to fix this. Is there a possibility that when
given 2 non-overlapping images, montage recognize that it is the case and
avoid going to the infinite loop or simply stitch together those images.

Any insight would be a great help. If I failed to explain anything ,please
let me know

-Akshat Singhal
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/astropy/attachments/20150423/5eef2ed7/attachment.html>

From aniello.grado at gmail.com  Thu Apr 23 00:54:50 2015
From: aniello.grado at gmail.com (Aniello Grado)
Date: Thu, 23 Apr 2015 06:54:50 +0200
Subject: [AstroPy] Montage_wrapper
In-Reply-To: <CAMYg6Pr7ckjxdR91780+6gkrOkmGskkJ0OD0wcCzKLyTK6M1vg@mail.gmail.com>
References: <CAMYg6Pr7ckjxdR91780+6gkrOkmGskkJ0OD0wcCzKLyTK6M1vg@mail.gmail.com>
Message-ID: <CAOEf3j9_f+PAmpj=h5hfJQxLDB-ifSBJNZYdh-GZ6410gXRWfQ@mail.gmail.com>

Why you don't use SWarp?

Lino

On Thu, Apr 23, 2015 at 1:57 AM, Akshat Singhal <akshat.singhal014 at gmail.com
> wrote:

> Sir/Ma'am,
>
> I am using Montage and Montage_wrapper in astropy to co-add large set of
> CRTS (Catalina Real-Time Transient Survey) images of different area of sky
> for my project.
> We wished to co-add all the images which overlap with certain region in
> the sky (which is roughly 100s of images). We realized that to create a
> mosaic (co-adding) of all those 100s of images combining all at once is a
> slower  process compare to combining few at once independently and then
> again combining the resulted mosaic and so on.
> What we noticed that when we co-add images which have no overlap with each
> other (although have partial overlap with the region in sky), the montage
> either returns a blank image or goes into infinite loop and eat up the
> entire RAM. The likelihood of this event increases if the region of sky is
> large and/or the number of images co-added at a time are less.
> We couldn't figure out a way to fix this. Is there a possibility that when
> given 2 non-overlapping images, montage recognize that it is the case and
> avoid going to the infinite loop or simply stitch together those images.
>
> Any insight would be a great help. If I failed to explain anything ,please
> let me know
>
> -Akshat Singhal
>
>
>
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org
> http://mail.scipy.org/mailman/listinfo/astropy
>
>


-- 
<<<<<<<<<<<<<<<<<<<<>>>>>>>>>>>>>>>>>>>>>

Aniello Grado, PhD

Ricercatore Astronomo
INAF- Osservatorio Astronomico di Capodimonte
Via Moiariello 16 80131 Napoli
ITALIA
Tel: 0039 0815575547
Fax: 0039 0815575433

<<<<<<<<<<<<<<<<<<<<>>>>>>>>>>>>>>>>>>>>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/astropy/attachments/20150423/f0622f60/attachment.html>

From needfr at sina.com  Thu Apr 23 08:42:52 2015
From: needfr at sina.com (=?GBK?B?wO671A==?=)
Date: Thu, 23 Apr 2015 20:42:52 +0800 
Subject: [AstroPy] How to convert pixel position to latitude and longitude?
Message-ID: <20150423124252.7FC7A10BBD82@webmail.sinamail.sina.com.cn>

In HMI fits file , the data block represents the solar disk. I want to get the latitude and longitude of pixels on the solar disk.I thought the astropy.wcs package can do this, but it did not work! Perhaps , i did not find a right way to use it. Could you help me, thanks a lot! --------------------------------

NULL

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/astropy/attachments/20150423/b9751c05/attachment.html>

From mdroe at stsci.edu  Fri Apr 24 06:31:56 2015
From: mdroe at stsci.edu (Michael Droettboom)
Date: Fri, 24 Apr 2015 06:31:56 -0400
Subject: [AstroPy] How to convert pixel position to latitude and
	longitude?
In-Reply-To: <20150423124252.7FC7A10BBD82@webmail.sinamail.sina.com.cn>
References: <20150423124252.7FC7A10BBD82@webmail.sinamail.sina.com.cn>
Message-ID: <553A1B9C.2060606@stsci.edu>

Can you be more specific?  What did you try and what output did you get?

Cheers,
Mike

On 04/23/2015 08:42 AM, ?? wrote:
>
> In HMI fits file , the data block represents the solar disk. I want to 
> get the latitude and longitude of pixels on the solar disk.
>
> I thought the astropy.wcs package can do this, but it did not work! 
> Perhaps , i did not find a right way to use it. Could you help me, 
> thanks a lot!
>
> --------------------------------
>
> NULL
>
>
>
>
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org
> http://mail.scipy.org/mailman/listinfo/astropy

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/astropy/attachments/20150424/59d7c38d/attachment.html>

From leo.p.singer at nasa.gov  Fri Apr 24 13:06:17 2015
From: leo.p.singer at nasa.gov (Singer, Leo P. (GSFC-661.0)[OAK RIDGE ASSOCIATED UNIVERSITIES (ORAU)])
Date: Fri, 24 Apr 2015 17:06:17 +0000
Subject: [AstroPy] Enable doctests in Astropy-affiliated packages
Message-ID: <B831F6A6-C3AE-4E41-8258-AD3624FF49ED@nasa.gov>

Hi,

I am working on a pull request for an Astropy-affiliated package (https://github.com/astrofrog/reproject/pull/27). How do we enable doctests in this package using the Astropy testing framework?

Thanks,
Leo P. Singer
NASA Postdoctoral Program Fellow
Goddard Space Flight Center
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/astropy/attachments/20150424/b71cd88e/attachment.html>

From stuart at cadair.com  Mon Apr 27 04:53:27 2015
From: stuart at cadair.com (Stuart Mumford)
Date: Mon, 27 Apr 2015 09:53:27 +0100
Subject: [AstroPy] How to convert pixel position to latitude and
	longitude?
In-Reply-To: <20150423124252.7FC7A10BBD82@webmail.sinamail.sina.com.cn>
References: <20150423124252.7FC7A10BBD82@webmail.sinamail.sina.com.cn>
Message-ID: <553DF907.9010603@cadair.com>

Hello,

The SunPy library has the tools to do what you wish.

First create a SunPy map: 
http://docs.sunpy.org/en/latest/code_ref/map.html#creating-map-objects

Then extract the HPC coordinate for a given  pixel: 
http://docs.sunpy.org/en/latest/code_ref/map.html#sunpy.map.mapbase.GenericMap.pixel_to_data

Then convert that pixel to HGS lon/lat: 
http://docs.sunpy.org/en/latest/api/sunpy.wcs.convert_hpc_hg.html#sunpy.wcs.convert_hpc_hg

If you wish for more help, specific to the SunPy library, you might be 
interested in the sunpy mailing list: sunpy at googlegroups.com

Stuart

On 23/04/15 13:42, ?? wrote:
>
> In HMI fits file , the data block represents the solar disk. I want to 
> get the latitude and longitude of pixels on the solar disk.
>
> I thought the astropy.wcs package can do this, but it did not work! 
> Perhaps , i did not find a right way to use it. Could you help me, 
> thanks a lot!
>
> --------------------------------
>
> NULL
>
>
>
>
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org
> http://mail.scipy.org/mailman/listinfo/astropy

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/astropy/attachments/20150427/2710bd88/attachment.html>

From ejsaiet at alaska.edu  Mon Apr 27 05:15:35 2015
From: ejsaiet at alaska.edu (Eyal Saiet)
Date: Mon, 27 Apr 2015 01:15:35 -0800
Subject: [AstroPy] SAR processing tools in Python
Message-ID: <CA+ysReAxY_bBRZrGQevsx9xjsNshB9WoxrEY+dmsxg2bdE+GWQ@mail.gmail.com>

Hello,
Does Astropy or any other known python library support SAR processing that
you would recommend? Has anyone used pycsa(link
<https://code.google.com/p/pycsa/>)?
Thanks
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/astropy/attachments/20150427/7401156e/attachment.html>

From embray at stsci.edu  Mon Apr 27 09:08:06 2015
From: embray at stsci.edu (Erik Bray)
Date: Mon, 27 Apr 2015 09:08:06 -0400
Subject: [AstroPy] Enable doctests in Astropy-affiliated packages
In-Reply-To: <B831F6A6-C3AE-4E41-8258-AD3624FF49ED@nasa.gov>
References: <B831F6A6-C3AE-4E41-8258-AD3624FF49ED@nasa.gov>
Message-ID: <553E34B6.1050605@stsci.edu>

Hi Leo,

To enable doctests, make sure that the setup.cfg in the project's root contains 
the line:

doctest_plus = enabled

in the [pytest] section.

I'm not sure why that isn't there by default--maybe it should be.

Hope that helps,
Erik

(P.S. This sort of question is probably better suited to ask right on GitHub 
when working on the P.R., or on the astropy-dev mailing list, which is 
read/replied to by people doing active development on Astropy affiliated packages)

On 04/24/2015 01:06 PM, Singer, Leo P. (GSFC-661.0)[OAK RIDGE ASSOCIATED 
UNIVERSITIES (ORAU)] wrote:
> Hi,
>
> I am working on a pull request for an Astropy-affiliated package
> (https://github.com/astrofrog/reproject/pull/27). How do we enable doctests in
> this package using the Astropy testing framework?
>
> Thanks,
> Leo P. Singer
> NASA Postdoctoral Program Fellow
> Goddard Space Flight Center
>
>
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org
> http://mail.scipy.org/mailman/listinfo/astropy
>



From avillaum at ucsc.edu  Wed Apr 29 13:40:09 2015
From: avillaum at ucsc.edu (Alexa Villaume)
Date: Wed, 29 Apr 2015 10:40:09 -0700
Subject: [AstroPy] Constructing an astropy table with colnames and dtypes of
	existing table
Message-ID: <77C17E06-E1DD-4442-9BDD-1A8588EE52C8@ucsc.edu>

Hi everybody,

I?m trying to construct an astropy table using the exact same columns and corresponding dtypes of an existing table. The documentation goes into using existing columns but it wasn?t as clear to me on how to handle the data types. For example, I have an existing table ?irtf? and I want to make a table ?irtf_hb? if I do,

irtf_hb = Table(names=irtf.colnames) 

the table is initialized with all the right column names but the dtypes are all floats. However, I can?t do this,

irtf_hb = Table(names=irtf.colnames, dtype=irtf.dtype)

Because I get an error, ?ValueError: dtype must be a list or None?. I haven?t had any luck with trying to change the type of irtf.dtype or using list comprehension to extract a list of dtypes. 

Is there a simple way to use the dtypes of an existing table to initialize a new table?

Thank you,
Alexa

From andrew.hearin at yale.edu  Wed Apr 29 13:53:13 2015
From: andrew.hearin at yale.edu (Andrew Hearin)
Date: Wed, 29 Apr 2015 13:53:13 -0400
Subject: [AstroPy] Constructing an astropy table with colnames and
	dtypes of existing table
In-Reply-To: <77C17E06-E1DD-4442-9BDD-1A8588EE52C8@ucsc.edu>
References: <77C17E06-E1DD-4442-9BDD-1A8588EE52C8@ucsc.edu>
Message-ID: <25726754-672E-4233-89D7-A279282AAB48@yale.edu>

Hi Alexa,

If your table instances is called t, how about the following:

t2 = Table(np.empty_like(t.as_array()))

Cheers,
Andrew


On Apr 29, 2015, at 1:40 PM, Alexa Villaume <avillaum at ucsc.edu> wrote:

> Hi everybody,
> 
> I?m trying to construct an astropy table using the exact same columns and corresponding dtypes of an existing table. The documentation goes into using existing columns but it wasn?t as clear to me on how to handle the data types. For example, I have an existing table ?irtf? and I want to make a table ?irtf_hb? if I do,
> 
> irtf_hb = Table(names=irtf.colnames) 
> 
> the table is initialized with all the right column names but the dtypes are all floats. However, I can?t do this,
> 
> irtf_hb = Table(names=irtf.colnames, dtype=irtf.dtype)
> 
> Because I get an error, ?ValueError: dtype must be a list or None?. I haven?t had any luck with trying to change the type of irtf.dtype or using list comprehension to extract a list of dtypes. 
> 
> Is there a simple way to use the dtypes of an existing table to initialize a new table?
> 
> Thank you,
> Alexa
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org
> https://urldefense.proofpoint.com/v2/url?u=http-3A__mail.scipy.org_mailman_listinfo_astropy&d=AwIF-g&c=-dg2m7zWuuDZ0MUcV7Sdqw&r=AHkQ8HPUDwzl0x62ybAnwN_OEebPRGDtcjUPBcnLYw4&m=rGucwbRRBSwsQ4tj_W9Ic1p8QMfz7AiI-PqwLZDvry4&s=54VhWWCMNDv_liugJ2cksXWVx4hg1OgEDLTEQH_EOk8&e= 



From thomas.robitaille at gmail.com  Wed Apr 29 14:08:25 2015
From: thomas.robitaille at gmail.com (Thomas Robitaille)
Date: Wed, 29 Apr 2015 19:08:25 +0100
Subject: [AstroPy] Constructing an astropy table with colnames and
 dtypes of existing table
In-Reply-To: <77C17E06-E1DD-4442-9BDD-1A8588EE52C8@ucsc.edu>
References: <77C17E06-E1DD-4442-9BDD-1A8588EE52C8@ucsc.edu>
Message-ID: <55411E19.1050104@gmail.com>

Hi Alexa,

First, I think it might make sense for us to fix Table to allow Numpy
dtypes to be passed to dtype= in Table (the only subtlety is that if a
dtype with field names is passed in addition to names= we need to decide
which takes precedence). Maybe Tom Aldcroft can comment on this?

In the mean time I would suggest the easiest way to do this is to
separate the field names from the types using the following:

names, types = zip(*irtf.dtype.descr)

then you can pass 'types' to dtype in the Table constructor. This will
create an empty table with no rows (as opposed to Andrew's example which
will create a table with the same number of rows).

Cheers,
Tom

Alexa Villaume wrote:
> Hi everybody,
> 
> I?m trying to construct an astropy table using the exact same columns and corresponding dtypes of an existing table. The documentation goes into using existing columns but it wasn?t as clear to me on how to handle the data types. For example, I have an existing table ?irtf? and I want to make a table ?irtf_hb? if I do,
> 
> irtf_hb = Table(names=irtf.colnames) 
> 
> the table is initialized with all the right column names but the dtypes are all floats. However, I can?t do this,
> 
> irtf_hb = Table(names=irtf.colnames, dtype=irtf.dtype)
> 
> Because I get an error, ?ValueError: dtype must be a list or None?. I haven?t had any luck with trying to change the type of irtf.dtype or using list comprehension to extract a list of dtypes. 
> 
> Is there a simple way to use the dtypes of an existing table to initialize a new table?
> 
> Thank you,
> Alexa
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org
> http://mail.scipy.org/mailman/listinfo/astropy


From aldcroft at head.cfa.harvard.edu  Wed Apr 29 14:13:26 2015
From: aldcroft at head.cfa.harvard.edu (Aldcroft, Thomas)
Date: Wed, 29 Apr 2015 14:13:26 -0400
Subject: [AstroPy] Constructing an astropy table with colnames and
 dtypes of existing table
In-Reply-To: <77C17E06-E1DD-4442-9BDD-1A8588EE52C8@ucsc.edu>
References: <77C17E06-E1DD-4442-9BDD-1A8588EE52C8@ucsc.edu>
Message-ID: <CAMtEP6wRi7NnaNx6h-o_SG+Qxdij-vZ+dyBWA_7poeqthJW3+Q@mail.gmail.com>

On Wed, Apr 29, 2015 at 1:40 PM, Alexa Villaume <avillaum at ucsc.edu> wrote:

> Hi everybody,
>
> I?m trying to construct an astropy table using the exact same columns and
> corresponding dtypes of an existing table. The documentation goes into
> using existing columns but it wasn?t as clear to me on how to handle the
> data types. For example, I have an existing table ?irtf? and I want to make
> a table ?irtf_hb? if I do,
>
> irtf_hb = Table(names=irtf.colnames)
>
> the table is initialized with all the right column names but the dtypes
> are all floats. However, I can?t do this,
>
> irtf_hb = Table(names=irtf.colnames, dtype=irtf.dtype)
>
> Because I get an error, ?ValueError: dtype must be a list or None?. I
> haven?t had any luck with trying to change the type of irtf.dtype or using
> list comprehension to extract a list of dtypes.
>
> Is there a simple way to use the dtypes of an existing table to initialize
> a new table?
>

You should be able to do:

>>> irtf_hb = Table(irtf[0:0])

This is a slight hack, but it's pretty simple and should give you a
zero-length table with all the original table properties.  I tested on a
simple case but haven't checked in detail.

In [3]: from astropy.table import Table

In [4]: t = Table([[1]])

In [5]: t2 = Table(t[0:0])

In [6]: t2
Out[6]:
<Table masked=False length=0>
 col0
int64
-----

In [7]:

- Tom


>
> Thank you,
> Alexa
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org
> http://mail.scipy.org/mailman/listinfo/astropy
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/astropy/attachments/20150429/402118f3/attachment.html>

From aldcroft at head.cfa.harvard.edu  Wed Apr 29 14:19:06 2015
From: aldcroft at head.cfa.harvard.edu (Aldcroft, Thomas)
Date: Wed, 29 Apr 2015 14:19:06 -0400
Subject: [AstroPy] Constructing an astropy table with colnames and
 dtypes of existing table
In-Reply-To: <CAMtEP6wRi7NnaNx6h-o_SG+Qxdij-vZ+dyBWA_7poeqthJW3+Q@mail.gmail.com>
References: <77C17E06-E1DD-4442-9BDD-1A8588EE52C8@ucsc.edu>
	<CAMtEP6wRi7NnaNx6h-o_SG+Qxdij-vZ+dyBWA_7poeqthJW3+Q@mail.gmail.com>
Message-ID: <CAMtEP6y1cLeYqd6fkCVFJJnK4GbbZwLu=1aWC69HNTbHT6nS-A@mail.gmail.com>

On Wed, Apr 29, 2015 at 2:13 PM, Aldcroft, Thomas <
aldcroft at head.cfa.harvard.edu> wrote:

>
>
> On Wed, Apr 29, 2015 at 1:40 PM, Alexa Villaume <avillaum at ucsc.edu> wrote:
>
>> Hi everybody,
>>
>> I?m trying to construct an astropy table using the exact same columns and
>> corresponding dtypes of an existing table. The documentation goes into
>> using existing columns but it wasn?t as clear to me on how to handle the
>> data types. For example, I have an existing table ?irtf? and I want to make
>> a table ?irtf_hb? if I do,
>>
>> irtf_hb = Table(names=irtf.colnames)
>>
>> the table is initialized with all the right column names but the dtypes
>> are all floats. However, I can?t do this,
>>
>> irtf_hb = Table(names=irtf.colnames, dtype=irtf.dtype)
>>
>> Because I get an error, ?ValueError: dtype must be a list or None?. I
>> haven?t had any luck with trying to change the type of irtf.dtype or using
>> list comprehension to extract a list of dtypes.
>>
>> Is there a simple way to use the dtypes of an existing table to
>> initialize a new table?
>>
>
> You should be able to do:
>
> >>> irtf_hb = Table(irtf[0:0])
>
> This is a slight hack, but it's pretty simple and should give you a
> zero-length table with all the original table properties.  I tested on a
> simple case but haven't checked in detail.
>

BTW, "all the original table properties" includes all column and table meta
data (e.g. units), so this should be the preferred idiom.

- Tom


>
> In [3]: from astropy.table import Table
>
> In [4]: t = Table([[1]])
>
> In [5]: t2 = Table(t[0:0])
>
> In [6]: t2
> Out[6]:
> <Table masked=False length=0>
>  col0
> int64
> -----
>
> In [7]:
>
> - Tom
>
>
>>
>> Thank you,
>> Alexa
>> _______________________________________________
>> AstroPy mailing list
>> AstroPy at scipy.org
>> http://mail.scipy.org/mailman/listinfo/astropy
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/astropy/attachments/20150429/fee06568/attachment.html>

From thomas.robitaille at gmail.com  Wed Apr 29 14:21:52 2015
From: thomas.robitaille at gmail.com (Thomas Robitaille)
Date: Wed, 29 Apr 2015 19:21:52 +0100
Subject: [AstroPy] Constructing an astropy table with colnames and
 dtypes of existing table
In-Reply-To: <CAMtEP6y1cLeYqd6fkCVFJJnK4GbbZwLu=1aWC69HNTbHT6nS-A@mail.gmail.com>
References: <77C17E06-E1DD-4442-9BDD-1A8588EE52C8@ucsc.edu>
	<CAMtEP6wRi7NnaNx6h-o_SG+Qxdij-vZ+dyBWA_7poeqthJW3+Q@mail.gmail.com>
	<CAMtEP6y1cLeYqd6fkCVFJJnK4GbbZwLu=1aWC69HNTbHT6nS-A@mail.gmail.com>
Message-ID: <CAGMHX_0kxZ5-UdKv24VRBacJaCj0a_Lwzh+HcLsXwxcBWKGS5Q@mail.gmail.com>

On 29 April 2015 at 19:19, Aldcroft, Thomas
<aldcroft at head.cfa.harvard.edu> wrote:
>
>
> On Wed, Apr 29, 2015 at 2:13 PM, Aldcroft, Thomas
> <aldcroft at head.cfa.harvard.edu> wrote:
>>
>>
>>
>> On Wed, Apr 29, 2015 at 1:40 PM, Alexa Villaume <avillaum at ucsc.edu> wrote:
>>>
>>> Hi everybody,
>>>
>>> I?m trying to construct an astropy table using the exact same columns and
>>> corresponding dtypes of an existing table. The documentation goes into using
>>> existing columns but it wasn?t as clear to me on how to handle the data
>>> types. For example, I have an existing table ?irtf? and I want to make a
>>> table ?irtf_hb? if I do,
>>>
>>> irtf_hb = Table(names=irtf.colnames)
>>>
>>> the table is initialized with all the right column names but the dtypes
>>> are all floats. However, I can?t do this,
>>>
>>> irtf_hb = Table(names=irtf.colnames, dtype=irtf.dtype)
>>>
>>> Because I get an error, ?ValueError: dtype must be a list or None?. I
>>> haven?t had any luck with trying to change the type of irtf.dtype or using
>>> list comprehension to extract a list of dtypes.
>>>
>>> Is there a simple way to use the dtypes of an existing table to
>>> initialize a new table?
>>
>>
>> You should be able to do:
>>
>> >>> irtf_hb = Table(irtf[0:0])
>>
>> This is a slight hack, but it's pretty simple and should give you a
>> zero-length table with all the original table properties.  I tested on a
>> simple case but haven't checked in detail.
>
>
> BTW, "all the original table properties" includes all column and table meta
> data (e.g. units), so this should be the preferred idiom.

How about a new class method that does something like
``np.empty_like`` called ``Table.empty_like``?

Cheers,
Tom

>
> - Tom
>
>>
>>
>> In [3]: from astropy.table import Table
>>
>> In [4]: t = Table([[1]])
>>
>> In [5]: t2 = Table(t[0:0])
>>
>> In [6]: t2
>> Out[6]:
>> <Table masked=False length=0>
>>  col0
>> int64
>> -----
>>
>> In [7]:
>>
>> - Tom
>>
>>>
>>>
>>> Thank you,
>>> Alexa
>>> _______________________________________________
>>> AstroPy mailing list
>>> AstroPy at scipy.org
>>> http://mail.scipy.org/mailman/listinfo/astropy
>>
>>
>
>
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org
> http://mail.scipy.org/mailman/listinfo/astropy
>


From andrew.hearin at yale.edu  Wed Apr 29 14:45:24 2015
From: andrew.hearin at yale.edu (Andrew Hearin)
Date: Wed, 29 Apr 2015 14:45:24 -0400
Subject: [AstroPy] Constructing an astropy table with colnames and
	dtypes of existing table
In-Reply-To: <CAGMHX_0kxZ5-UdKv24VRBacJaCj0a_Lwzh+HcLsXwxcBWKGS5Q@mail.gmail.com>
References: <77C17E06-E1DD-4442-9BDD-1A8588EE52C8@ucsc.edu>
	<CAMtEP6wRi7NnaNx6h-o_SG+Qxdij-vZ+dyBWA_7poeqthJW3+Q@mail.gmail.com>
	<CAMtEP6y1cLeYqd6fkCVFJJnK4GbbZwLu=1aWC69HNTbHT6nS-A@mail.gmail.com>
	<CAGMHX_0kxZ5-UdKv24VRBacJaCj0a_Lwzh+HcLsXwxcBWKGS5Q@mail.gmail.com>
Message-ID: <1199E149-0A48-44E7-8CF8-1ED3FCA720B1@yale.edu>

You beat me to the punch, Tom, I was going to suggest exactly this. Numpys built-in empty_like method is extremely convenient, and it would be great to have an analogous astropy table method. 


On Apr 29, 2015, at 2:21 PM, Thomas Robitaille <thomas.robitaille at gmail.com> wrote:

> On 29 April 2015 at 19:19, Aldcroft, Thomas
> <aldcroft at head.cfa.harvard.edu> wrote:
>> 
>> 
>> On Wed, Apr 29, 2015 at 2:13 PM, Aldcroft, Thomas
>> <aldcroft at head.cfa.harvard.edu> wrote:
>>> 
>>> 
>>> 
>>> On Wed, Apr 29, 2015 at 1:40 PM, Alexa Villaume <avillaum at ucsc.edu> wrote:
>>>> 
>>>> Hi everybody,
>>>> 
>>>> I?m trying to construct an astropy table using the exact same columns and
>>>> corresponding dtypes of an existing table. The documentation goes into using
>>>> existing columns but it wasn?t as clear to me on how to handle the data
>>>> types. For example, I have an existing table ?irtf? and I want to make a
>>>> table ?irtf_hb? if I do,
>>>> 
>>>> irtf_hb = Table(names=irtf.colnames)
>>>> 
>>>> the table is initialized with all the right column names but the dtypes
>>>> are all floats. However, I can?t do this,
>>>> 
>>>> irtf_hb = Table(names=irtf.colnames, dtype=irtf.dtype)
>>>> 
>>>> Because I get an error, ?ValueError: dtype must be a list or None?. I
>>>> haven?t had any luck with trying to change the type of irtf.dtype or using
>>>> list comprehension to extract a list of dtypes.
>>>> 
>>>> Is there a simple way to use the dtypes of an existing table to
>>>> initialize a new table?
>>> 
>>> 
>>> You should be able to do:
>>> 
>>>>>> irtf_hb = Table(irtf[0:0])
>>> 
>>> This is a slight hack, but it's pretty simple and should give you a
>>> zero-length table with all the original table properties.  I tested on a
>>> simple case but haven't checked in detail.
>> 
>> 
>> BTW, "all the original table properties" includes all column and table meta
>> data (e.g. units), so this should be the preferred idiom.
> 
> How about a new class method that does something like
> ``np.empty_like`` called ``Table.empty_like``?
> 
> Cheers,
> Tom
> 
>> 
>> - Tom
>> 
>>> 
>>> 
>>> In [3]: from astropy.table import Table
>>> 
>>> In [4]: t = Table([[1]])
>>> 
>>> In [5]: t2 = Table(t[0:0])
>>> 
>>> In [6]: t2
>>> Out[6]:
>>> <Table masked=False length=0>
>>> col0
>>> int64
>>> -----
>>> 
>>> In [7]:
>>> 
>>> - Tom
>>> 
>>>> 
>>>> 
>>>> Thank you,
>>>> Alexa
>>>> _______________________________________________
>>>> AstroPy mailing list
>>>> AstroPy at scipy.org
>>>> https://urldefense.proofpoint.com/v2/url?u=http-3A__mail.scipy.org_mailman_listinfo_astropy&d=AwIGaQ&c=-dg2m7zWuuDZ0MUcV7Sdqw&r=AHkQ8HPUDwzl0x62ybAnwN_OEebPRGDtcjUPBcnLYw4&m=tsHzQ6NpihEoX33pmnmiHF6nSNu86BozkIV4v_KCP6s&s=FRlFhyfQHtHEHMGpEAq6bnpkDKvPbatH7Q5WJ492XNU&e= 
>>> 
>>> 
>> 
>> 
>> _______________________________________________
>> AstroPy mailing list
>> AstroPy at scipy.org
>> https://urldefense.proofpoint.com/v2/url?u=http-3A__mail.scipy.org_mailman_listinfo_astropy&d=AwIGaQ&c=-dg2m7zWuuDZ0MUcV7Sdqw&r=AHkQ8HPUDwzl0x62ybAnwN_OEebPRGDtcjUPBcnLYw4&m=tsHzQ6NpihEoX33pmnmiHF6nSNu86BozkIV4v_KCP6s&s=FRlFhyfQHtHEHMGpEAq6bnpkDKvPbatH7Q5WJ492XNU&e= 
>> 
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org
> https://urldefense.proofpoint.com/v2/url?u=http-3A__mail.scipy.org_mailman_listinfo_astropy&d=AwIGaQ&c=-dg2m7zWuuDZ0MUcV7Sdqw&r=AHkQ8HPUDwzl0x62ybAnwN_OEebPRGDtcjUPBcnLYw4&m=tsHzQ6NpihEoX33pmnmiHF6nSNu86BozkIV4v_KCP6s&s=FRlFhyfQHtHEHMGpEAq6bnpkDKvPbatH7Q5WJ492XNU&e=

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/astropy/attachments/20150429/a7b41f97/attachment.html>

From avillaum at ucsc.edu  Wed Apr 29 14:57:22 2015
From: avillaum at ucsc.edu (Alexa Villaume)
Date: Wed, 29 Apr 2015 11:57:22 -0700
Subject: [AstroPy] Constructing an astropy table with colnames and
	dtypes of existing table
In-Reply-To: <1199E149-0A48-44E7-8CF8-1ED3FCA720B1@yale.edu>
References: <77C17E06-E1DD-4442-9BDD-1A8588EE52C8@ucsc.edu>
	<CAMtEP6wRi7NnaNx6h-o_SG+Qxdij-vZ+dyBWA_7poeqthJW3+Q@mail.gmail.com>
	<CAMtEP6y1cLeYqd6fkCVFJJnK4GbbZwLu=1aWC69HNTbHT6nS-A@mail.gmail.com>
	<CAGMHX_0kxZ5-UdKv24VRBacJaCj0a_Lwzh+HcLsXwxcBWKGS5Q@mail.gmail.com>
	<1199E149-0A48-44E7-8CF8-1ED3FCA720B1@yale.edu>
Message-ID: <17F13F86-8B9F-4292-B9F6-3F49CCC5F8AB@ucsc.edu>

Thank you, everyone, for your helpful responses. 

If it?s of any interest, I ended up going with Tom A.?s solution because one of my columns holds arrays instead of single values. The dtype entry for that looks like ?(?SPEC?, ?>f4?, (7925,)? which was fully copied in Tom A.?s solution but not Tom R.?s. 

Best,
Alexa

On Apr 29, 2015, at 11:45 AM, Andrew Hearin <andrew.hearin at yale.edu> wrote:

> You beat me to the punch, Tom, I was going to suggest exactly this. Numpys built-in empty_like method is extremely convenient, and it would be great to have an analogous astropy table method. 
> 
> 
> On Apr 29, 2015, at 2:21 PM, Thomas Robitaille <thomas.robitaille at gmail.com> wrote:
> 
>> On 29 April 2015 at 19:19, Aldcroft, Thomas
>> <aldcroft at head.cfa.harvard.edu> wrote:
>>> 
>>> 
>>> On Wed, Apr 29, 2015 at 2:13 PM, Aldcroft, Thomas
>>> <aldcroft at head.cfa.harvard.edu> wrote:
>>>> 
>>>> 
>>>> 
>>>> On Wed, Apr 29, 2015 at 1:40 PM, Alexa Villaume <avillaum at ucsc.edu> wrote:
>>>>> 
>>>>> Hi everybody,
>>>>> 
>>>>> I?m trying to construct an astropy table using the exact same columns and
>>>>> corresponding dtypes of an existing table. The documentation goes into using
>>>>> existing columns but it wasn?t as clear to me on how to handle the data
>>>>> types. For example, I have an existing table ?irtf? and I want to make a
>>>>> table ?irtf_hb? if I do,
>>>>> 
>>>>> irtf_hb = Table(names=irtf.colnames)
>>>>> 
>>>>> the table is initialized with all the right column names but the dtypes
>>>>> are all floats. However, I can?t do this,
>>>>> 
>>>>> irtf_hb = Table(names=irtf.colnames, dtype=irtf.dtype)
>>>>> 
>>>>> Because I get an error, ?ValueError: dtype must be a list or None?. I
>>>>> haven?t had any luck with trying to change the type of irtf.dtype or using
>>>>> list comprehension to extract a list of dtypes.
>>>>> 
>>>>> Is there a simple way to use the dtypes of an existing table to
>>>>> initialize a new table?
>>>> 
>>>> 
>>>> You should be able to do:
>>>> 
>>>>>>> irtf_hb = Table(irtf[0:0])
>>>> 
>>>> This is a slight hack, but it's pretty simple and should give you a
>>>> zero-length table with all the original table properties.  I tested on a
>>>> simple case but haven't checked in detail.
>>> 
>>> 
>>> BTW, "all the original table properties" includes all column and table meta
>>> data (e.g. units), so this should be the preferred idiom.
>> 
>> How about a new class method that does something like
>> ``np.empty_like`` called ``Table.empty_like``?
>> 
>> Cheers,
>> Tom
>> 
>>> 
>>> - Tom
>>> 
>>>> 
>>>> 
>>>> In [3]: from astropy.table import Table
>>>> 
>>>> In [4]: t = Table([[1]])
>>>> 
>>>> In [5]: t2 = Table(t[0:0])
>>>> 
>>>> In [6]: t2
>>>> Out[6]:
>>>> <Table masked=False length=0>
>>>> col0
>>>> int64
>>>> -----
>>>> 
>>>> In [7]:
>>>> 
>>>> - Tom
>>>> 
>>>>> 
>>>>> 
>>>>> Thank you,
>>>>> Alexa
>>>>> _______________________________________________
>>>>> AstroPy mailing list
>>>>> AstroPy at scipy.org
>>>>> https://urldefense.proofpoint.com/v2/url?u=http-3A__mail.scipy.org_mailman_listinfo_astropy&d=AwIGaQ&c=-dg2m7zWuuDZ0MUcV7Sdqw&r=AHkQ8HPUDwzl0x62ybAnwN_OEebPRGDtcjUPBcnLYw4&m=tsHzQ6NpihEoX33pmnmiHF6nSNu86BozkIV4v_KCP6s&s=FRlFhyfQHtHEHMGpEAq6bnpkDKvPbatH7Q5WJ492XNU&e= 
>>>> 
>>>> 
>>> 
>>> 
>>> _______________________________________________
>>> AstroPy mailing list
>>> AstroPy at scipy.org
>>> https://urldefense.proofpoint.com/v2/url?u=http-3A__mail.scipy.org_mailman_listinfo_astropy&d=AwIGaQ&c=-dg2m7zWuuDZ0MUcV7Sdqw&r=AHkQ8HPUDwzl0x62ybAnwN_OEebPRGDtcjUPBcnLYw4&m=tsHzQ6NpihEoX33pmnmiHF6nSNu86BozkIV4v_KCP6s&s=FRlFhyfQHtHEHMGpEAq6bnpkDKvPbatH7Q5WJ492XNU&e= 
>>> 
>> _______________________________________________
>> AstroPy mailing list
>> AstroPy at scipy.org
>> https://urldefense.proofpoint.com/v2/url?u=http-3A__mail.scipy.org_mailman_listinfo_astropy&d=AwIGaQ&c=-dg2m7zWuuDZ0MUcV7Sdqw&r=AHkQ8HPUDwzl0x62ybAnwN_OEebPRGDtcjUPBcnLYw4&m=tsHzQ6NpihEoX33pmnmiHF6nSNu86BozkIV4v_KCP6s&s=FRlFhyfQHtHEHMGpEAq6bnpkDKvPbatH7Q5WJ492XNU&e=
> 
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org
> http://mail.scipy.org/mailman/listinfo/astropy



From aldcroft at head.cfa.harvard.edu  Wed Apr 29 15:11:46 2015
From: aldcroft at head.cfa.harvard.edu (Aldcroft, Thomas)
Date: Wed, 29 Apr 2015 15:11:46 -0400
Subject: [AstroPy] Constructing an astropy table with colnames and
 dtypes of existing table
In-Reply-To: <17F13F86-8B9F-4292-B9F6-3F49CCC5F8AB@ucsc.edu>
References: <77C17E06-E1DD-4442-9BDD-1A8588EE52C8@ucsc.edu>
	<CAMtEP6wRi7NnaNx6h-o_SG+Qxdij-vZ+dyBWA_7poeqthJW3+Q@mail.gmail.com>
	<CAMtEP6y1cLeYqd6fkCVFJJnK4GbbZwLu=1aWC69HNTbHT6nS-A@mail.gmail.com>
	<CAGMHX_0kxZ5-UdKv24VRBacJaCj0a_Lwzh+HcLsXwxcBWKGS5Q@mail.gmail.com>
	<1199E149-0A48-44E7-8CF8-1ED3FCA720B1@yale.edu>
	<17F13F86-8B9F-4292-B9F6-3F49CCC5F8AB@ucsc.edu>
Message-ID: <CAMtEP6yU6RQMK4R-6rc7zXxzdvKJmxG6d2F4phSvrE5icWwbXQ@mail.gmail.com>

If anyone is interested in continuing the discussion on github, see:

https://github.com/astropy/astropy/issues/3742

- Tom

On Wed, Apr 29, 2015 at 2:57 PM, Alexa Villaume <avillaum at ucsc.edu> wrote:

> Thank you, everyone, for your helpful responses.
>
> If it?s of any interest, I ended up going with Tom A.?s solution because
> one of my columns holds arrays instead of single values. The dtype entry
> for that looks like ?(?SPEC?, ?>f4?, (7925,)? which was fully copied in Tom
> A.?s solution but not Tom R.?s.
>
> Best,
> Alexa
>
> On Apr 29, 2015, at 11:45 AM, Andrew Hearin <andrew.hearin at yale.edu>
> wrote:
>
> > You beat me to the punch, Tom, I was going to suggest exactly this.
> Numpys built-in empty_like method is extremely convenient, and it would be
> great to have an analogous astropy table method.
> >
> >
> > On Apr 29, 2015, at 2:21 PM, Thomas Robitaille <
> thomas.robitaille at gmail.com> wrote:
> >
> >> On 29 April 2015 at 19:19, Aldcroft, Thomas
> >> <aldcroft at head.cfa.harvard.edu> wrote:
> >>>
> >>>
> >>> On Wed, Apr 29, 2015 at 2:13 PM, Aldcroft, Thomas
> >>> <aldcroft at head.cfa.harvard.edu> wrote:
> >>>>
> >>>>
> >>>>
> >>>> On Wed, Apr 29, 2015 at 1:40 PM, Alexa Villaume <avillaum at ucsc.edu>
> wrote:
> >>>>>
> >>>>> Hi everybody,
> >>>>>
> >>>>> I?m trying to construct an astropy table using the exact same
> columns and
> >>>>> corresponding dtypes of an existing table. The documentation goes
> into using
> >>>>> existing columns but it wasn?t as clear to me on how to handle the
> data
> >>>>> types. For example, I have an existing table ?irtf? and I want to
> make a
> >>>>> table ?irtf_hb? if I do,
> >>>>>
> >>>>> irtf_hb = Table(names=irtf.colnames)
> >>>>>
> >>>>> the table is initialized with all the right column names but the
> dtypes
> >>>>> are all floats. However, I can?t do this,
> >>>>>
> >>>>> irtf_hb = Table(names=irtf.colnames, dtype=irtf.dtype)
> >>>>>
> >>>>> Because I get an error, ?ValueError: dtype must be a list or None?. I
> >>>>> haven?t had any luck with trying to change the type of irtf.dtype or
> using
> >>>>> list comprehension to extract a list of dtypes.
> >>>>>
> >>>>> Is there a simple way to use the dtypes of an existing table to
> >>>>> initialize a new table?
> >>>>
> >>>>
> >>>> You should be able to do:
> >>>>
> >>>>>>> irtf_hb = Table(irtf[0:0])
> >>>>
> >>>> This is a slight hack, but it's pretty simple and should give you a
> >>>> zero-length table with all the original table properties.  I tested
> on a
> >>>> simple case but haven't checked in detail.
> >>>
> >>>
> >>> BTW, "all the original table properties" includes all column and table
> meta
> >>> data (e.g. units), so this should be the preferred idiom.
> >>
> >> How about a new class method that does something like
> >> ``np.empty_like`` called ``Table.empty_like``?
> >>
> >> Cheers,
> >> Tom
> >>
> >>>
> >>> - Tom
> >>>
> >>>>
> >>>>
> >>>> In [3]: from astropy.table import Table
> >>>>
> >>>> In [4]: t = Table([[1]])
> >>>>
> >>>> In [5]: t2 = Table(t[0:0])
> >>>>
> >>>> In [6]: t2
> >>>> Out[6]:
> >>>> <Table masked=False length=0>
> >>>> col0
> >>>> int64
> >>>> -----
> >>>>
> >>>> In [7]:
> >>>>
> >>>> - Tom
> >>>>
> >>>>>
> >>>>>
> >>>>> Thank you,
> >>>>> Alexa
> >>>>> _______________________________________________
> >>>>> AstroPy mailing list
> >>>>> AstroPy at scipy.org
> >>>>>
> https://urldefense.proofpoint.com/v2/url?u=http-3A__mail.scipy.org_mailman_listinfo_astropy&d=AwIGaQ&c=-dg2m7zWuuDZ0MUcV7Sdqw&r=AHkQ8HPUDwzl0x62ybAnwN_OEebPRGDtcjUPBcnLYw4&m=tsHzQ6NpihEoX33pmnmiHF6nSNu86BozkIV4v_KCP6s&s=FRlFhyfQHtHEHMGpEAq6bnpkDKvPbatH7Q5WJ492XNU&e=
> >>>>
> >>>>
> >>>
> >>>
> >>> _______________________________________________
> >>> AstroPy mailing list
> >>> AstroPy at scipy.org
> >>>
> https://urldefense.proofpoint.com/v2/url?u=http-3A__mail.scipy.org_mailman_listinfo_astropy&d=AwIGaQ&c=-dg2m7zWuuDZ0MUcV7Sdqw&r=AHkQ8HPUDwzl0x62ybAnwN_OEebPRGDtcjUPBcnLYw4&m=tsHzQ6NpihEoX33pmnmiHF6nSNu86BozkIV4v_KCP6s&s=FRlFhyfQHtHEHMGpEAq6bnpkDKvPbatH7Q5WJ492XNU&e=
> >>>
> >> _______________________________________________
> >> AstroPy mailing list
> >> AstroPy at scipy.org
> >>
> https://urldefense.proofpoint.com/v2/url?u=http-3A__mail.scipy.org_mailman_listinfo_astropy&d=AwIGaQ&c=-dg2m7zWuuDZ0MUcV7Sdqw&r=AHkQ8HPUDwzl0x62ybAnwN_OEebPRGDtcjUPBcnLYw4&m=tsHzQ6NpihEoX33pmnmiHF6nSNu86BozkIV4v_KCP6s&s=FRlFhyfQHtHEHMGpEAq6bnpkDKvPbatH7Q5WJ492XNU&e=
> >
> > _______________________________________________
> > AstroPy mailing list
> > AstroPy at scipy.org
> > http://mail.scipy.org/mailman/listinfo/astropy
>
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org
> http://mail.scipy.org/mailman/listinfo/astropy
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/astropy/attachments/20150429/ea074f45/attachment.html>

From embray at stsci.edu  Wed Apr 29 16:20:52 2015
From: embray at stsci.edu (Erik Bray)
Date: Wed, 29 Apr 2015 16:20:52 -0400
Subject: [AstroPy] Constructing an astropy table with colnames and
 dtypes of existing table
In-Reply-To: <CAMtEP6yU6RQMK4R-6rc7zXxzdvKJmxG6d2F4phSvrE5icWwbXQ@mail.gmail.com>
References: <77C17E06-E1DD-4442-9BDD-1A8588EE52C8@ucsc.edu>	<CAMtEP6wRi7NnaNx6h-o_SG+Qxdij-vZ+dyBWA_7poeqthJW3+Q@mail.gmail.com>	<CAMtEP6y1cLeYqd6fkCVFJJnK4GbbZwLu=1aWC69HNTbHT6nS-A@mail.gmail.com>	<CAGMHX_0kxZ5-UdKv24VRBacJaCj0a_Lwzh+HcLsXwxcBWKGS5Q@mail.gmail.com>	<1199E149-0A48-44E7-8CF8-1ED3FCA720B1@yale.edu>	<17F13F86-8B9F-4292-B9F6-3F49CCC5F8AB@ucsc.edu>
	<CAMtEP6yU6RQMK4R-6rc7zXxzdvKJmxG6d2F4phSvrE5icWwbXQ@mail.gmail.com>
Message-ID: <55413D24.5080707@stsci.edu>

On 04/29/2015 03:11 PM, Aldcroft, Thomas wrote:
> If anyone is interested in continuing the discussion on github, see:
>
> https://github.com/astropy/astropy/issues/3742

For what it's worth though, I feel like Alexa's original attempt should have 
just worked though.  If the original dtype is a multi-field dtype (i.e. has a 
non-empty .names attribute, which is true for the dtypes on tables), then it 
should be sufficient to create a new table with:

new_table = Table(dtype=orig_table.dtype)

it wouldn't even be necessary to explicitly provide a names= argument, since 
those can come from the original table's dtype (although if names= is provided 
that should override the names in the dtype as long as it's the correct number 
of names).

Having empty_like and such sounds fine too, but I don't see why the above 
example shouldn't work...

Erik

> On Wed, Apr 29, 2015 at 2:57 PM, Alexa Villaume <avillaum at ucsc.edu
> <mailto:avillaum at ucsc.edu>> wrote:
>
>     Thank you, everyone, for your helpful responses.
>
>     If it?s of any interest, I ended up going with Tom A.?s solution because one
>     of my columns holds arrays instead of single values. The dtype entry for
>     that looks like ?(?SPEC?, ?>f4?, (7925,)? which was fully copied in Tom A.?s
>     solution but not Tom R.?s.
>
>     Best,
>     Alexa
>
>     On Apr 29, 2015, at 11:45 AM, Andrew Hearin <andrew.hearin at yale.edu
>     <mailto:andrew.hearin at yale.edu>> wrote:
>
>      > You beat me to the punch, Tom, I was going to suggest exactly this.
>     Numpys built-in empty_like method is extremely convenient, and it would be
>     great to have an analogous astropy table method.
>      >
>      >
>      > On Apr 29, 2015, at 2:21 PM, Thomas Robitaille
>     <thomas.robitaille at gmail.com <mailto:thomas.robitaille at gmail.com>> wrote:
>      >
>      >> On 29 April 2015 at 19:19, Aldcroft, Thomas
>      >> <aldcroft at head.cfa.harvard.edu <mailto:aldcroft at head.cfa.harvard.edu>>
>     wrote:
>      >>>
>      >>>
>      >>> On Wed, Apr 29, 2015 at 2:13 PM, Aldcroft, Thomas
>      >>> <aldcroft at head.cfa.harvard.edu <mailto:aldcroft at head.cfa.harvard.edu>>
>     wrote:
>      >>>>
>      >>>>
>      >>>>
>      >>>> On Wed, Apr 29, 2015 at 1:40 PM, Alexa Villaume <avillaum at ucsc.edu
>     <mailto:avillaum at ucsc.edu>> wrote:
>      >>>>>
>      >>>>> Hi everybody,
>      >>>>>
>      >>>>> I?m trying to construct an astropy table using the exact same columns and
>      >>>>> corresponding dtypes of an existing table. The documentation goes
>     into using
>      >>>>> existing columns but it wasn?t as clear to me on how to handle the data
>      >>>>> types. For example, I have an existing table ?irtf? and I want to make a
>      >>>>> table ?irtf_hb? if I do,
>      >>>>>
>      >>>>> irtf_hb = Table(names=irtf.colnames)
>      >>>>>
>      >>>>> the table is initialized with all the right column names but the dtypes
>      >>>>> are all floats. However, I can?t do this,
>      >>>>>
>      >>>>> irtf_hb = Table(names=irtf.colnames, dtype=irtf.dtype)
>      >>>>>
>      >>>>> Because I get an error, ?ValueError: dtype must be a list or None?. I
>      >>>>> haven?t had any luck with trying to change the type of irtf.dtype or
>     using
>      >>>>> list comprehension to extract a list of dtypes.
>      >>>>>
>      >>>>> Is there a simple way to use the dtypes of an existing table to
>      >>>>> initialize a new table?
>      >>>>
>      >>>>
>      >>>> You should be able to do:
>      >>>>
>      >>>>>>> irtf_hb = Table(irtf[0:0])
>      >>>>
>      >>>> This is a slight hack, but it's pretty simple and should give you a
>      >>>> zero-length table with all the original table properties.  I tested on a
>      >>>> simple case but haven't checked in detail.
>      >>>
>      >>>
>      >>> BTW, "all the original table properties" includes all column and table meta
>      >>> data (e.g. units), so this should be the preferred idiom.
>      >>
>      >> How about a new class method that does something like
>      >> ``np.empty_like`` called ``Table.empty_like``?
>      >>
>      >> Cheers,
>      >> Tom
>      >>
>      >>>
>      >>> - Tom
>      >>>
>      >>>>
>      >>>>
>      >>>> In [3]: from astropy.table import Table
>      >>>>
>      >>>> In [4]: t = Table([[1]])
>      >>>>
>      >>>> In [5]: t2 = Table(t[0:0])
>      >>>>
>      >>>> In [6]: t2
>      >>>> Out[6]:
>      >>>> <Table masked=False length=0>
>      >>>> col0
>      >>>> int64
>      >>>> -----
>      >>>>
>      >>>> In [7]:
>      >>>>
>      >>>> - Tom
>      >>>>
>      >>>>>
>      >>>>>
>      >>>>> Thank you,
>      >>>>> Alexa
>      >>>>> _______________________________________________
>      >>>>> AstroPy mailing list
>      >>>>> AstroPy at scipy.org <mailto:AstroPy at scipy.org>
>      >>>>>
>     https://urldefense.proofpoint.com/v2/url?u=http-3A__mail.scipy.org_mailman_listinfo_astropy&d=AwIGaQ&c=-dg2m7zWuuDZ0MUcV7Sdqw&r=AHkQ8HPUDwzl0x62ybAnwN_OEebPRGDtcjUPBcnLYw4&m=tsHzQ6NpihEoX33pmnmiHF6nSNu86BozkIV4v_KCP6s&s=FRlFhyfQHtHEHMGpEAq6bnpkDKvPbatH7Q5WJ492XNU&e=
>      >>>>
>      >>>>
>      >>>
>      >>>
>      >>> _______________________________________________
>      >>> AstroPy mailing list
>      >>> AstroPy at scipy.org <mailto:AstroPy at scipy.org>
>      >>>
>     https://urldefense.proofpoint.com/v2/url?u=http-3A__mail.scipy.org_mailman_listinfo_astropy&d=AwIGaQ&c=-dg2m7zWuuDZ0MUcV7Sdqw&r=AHkQ8HPUDwzl0x62ybAnwN_OEebPRGDtcjUPBcnLYw4&m=tsHzQ6NpihEoX33pmnmiHF6nSNu86BozkIV4v_KCP6s&s=FRlFhyfQHtHEHMGpEAq6bnpkDKvPbatH7Q5WJ492XNU&e=
>      >>>
>      >> _______________________________________________
>      >> AstroPy mailing list
>      >> AstroPy at scipy.org <mailto:AstroPy at scipy.org>
>      >>
>     https://urldefense.proofpoint.com/v2/url?u=http-3A__mail.scipy.org_mailman_listinfo_astropy&d=AwIGaQ&c=-dg2m7zWuuDZ0MUcV7Sdqw&r=AHkQ8HPUDwzl0x62ybAnwN_OEebPRGDtcjUPBcnLYw4&m=tsHzQ6NpihEoX33pmnmiHF6nSNu86BozkIV4v_KCP6s&s=FRlFhyfQHtHEHMGpEAq6bnpkDKvPbatH7Q5WJ492XNU&e=
>      >
>      > _______________________________________________
>      > AstroPy mailing list
>      > AstroPy at scipy.org <mailto:AstroPy at scipy.org>
>      > http://mail.scipy.org/mailman/listinfo/astropy
>
>     _______________________________________________
>     AstroPy mailing list
>     AstroPy at scipy.org <mailto:AstroPy at scipy.org>
>     http://mail.scipy.org/mailman/listinfo/astropy
>
>
>
>
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org
> http://mail.scipy.org/mailman/listinfo/astropy
>