From astropython at gmail.com Mon May 2 08:14:41 2011 From: astropython at gmail.com (Astronomical Python) Date: Mon, 2 May 2011 08:14:41 -0400 Subject: [AstroPy] ATpy 0.9.5 Release Message-ID: We are pleased to announce the release of ATpy 0.9.5! ATpy is a Python package providing a high-level Table class that can contain data and meta-data, and includes column manipulation, row selection, and sorting methods. In addition, read and write methods are provided to to seamlessly read and write table data to a number of formats, building on existing Python modules. More information and links to download the latest version of ATpy can be found at http://atpy.github.com/ Note that we have now migrated the website and source code repository to GitHub. The website is now located at http://atpy.github.com/, while the source code can be found at https://github.com/atpy/atpy. The main changes in this version are: - ATpy is now released under an MIT license - support for writing out arbitrary tables via the asciitable module - support for copying all column metadata in one go when creating a new column based on an old column, for example: >>> t.add_column('b', column_header=t.columns[a]) - preliminary support for querying online services such as IRSA and the virtual observatory In addition, this release includes improvements to HDF5 input/output and minor bug fixes. A detailed list of changes can be found in the CHANGES file. Please do not hesitate to let us know if you encounter any problems with this release, Cheers, Thomas Robitaille and Eli Bressert From aldcroft at head.cfa.harvard.edu Tue May 3 09:34:35 2011 From: aldcroft at head.cfa.harvard.edu (Tom Aldcroft) Date: Tue, 3 May 2011 09:34:35 -0400 Subject: [AstroPy] Putting astronomy packages on PyPI Message-ID: I see there has been some discussion previously on the idea of putting packages on PyPI (in particular for vo-0.6). There did not seem to have been a resolution so I want to ask once more why it is not possible or desirable to do so. As background, over the last two months my colleagues and I at CfA have been running a series of workshops on Practical Python for Astronomers. The first and most difficult workshop was building a usable Python installation on individual laptops supporting Mac (mostly), linux, and Windows. Once we got the base installation done with Python, NumPy, SciPy and matplotlib then there were a bunch of other packages that we use during the workshops. The instructions for installing these were: easy_install asciitable easy_install http://www.stsci.edu/resources/software_hardware/pyfits/pyfits-2.4.0.tar.gz easy_install pywcs easy_install atpy easy_install aplpy easy_install http://stsdas.stsci.edu/astrolib/vo-0.6.tar.gz easy_install http://stsdas.stsci.edu/astrolib/coords-0.37.tar.gz easy_install pyparsing easy_install pyregion For the most part this worked quite well across all the platforms. So why can't pyfits, vo, and coords be there on PyPI to supply the download URL to easy_install in a uniform and non-version-dependent way? (The fact that I'm singling out those three should be taken as a compliment that they are useful and worth the bother). Of course you cannot guarantee that this will *always* build properly, but it's certainly worth an initial try the "easy" way. If package authors could make their source distribution available via PyPI would be a nice step toward making Python easier to adopt for astronomers. Again I understand there can be complications and ensuring perfection is impossible, but sometimes better is the enemy of good. - Tom From erik.tollerud at gmail.com Tue May 3 19:41:35 2011 From: erik.tollerud at gmail.com (Erik Tollerud) Date: Tue, 3 May 2011 16:41:35 -0700 Subject: [AstroPy] Putting astronomy packages on PyPI In-Reply-To: References: Message-ID: I definitely agree that putting packages on PyPI is a desirable thing, and it really is very easy to do. Having all these packages on PyPI would also make it possible to write automated installer tools to install everything in one go (e.g. astpys-setup in astropysics works because the PyPI data is easy to access automatically). One tip that I think may be of particular use to packages that require compilation: we should stop using "easy_install" and switch to "pip install". Setuptools and easy_install are not being developed anymore, and are python 3.0 incompatible. The replacement for setuptools is distribute (http://pypi.python.org/pypi/distribute), which emulates all of the setuptools functionality and is actively being developed (although in the future it is going to be replaced by a standard library tool). The replacement for easy_install is pip (http://pypi.python.org/pypi/pip/1.0.1). pip is different from easy_install in that it installs from source where possible, which makes it much more robust in most settings - my experience is that it usually just works, unlike easy_install (at least for more complicated packages). Pip is also much cleaner about where it puts packages, and doesn't cause all the strange path problems you sometimes see when you install multiple versions of something in easy_install. Pip also includes an "uninstall" option, which is very useful for certain situations. By the way, pyfits is on PyPI (http://pypi.python.org/pypi/pyfits/2.4.0) - "pip install pyfits" works for me (both in OS X and Linux). On Tue, May 3, 2011 at 6:34 AM, Tom Aldcroft wrote: > I see there has been some discussion previously on the idea of putting > packages on PyPI (in particular for vo-0.6). ?There did not seem to > have been a resolution so I want to ask once more why it is not > possible or desirable to do so. > > As background, over the last two months my colleagues and I at CfA > have been running a series of workshops on Practical Python for > Astronomers. ?The first and most difficult workshop was building a > usable Python installation on individual laptops supporting Mac > (mostly), linux, and Windows. ?Once we got the base installation done > with Python, NumPy, SciPy and matplotlib then there were a bunch of > other packages that we use during the workshops. ?The instructions for > installing these were: > > easy_install asciitable > easy_install http://www.stsci.edu/resources/software_hardware/pyfits/pyfits-2.4.0.tar.gz > easy_install pywcs > easy_install atpy > easy_install aplpy > easy_install http://stsdas.stsci.edu/astrolib/vo-0.6.tar.gz > easy_install http://stsdas.stsci.edu/astrolib/coords-0.37.tar.gz > easy_install pyparsing > easy_install pyregion > > For the most part this worked quite well across all the platforms. ?So > why can't pyfits, vo, and coords be there on PyPI to supply the > download URL to easy_install in a uniform and non-version-dependent > way? ?(The fact that I'm singling out those three should be taken as a > compliment that they are useful and worth the bother). ?Of course you > cannot guarantee that this will *always* build properly, but it's > certainly worth an initial try the "easy" way. > > If package authors could make their source distribution available via > PyPI would be a nice step toward making Python easier to adopt for > astronomers. ?Again I understand there can be complications and > ensuring perfection is impossible, but sometimes better is the enemy > of good. > > - Tom > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy > -- Erik Tollerud From luigi at lambrate.inaf.it Wed May 4 05:03:15 2011 From: luigi at lambrate.inaf.it (Luigi Paioro) Date: Wed, 04 May 2011 11:03:15 +0200 Subject: [AstroPy] Putting astronomy packages on PyPI In-Reply-To: References: Message-ID: <4DC11653.8030909@lambrate.inaf.it> I agree with you. A common packaging system for the astronomical software would be desirable in general, but, at least for the Python based software, we already have PyPI + pip, and I think there should be a shared effort to promote such system. This is one of the reasons that has led me to put SAMPy available on PyPI. Take also into account that, in principle, distutils/setuptools packages can be extended in order provide additional features specific for the astronomical community, if needed. For instance, within the context of the OPTICON funded FASE project, I have already coded an extension which allows to compile C code as libraries and/or executables (along with extension modules, of course)... so far it is not as flexible as `make` but, at least on Linux, it works... I can imaging a future implementation supporting even other languages (Java, FORTRAN, IDL, ...). This futuristic vision, if realized, would be a great revolution in the astronomical software, but everything starts from little efforts like just putting packages on PyPI, as Erik and Tom are sustaining. Best regards, Luigi Il 05/04/11 01:41, Erik Tollerud ha scritto: > I definitely agree that putting packages on PyPI is a desirable thing, > and it really is very easy to do. Having all these packages on PyPI > would also make it possible to write automated installer tools to > install everything in one go (e.g. astpys-setup in astropysics works > because the PyPI data is easy to access automatically). > > One tip that I think may be of particular use to packages that require > compilation: we should stop using "easy_install" and switch to "pip > install". > > Setuptools and easy_install are not being developed anymore, and are > python 3.0 incompatible. The replacement for setuptools is distribute > (http://pypi.python.org/pypi/distribute), which emulates all of the > setuptools functionality and is actively being developed (although in > the future it is going to be replaced by a standard library tool). > The replacement for easy_install is pip > (http://pypi.python.org/pypi/pip/1.0.1). pip is different from > easy_install in that it installs from source where possible, which > makes it much more robust in most settings - my experience is that it > usually just works, unlike easy_install (at least for more complicated > packages). Pip is also much cleaner about where it puts packages, and > doesn't cause all the strange path problems you sometimes see when you > install multiple versions of something in easy_install. Pip also > includes an "uninstall" option, which is very useful for certain > situations. > > By the way, pyfits is on PyPI > (http://pypi.python.org/pypi/pyfits/2.4.0) - "pip install pyfits" > works for me (both in OS X and Linux). > > > > On Tue, May 3, 2011 at 6:34 AM, Tom Aldcroft > wrote: >> I see there has been some discussion previously on the idea of putting >> packages on PyPI (in particular for vo-0.6). There did not seem to >> have been a resolution so I want to ask once more why it is not >> possible or desirable to do so. >> >> As background, over the last two months my colleagues and I at CfA >> have been running a series of workshops on Practical Python for >> Astronomers. The first and most difficult workshop was building a >> usable Python installation on individual laptops supporting Mac >> (mostly), linux, and Windows. Once we got the base installation done >> with Python, NumPy, SciPy and matplotlib then there were a bunch of >> other packages that we use during the workshops. The instructions for >> installing these were: >> >> easy_install asciitable >> easy_install http://www.stsci.edu/resources/software_hardware/pyfits/pyfits-2.4.0.tar.gz >> easy_install pywcs >> easy_install atpy >> easy_install aplpy >> easy_install http://stsdas.stsci.edu/astrolib/vo-0.6.tar.gz >> easy_install http://stsdas.stsci.edu/astrolib/coords-0.37.tar.gz >> easy_install pyparsing >> easy_install pyregion >> >> For the most part this worked quite well across all the platforms. So >> why can't pyfits, vo, and coords be there on PyPI to supply the >> download URL to easy_install in a uniform and non-version-dependent >> way? (The fact that I'm singling out those three should be taken as a >> compliment that they are useful and worth the bother). Of course you >> cannot guarantee that this will *always* build properly, but it's >> certainly worth an initial try the "easy" way. >> >> If package authors could make their source distribution available via >> PyPI would be a nice step toward making Python easier to adopt for >> astronomers. Again I understand there can be complications and >> ensuring perfection is impossible, but sometimes better is the enemy >> of good. From thomas.robitaille at gmail.com Wed May 4 08:15:57 2011 From: thomas.robitaille at gmail.com (Thomas Robitaille) Date: Wed, 4 May 2011 08:15:57 -0400 Subject: [AstroPy] Putting astronomy packages on PyPI In-Reply-To: <4DC11653.8030909@lambrate.inaf.it> References: <4DC11653.8030909@lambrate.inaf.it> Message-ID: I also agree with putting packages on PyPI, and the suggestion of using pip over easy_install. I just wanted to add one comment about PyPI for anyone who is still not convinced - you don't actually have to put a tar file on PyPI if you don't want, and you definitely don't have to provide binary distributions. Take a look at ATpy for example: http://pypi.python.org/pypi/ATpy - all this is is some meta-data pointing to the source tar file hosted on GitHub. This is the bare minimum, but still allows people to use 'pip install atpy'. It's also possible to set up your environment so that all you do is type 'python setup.py register' and the latest version of your package will be added to PyPI: http://packages.python.org/an_example_pypi_project/setuptools.html#intermezzo-pypirc-file-and-gpg Finally one thing that we should look at closely is pip bundles: http://www.pip-installer.org/en/latest/index.html#bundles Apparently this is not stable at this stage, but the idea would be to be able to distribute many packages together, which should clearly be of interest to us. Cheers, Tom On Wednesday, May 4, 2011 at 5:03 AM, Luigi Paioro wrote: > I agree with you. A common packaging system for the astronomical > software would be desirable in general, but, at least for the Python > based software, we already have PyPI + pip, and I think there should be > a shared effort to promote such system. This is one of the reasons that > has led me to put SAMPy available on PyPI. > > Take also into account that, in principle, distutils/setuptools packages > can be extended in order provide additional features specific for the > astronomical community, if needed. For instance, within the context of > the OPTICON funded FASE project, I have already coded an extension which > allows to compile C code as libraries and/or executables (along with > extension modules, of course)... so far it is not as flexible as `make` > but, at least on Linux, it works... I can imaging a future > implementation supporting even other languages (Java, FORTRAN, IDL, > ...). This futuristic vision, if realized, would be a great revolution > in the astronomical software, but everything starts from little efforts > like just putting packages on PyPI, as Erik and Tom are sustaining. > > Best regards, > > Luigi > > > Il 05/04/11 01:41, Erik Tollerud ha scritto: > > I definitely agree that putting packages on PyPI is a desirable thing, > > and it really is very easy to do. Having all these packages on PyPI > > would also make it possible to write automated installer tools to > > install everything in one go (e.g. astpys-setup in astropysics works > > because the PyPI data is easy to access automatically). > > > > One tip that I think may be of particular use to packages that require > > compilation: we should stop using "easy_install" and switch to "pip > > install". > > > > Setuptools and easy_install are not being developed anymore, and are > > python 3.0 incompatible. The replacement for setuptools is distribute > > (http://pypi.python.org/pypi/distribute), which emulates all of the > > setuptools functionality and is actively being developed (although in > > the future it is going to be replaced by a standard library tool). > > The replacement for easy_install is pip > > (http://pypi.python.org/pypi/pip/1.0.1). pip is different from > > easy_install in that it installs from source where possible, which > > makes it much more robust in most settings - my experience is that it > > usually just works, unlike easy_install (at least for more complicated > > packages). Pip is also much cleaner about where it puts packages, and > > doesn't cause all the strange path problems you sometimes see when you > > install multiple versions of something in easy_install. Pip also > > includes an "uninstall" option, which is very useful for certain > > situations. > > > > By the way, pyfits is on PyPI > > (http://pypi.python.org/pypi/pyfits/2.4.0) - "pip install pyfits" > > works for me (both in OS X and Linux). > > > > > > > > On Tue, May 3, 2011 at 6:34 AM, Tom Aldcroft > > wrote: > > > I see there has been some discussion previously on the idea of putting > > > packages on PyPI (in particular for vo-0.6). There did not seem to > > > have been a resolution so I want to ask once more why it is not > > > possible or desirable to do so. > > > > > > As background, over the last two months my colleagues and I at CfA > > > have been running a series of workshops on Practical Python for > > > Astronomers. The first and most difficult workshop was building a > > > usable Python installation on individual laptops supporting Mac > > > (mostly), linux, and Windows. Once we got the base installation done > > > with Python, NumPy, SciPy and matplotlib then there were a bunch of > > > other packages that we use during the workshops. The instructions for > > > installing these were: > > > > > > easy_install asciitable > > > easy_install http://www.stsci.edu/resources/software_hardware/pyfits/pyfits-2.4.0.tar.gz > > > easy_install pywcs > > > easy_install atpy > > > easy_install aplpy > > > easy_install http://stsdas.stsci.edu/astrolib/vo-0.6.tar.gz > > > easy_install http://stsdas.stsci.edu/astrolib/coords-0.37.tar.gz > > > easy_install pyparsing > > > easy_install pyregion > > > > > > For the most part this worked quite well across all the platforms. So > > > why can't pyfits, vo, and coords be there on PyPI to supply the > > > download URL to easy_install in a uniform and non-version-dependent > > > way? (The fact that I'm singling out those three should be taken as a > > > compliment that they are useful and worth the bother). Of course you > > > cannot guarantee that this will *always* build properly, but it's > > > certainly worth an initial try the "easy" way. > > > > > > If package authors could make their source distribution available via > > > PyPI would be a nice step toward making Python easier to adopt for > > > astronomers. Again I understand there can be complications and > > > ensuring perfection is impossible, but sometimes better is the enemy > > > of good. > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy > From embray at stsci.edu Wed May 4 12:17:46 2011 From: embray at stsci.edu (Erik Bray) Date: Wed, 4 May 2011 12:17:46 -0400 Subject: [AstroPy] Fwd: [Fwd: Putting astronomy packages on PyPI] In-Reply-To: References: <16DCC901-2C8A-4DCC-AE5F-620AB8D43299@stsci.edu> Message-ID: <4DC17C2A.6030609@stsci.edu> Sorry, I didn't look at AstroPy at all yesterday. Everything suggested by the original poster will be possible with the work I'm doing (and in fact already is possibly for all of stsci_python except betadrizzle, though I haven't done most of Astrolib yet). Once we get some source dists for these projects on PyPI it'll work either with easy_install or (preferably) `pip install`. On 05/04/2011 12:14 PM, Perry Greenfield wrote: > Hi Erik, > > What did you want to do about this? > > Begin forwarded message: > >> From: Perry Greenfield >> Date: May 3, 2011 10:36:48 AM EDT >> To: "Victoria G. Laidler" >> Cc: Mark Sienkiewicz , >> Subject: Re: [Fwd: [AstroPy] Putting astronomy packages on PyPI] >> >> I've asked Erik if he wants to address this since he's been working on >> just this sort of thing. >> >> On May 3, 2011, at 10:10 AM, Victoria G. Laidler wrote: >> >>> Not sure if you all follow Astropy, but here's a specific request for >>> distribution options for three of our astrolib packages. >>> >>> If we were going to do this with anything in Astrolib, I'd probably >>> like to include pysynphot as well. >>> >>> -------- Original Message -------- >>> Subject: [AstroPy] Putting astronomy packages on PyPI >>> Date: Tue, 3 May 2011 09:34:35 -0400 >>> From: Tom Aldcroft >>> To: astropy >>> >>> >>> >>> I see there has been some discussion previously on the idea of putting >>> packages on PyPI (in particular for vo-0.6). There did not seem to >>> have been a resolution so I want to ask once more why it is not >>> possible or desirable to do so. >>> >>> As background, over the last two months my colleagues and I at CfA >>> have been running a series of workshops on Practical Python for >>> Astronomers. The first and most difficult workshop was building a >>> usable Python installation on individual laptops supporting Mac >>> (mostly), linux, and Windows. Once we got the base installation done >>> with Python, NumPy, SciPy and matplotlib then there were a bunch of >>> other packages that we use during the workshops. The instructions for >>> installing these were: >>> >>> easy_install asciitable >>> easy_install >>> http://www.stsci.edu/resources/software_hardware/pyfits/pyfits-2.4.0.tar.gz >>> >>> easy_install pywcs >>> easy_install atpy >>> easy_install aplpy >>> easy_install http://stsdas.stsci.edu/astrolib/vo-0.6.tar.gz >>> easy_install http://stsdas.stsci.edu/astrolib/coords-0.37.tar.gz >>> easy_install pyparsing >>> easy_install pyregion >>> >>> For the most part this worked quite well across all the platforms. So >>> why can't pyfits, vo, and coords be there on PyPI to supply the >>> download URL to easy_install in a uniform and non-version-dependent >>> way? (The fact that I'm singling out those three should be taken as a >>> compliment that they are useful and worth the bother). Of course you >>> cannot guarantee that this will *always* build properly, but it's >>> certainly worth an initial try the "easy" way. >>> >>> If package authors could make their source distribution available via >>> PyPI would be a nice step toward making Python easier to adopt for >>> astronomers. Again I understand there can be complications and >>> ensuring perfection is impossible, but sometimes better is the enemy >>> of good. >>> >>> - Tom >>> _______________________________________________ >>> AstroPy mailing list >>> AstroPy at scipy.org >>> http://mail.scipy.org/mailman/listinfo/astropy >>> >> > From embray at stsci.edu Wed May 4 12:21:09 2011 From: embray at stsci.edu (Erik Bray) Date: Wed, 4 May 2011 12:21:09 -0400 Subject: [AstroPy] Putting astronomy packages on PyPI In-Reply-To: References: <4DC11653.8030909@lambrate.inaf.it> Message-ID: <4DC17CF5.6070601@stsci.edu> On 05/04/2011 08:15 AM, Thomas Robitaille wrote: > Finally one thing that we should look at closely is pip bundles: > > http://www.pip-installer.org/en/latest/index.html#bundles > > Apparently this is not stable at this stage, but the idea would be to be able to distribute many packages together, which should clearly be of interest to us. > > Cheers, > Tom I've looked into using pip bundles for stsci_python and Astrolib. But as you said, it's warning about instability made me lose interest. Also, it won't work with easy_install which some people still use no matter how many times we tell them to use pip :) Instead, each project that make up Astrolib and stsci_python are individually installable, and will bring with them all their requirements. In addition, there Astrolib and stsci_python "meta-packges" that just have as requirements all the projects that comprise them. Erik From ubutsu at gmail.com Fri May 6 13:11:50 2011 From: ubutsu at gmail.com (Taro Sato) Date: Fri, 6 May 2011 14:11:50 -0300 Subject: [AstroPy] PyFITS: Appending tables with variable length array...? Message-ID: Hi there. I'm trying to do something simple with PyFITS, appending tables but running into a problem. The manual provides a general procedure for tables with no varlen arrays, which I mostly follow in the code snippet below. import pyfits as pf ... t1 = pf.open(fits1) t2 = pf.open(fits2) nr1 = t1[1].data.shape[0] nr2 = t2[1].data.shape[0] nr = nr1 + nr2 hdu = pf.new_table(t1[1].columns, nrows=nr) for i in range(len(t1[1].columns)): if i == 4: # this is a variable length array column; need to # work on row by row for j, x in enumerate(t2[1].data.field(i)): print((nr, nr1+j)) hdu.data.field(i)[nr1+j] = x continue # for non variable length column, this is fine hdu.data.field(i)[nr1:] = t2[1].data.field(i) The conditional was added to take care of a variable length array column (#5). But this doesn't work... So my question is, how do we do this? Is it at all possible to update values in a varlen array column? Is my only option to create a new table from scratch? Thank you for your time, Taro From embray at stsci.edu Fri May 6 16:56:03 2011 From: embray at stsci.edu (Erik Bray) Date: Fri, 6 May 2011 16:56:03 -0400 Subject: [AstroPy] PyFITS: Appending tables with variable length array...? In-Reply-To: References: Message-ID: <4DC46063.10204@stsci.edu> Yeah, that looks like a bug to me. When you create the new hdu with nrows=nr, the length of the VLA column does not get extended. Normally you shouldn't have to do any special treatment for VLA columns at all, actually. Here's a workaround that worked for me: for idx in range(len(t1[1].columns)): col = t1[1].columns[idx] if col.format[0] == 'P': hdu.columns[idx].array = np.resize(col.array, nr) hdu.data._convert[idx] = hdu.columns[idx].array hdu.data.field(idx)[nr1:] = t2[1].data.field(idx) It's a hack, but it should work--this replaces the existing VLA column with one that's extended out the right size. I'll add a bug report for this issue. Thanks, Erik On 05/06/2011 01:11 PM, Taro Sato wrote: > Hi there. I'm trying to do something simple with PyFITS, appending > tables but running into a problem. > > The manual provides a general procedure for tables with no varlen > arrays, which I mostly follow in the code snippet below. > > import pyfits as pf > > ... > > t1 = pf.open(fits1) > t2 = pf.open(fits2) > > nr1 = t1[1].data.shape[0] > nr2 = t2[1].data.shape[0] > nr = nr1 + nr2 > hdu = pf.new_table(t1[1].columns, nrows=nr) > for i in range(len(t1[1].columns)): > if i == 4: > # this is a variable length array column; need to > # work on row by row > for j, x in enumerate(t2[1].data.field(i)): > print((nr, nr1+j)) > hdu.data.field(i)[nr1+j] = x > continue > # for non variable length column, this is fine > hdu.data.field(i)[nr1:] = t2[1].data.field(i) > > > The conditional was added to take care of a variable length array > column (#5). But this doesn't work... > > So my question is, how do we do this? Is it at all possible to update > values in a varlen array column? Is my only option to create a new > table from scratch? > > Thank you for your time, > Taro > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy From taro at ap.smu.ca Fri May 6 19:05:19 2011 From: taro at ap.smu.ca (Taro Sato) Date: Fri, 6 May 2011 20:05:19 -0300 Subject: [AstroPy] PyFITS: Appending tables with variable length array...? In-Reply-To: <4DC46063.10204@stsci.edu> References: <4DC46063.10204@stsci.edu> Message-ID: On Fri, May 6, 2011 at 5:56 PM, Erik Bray wrote: > Here's a workaround that worked for me: > > for idx in range(len(t1[1].columns)): > ? ? col = t1[1].columns[idx] > ? ? if col.format[0] == 'P': > ? ? ? ? hdu.columns[idx].array = np.resize(col.array, nr) > ? ? ? ? hdu.data._convert[idx] = hdu.columns[idx].array > ? ? hdu.data.field(idx)[nr1:] = t2[1].data.field(idx) > > > It's a hack, but it should work--this replaces the existing VLA column > with one that's extended out the right size. ?I'll add a bug report for > this issue. > > Thanks, > Erik Thanks very much for your response, Erik. Unfortunately it doesn't work for me quite yet... I see what you are doing, but for some reason the column doesn't remain a _VLF object after setting _convert. (Before setting _convert I confirm it's still a _VLF.) For me, right after _convert setting it becomes a flat np.array of int32 and it complains about me trying to set an array element with a sequence. Any idea how to work around this? From johann.cohentanugi at gmail.com Mon May 9 10:00:52 2011 From: johann.cohentanugi at gmail.com (Johann Cohen-Tanugi) Date: Mon, 09 May 2011 16:00:52 +0200 Subject: [AstroPy] about ASMOOTH code as described in http://www.arxiv.com/abs/astro-ph/0601306 Message-ID: <4DC7F394.2010403@gmail.com> hi there, Does anyone know about a python implementation of the algorithm above? best, johann From taro at ap.smu.ca Mon May 9 16:40:41 2011 From: taro at ap.smu.ca (Taro Sato) Date: Mon, 9 May 2011 17:40:41 -0300 Subject: [AstroPy] PyFITS: Appending tables with variable length array...? In-Reply-To: <4DC7FC77.40603@stsci.edu> References: <4DC46063.10204@stsci.edu> <4DC7FC77.40603@stsci.edu> Message-ID: On Mon, May 9, 2011 at 11:38 AM, Erik Bray wrote: > Hi Taro, > > On 05/06/2011 07:05 PM, Taro Sato wrote: >> >> On Fri, May 6, 2011 at 5:56 PM, Erik Bray ?wrote: >>> >>> Here's a workaround that worked for me: >>> >>> for idx in range(len(t1[1].columns)): >>> ? ? col = t1[1].columns[idx] >>> ? ? if col.format[0] == 'P': >>> ? ? ? ? hdu.columns[idx].array = np.resize(col.array, nr) >>> ? ? ? ? hdu.data._convert[idx] = hdu.columns[idx].array >>> ? ? hdu.data.field(idx)[nr1:] = t2[1].data.field(idx) >>> >> Thanks very much for your response, Erik. ?Unfortunately it doesn't >> work for me quite yet... >> >> I see what you are doing, but for some reason the column doesn't >> remain a _VLF object after setting _convert. ?(Before setting _convert >> I confirm it's still a _VLF.) ?For me, right after _convert setting it >> becomes a flat np.array of int32 and it complains about me trying to >> set an array element with a sequence. >> >> Any idea how to work around this? > > Sorry I couldn't get back to you sooner. ?I see what you're saying; I might > have made a mistake in writing down the sample code. ?There are maybe two > things to try here: > > Where it assigns to hdu.data._convert[idx], instead try: > > hdu.data._convert[idx] = None > > Or if that doesn't work you can manually wrap the array in a _VLF object: > > hdu.data._convert[idx] = pyfits.core._VLF(hdu.columns[idx].array) > > I created a ticket for this bug here: > http://trac6.assembla.com/pyfits/ticket/54 > > Thanks, and I hope that works. > > Erik > > Erik -- Thanks for your response (and absolutely no need for apology since I wasn't expecting a tech support anything like that...). I think I managed to get it to work for me, but I need to do a tiny bit more of dirty hacking of the private parts... I love/hate that Python doesn't hide anything... So indecent... I needed to assign _VLF to _convert[idx], but unfortunately that somehow messed up the structure of _VLF by flattening the original _VLF array (into an array of int, float, etc.). Within PyFITS it appears _VLF is always assumed to be an array of arrays (i.e., 2D) so I had to manually maintain it that way. Here's the code #### BEGIN #### import pyfits as pf ... t1 = pf.open(fits1) t2 = pf.open(fits2) nr1 = t1[1].data.shape[0] nr2 = t2[1].data.shape[0] nr = nr1 + nr2 hdu = pf.new_table(t1[1].columns, nrows=nr) for idx in range(len(t1[1].columns)): col = t1[1].columns[idx] if col.format[0] == 'P': orig = t1[1].data.field(idx)[:nr1] hdu.columns[idx].array = np.resize(col.array, nr) hdu.data._convert[idx] = pf.core._VLF(hdu.columns[idx].array) hdu.data._convert[idx][:nr1] = orig hdu.data.field(idx)[nr1:] = t2[1].data.field(idx) ... #### END #### For now it's working for me. This post is just for completeness. Hope it gets implemented more elegantly in PyFITS. Cheers, Taro From glenn.caltech at gmail.com Fri May 13 12:58:00 2011 From: glenn.caltech at gmail.com (G Jones) Date: Fri, 13 May 2011 09:58:00 -0700 Subject: [AstroPy] PyFITS memory usage Message-ID: Hello, I am seeing what looks like a memory leak when using PyFITS. It may not actually be a leak, but I am curious if anyone can explain the behavior. I am processing PSRFITS files which are ~40 GB and have a table with the data in it. Since the files are so large, I open them with memmap=True. I would expect the following simple script to have a constant memory footprint, but instead I see continuously growing memory usage. import pyfits hdul = pyfits.open('myfile.fits',mode="readonly", memmap=True) primary,secondary = hdul for k in range(100): x = secondary.data[k].field('DATA').astype('float32') I can see that the data is being read in from disk as it should be, but I don't expect the memory usage to increase, since x is constantly overwritten. The versions of programs I am using are: RHEL 5 x86_64 (2.6.18-194.32.1.el5) python 2.7.1 numpy 2.0.0.dev pyfits 2.4.0 Thanks, Glenn -------------- next part -------------- An HTML attachment was scrubbed... URL: From taro at ap.smu.ca Fri May 13 13:08:02 2011 From: taro at ap.smu.ca (Taro Sato) Date: Fri, 13 May 2011 14:08:02 -0300 Subject: [AstroPy] PyFITS memory usage In-Reply-To: References: Message-ID: On Fri, May 13, 2011 at 1:58 PM, G Jones wrote: > Hello, > I am seeing what looks like a memory leak when using PyFITS. It may not > actually be a leak, but I am curious if anyone can explain the behavior. I > am processing PSRFITS files which are ~40 GB and have a table with the data > in it. Since the files are so large, I open them with memmap=True. I would > expect the following simple script to have a constant memory footprint, but > instead I see continuously growing memory usage. > > import pyfits > > hdul = pyfits.open('myfile.fits',mode="readonly", memmap=True) > primary,secondary = hdul > for k in range(100): > ? x = secondary.data[k].field('DATA').astype('float32') > > > I can see that the data is being read in from disk as it should be, but I > don't expect the memory usage to increase, since x is constantly > overwritten. The versions of programs I am using are: What if you do: for k in range(100): ... del x to sort of force memory release within the loop? Not sure how the refcounting/garbage collection works in Python anymore, but I've been successful in the past with this strategy, when I had tight memory budget. -- Taro Sato Department of Astronomy & Physics Saint Mary's University? ? ? ? ? ?? email: taro at ap.smu.ca Halifax, NS B3H 3C3? ? ? ? ? ? ? ?? phone: (902)420-5027 Canada? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? web: http://ap.smu.ca/~taro From brandon.craig.rhodes at gmail.com Fri May 13 13:20:29 2011 From: brandon.craig.rhodes at gmail.com (Brandon Craig Rhodes) Date: Fri, 13 May 2011 13:20:29 -0400 Subject: [AstroPy] PyFITS memory usage In-Reply-To: (G. Jones's message of "Fri, 13 May 2011 09:58:00 -0700") References: Message-ID: <871v02qzz6.fsf@asaph.rhodesmill.org> G Jones writes: > I am processing PSRFITS files which are ~40 GB and have a table with > the data in it. Since the files are so large, I open them with > memmap=True. I would expect the following simple script to have a > constant memory footprint, but instead I see continuously growing > memory usage. ... I can see that the data is being read in from disk > as it should be, but I don't expect the memory usage to increase, > since x is constantly overwritten. What are you using to measure its memory use - what tool is showing you that its memory use is going up? It will help us answer if we know which memory-use metric (of the several offered by tools like "ps" and "top") are the ones you see increasing. -- Brandon Craig Rhodes brandon at rhodesmill.org http://rhodesmill.org/brandon From embray at stsci.edu Fri May 13 13:29:29 2011 From: embray at stsci.edu (Erik Bray) Date: Fri, 13 May 2011 13:29:29 -0400 Subject: [AstroPy] PyFITS memory usage In-Reply-To: References: Message-ID: <4DCD6A79.4050601@stsci.edu> On 05/13/2011 12:58 PM, G Jones wrote: > Hello, > I am seeing what looks like a memory leak when using PyFITS. It may not > actually be a leak, but I am curious if anyone can explain the behavior. > I am processing PSRFITS files which are ~40 GB and have a table with the > data in it. Since the files are so large, I open them with memmap=True. > I would expect the following simple script to have a constant memory > footprint, but instead I see continuously growing memory usage. > > import pyfits > > hdul = pyfits.open('myfile.fits',mode="readonly", memmap=True) > primary,secondary = hdul > for k in range(100): > x = secondary.data[k].field('DATA').astype('float32') > > > I can see that the data is being read in from disk as it should be, but > I don't expect the memory usage to increase, since x is constantly > overwritten. The versions of programs I am using are: > RHEL 5 x86_64 (2.6.18-194.32.1.el5) > python 2.7.1 > numpy 2.0.0.dev > pyfits 2.4.0 There is a known memory leak in PyFITS 2.4.0 (and possibly below). See http://trac6.assembla.com/pyfits/ticket/49. This has been fixed in the current development version, and the fix should be in the next release. If you can patch your PyFITS installation, I have a patch for the memory leak issue that applies cleanly to 2.4.0. Though enough people have had this issue, that maybe we could release a 2.4.1 with this fix in the meantime? That might not be a bad idea. Erik From jaz at astro.ox.ac.uk Fri May 13 13:30:16 2011 From: jaz at astro.ox.ac.uk (Joe Zuntz) Date: Fri, 13 May 2011 17:30:16 +0000 Subject: [AstroPy] PyFITS memory usage In-Reply-To: References: Message-ID: <777F5C75-AB34-4FD0-9533-A4B897EC10C6@astro.ox.ac.uk> I've solved a similar problem in the past by importing gc and running gc.collect() after each file. Joe Zuntz Sent from my iPhone On 13 May 2011, at 17:58, "G Jones" wrote: > Hello, > I am seeing what looks like a memory leak when using PyFITS. It may not actually be a leak, but I am curious if anyone can explain the behavior. I am processing PSRFITS files which are ~40 GB and have a table with the data in it. Since the files are so large, I open them with memmap=True. I would expect the following simple script to have a constant memory footprint, but instead I see continuously growing memory usage. > > import pyfits > > hdul = pyfits.open('myfile.fits',mode="readonly", memmap=True) > primary,secondary = hdul > for k in range(100): > x = secondary.data[k].field('DATA').astype('float32') > > > I can see that the data is being read in from disk as it should be, but I don't expect the memory usage to increase, since x is constantly overwritten. The versions of programs I am using are: > RHEL 5 x86_64 (2.6.18-194.32.1.el5) > python 2.7.1 > numpy 2.0.0.dev > pyfits 2.4.0 > > Thanks, > Glenn > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy From glenn.caltech at gmail.com Fri May 13 15:49:25 2011 From: glenn.caltech at gmail.com (G Jones) Date: Fri, 13 May 2011 12:49:25 -0700 Subject: [AstroPy] PyFITS memory usage In-Reply-To: <871v02qzz6.fsf@asaph.rhodesmill.org> References: <871v02qzz6.fsf@asaph.rhodesmill.org> Message-ID: Hi Brandon, I am tracking memory usage using top which shows the "RES" and "SHR" columns growing steadily. Interestingly, using ksysguard, I see the blue "Application memory" curve staying roughly constant as I would expect. However, when the memory usage reported by top gets near the amount of physical memory on the system, I do see the system start to swap-thrash. So I believe the usage reported by top is accurate. Glenn On Fri, May 13, 2011 at 10:20 AM, Brandon Craig Rhodes < brandon.craig.rhodes at gmail.com> wrote: > G Jones writes: > > > I am processing PSRFITS files which are ~40 GB and have a table with > > the data in it. Since the files are so large, I open them with > > memmap=True. I would expect the following simple script to have a > > constant memory footprint, but instead I see continuously growing > > memory usage. ... I can see that the data is being read in from disk > > as it should be, but I don't expect the memory usage to increase, > > since x is constantly overwritten. > > What are you using to measure its memory use - what tool is showing you > that its memory use is going up? It will help us answer if we know > which memory-use metric (of the several offered by tools like "ps" and > "top") are the ones you see increasing. > > -- > Brandon Craig Rhodes brandon at rhodesmill.org > http://rhodesmill.org/brandon > -------------- next part -------------- An HTML attachment was scrubbed... URL: From marquett at iap.fr Mon May 16 11:02:00 2011 From: marquett at iap.fr (Jean-Baptiste Marquette) Date: Mon, 16 May 2011 17:02:00 +0200 Subject: [AstroPy] import problem (matplotlib ? numpy ?) on Mac Intel, python 2.6 Message-ID: Dear matplotlib/numpy gurus, This is my first attempt to use matplotlib & numpy, with the following small piece of code: #!/usr/bin/env jython **** jython because of the import of stilts from starjava **** # Fabrique les figures des champs du papier Catalogue __author__="marquett" __date__ ="$16 mai 2011 12:17:08$" import sys sys.path.append('/star/starjava/etc/ttools') sys.path.append('/Library/Frameworks/Python.framework//Versions/2.6/lib/python2.6/site-packages') import stilts import glob import matplotlib.pyplot as plt I got the following trace: Traceback (most recent call last): File "/Users/marquett/Downloads/FigCat/src/figcat.py", line 14, in import matplotlib.pyplot as plt File "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/matplotlib/__init__.py", line 135, in from matplotlib.rcsetup import (defaultParams, File "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/matplotlib/rcsetup.py", line 19, in from matplotlib.colors import is_color_like File "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/matplotlib/colors.py", line 52, in import numpy as np File "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/numpy/__init__.py", line 137, in import add_newdocs File "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/numpy/add_newdocs.py", line 9, in from numpy.lib import add_newdoc File "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/numpy/lib/__init__.py", line 4, in from type_check import * File "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/numpy/lib/type_check.py", line 8, in import numpy.core.numeric as _nx File "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/numpy/core/__init__.py", line 5, in import multiarray ImportError: No module named multiarray I use the latest 1.6.0 version of numpy and 1.0.0 one of matplotlib. My so little experience tells me that this is a deep numpy issue rather than a matplotlib one. Any hint welcome, thanks. JB Marquette -------------- next part -------------- An HTML attachment was scrubbed... URL: From perry at stsci.edu Mon May 16 11:28:08 2011 From: perry at stsci.edu (Perry Greenfield) Date: Mon, 16 May 2011 11:28:08 -0400 Subject: [AstroPy] import problem (matplotlib ? numpy ?) on Mac Intel, python 2.6 In-Reply-To: References: Message-ID: <42B1D5B9-44E6-4FC8-859E-8486B9EF7F5D@stsci.edu> matplotlib or numpy aren't going to work with jython. On May 16, 2011, at 11:02 AM, Jean-Baptiste Marquette wrote: > Dear matplotlib/numpy gurus, > > This is my first attempt to use matplotlib & numpy, with the > following small piece of code: > > #!/usr/bin/env jython **** jython because of the import of stilts > from starjava **** > > # Fabrique les figures des champs du papier Catalogue > > > __author__="marquett" > __date__ ="$16 mai 2011 12:17:08$" > > import sys > sys.path.append('/star/starjava/etc/ttools') > sys.path.append('/Library/Frameworks/Python.framework//Versions/2.6/ > lib/python2.6/site-packages') > import stilts > import glob > import matplotlib.pyplot as plt > > I got the following trace: > > Traceback (most recent call last): > File "/Users/marquett/Downloads/FigCat/src/figcat.py", line 14, in > > import matplotlib.pyplot as plt > File "/Library/Frameworks/Python.framework/Versions/2.6/lib/ > python2.6/site-packages/matplotlib/__init__.py", line 135, in > from matplotlib.rcsetup import (defaultParams, > File "/Library/Frameworks/Python.framework/Versions/2.6/lib/ > python2.6/site-packages/matplotlib/rcsetup.py", line 19, in > from matplotlib.colors import is_color_like > File "/Library/Frameworks/Python.framework/Versions/2.6/lib/ > python2.6/site-packages/matplotlib/colors.py", line 52, in > import numpy as np > File "/Library/Frameworks/Python.framework/Versions/2.6/lib/ > python2.6/site-packages/numpy/__init__.py", line 137, in > import add_newdocs > File "/Library/Frameworks/Python.framework/Versions/2.6/lib/ > python2.6/site-packages/numpy/add_newdocs.py", line 9, in > from numpy.lib import add_newdoc > File "/Library/Frameworks/Python.framework/Versions/2.6/lib/ > python2.6/site-packages/numpy/lib/__init__.py", line 4, in > from type_check import * > File "/Library/Frameworks/Python.framework/Versions/2.6/lib/ > python2.6/site-packages/numpy/lib/type_check.py", line 8, in > import numpy.core.numeric as _nx > File "/Library/Frameworks/Python.framework/Versions/2.6/lib/ > python2.6/site-packages/numpy/core/__init__.py", line 5, in > import multiarray > ImportError: No module named multiarray > > I use the latest 1.6.0 version of numpy and 1.0.0 one of matplotlib. > > My so little experience tells me that this is a deep numpy issue > rather than a matplotlib one. > > Any hint welcome, thanks. > > JB Marquette > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy From thomas.boch at astro.unistra.fr Mon May 16 11:29:03 2011 From: thomas.boch at astro.unistra.fr (Thomas Boch) Date: Mon, 16 May 2011 17:29:03 +0200 Subject: [AstroPy] import problem (matplotlib ? numpy ?) on Mac Intel, python 2.6 In-Reply-To: References: Message-ID: <4DD142BF.2040208@astro.unistra.fr> Hi Jean-Baptiste, I don't think numpy is currently supported by Jython. I do not know exactly what is your use of starjava, but you might want to have a look at AtPy (http://atpy.github.com/) which does a pretty good job at manipulating (astronomical) tables. Cheers, Thomas Boch On 16/05/11 17:02, Jean-Baptiste Marquette wrote: > Dear matplotlib/numpy gurus, > > This is my first attempt to use matplotlib & numpy, with the following > small piece of code: > > #!/usr/bin/env jython **** jython because of the import of stilts > from starjava **** > > # Fabrique les figures des champs du papier Catalogue > > > __author__="marquett" > __date__ ="$16 mai 2011 12:17:08$" > > import sys > sys.path.append('/star/starjava/etc/ttools') > sys.path.append('/Library/Frameworks/Python.framework//Versions/2.6/lib/python2.6/site-packages') > import stilts > import glob > import matplotlib.pyplot as plt > > I got the following trace: > > Traceback (most recent call last): > File "/Users/marquett/Downloads/FigCat/src/figcat.py", line 14, in > > import matplotlib.pyplot as plt > File > "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/matplotlib/__init__.py", > line 135, in > from matplotlib.rcsetup import (defaultParams, > File > "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/matplotlib/rcsetup.py", > line 19, in > from matplotlib.colors import is_color_like > File > "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/matplotlib/colors.py", > line 52, in > import numpy as np > File > "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/numpy/__init__.py", > line 137, in > import add_newdocs > File > "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/numpy/add_newdocs.py", > line 9, in > from numpy.lib import add_newdoc > File > "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/numpy/lib/__init__.py", > line 4, in > from type_check import * > File > "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/numpy/lib/type_check.py", > line 8, in > import numpy.core.numeric as _nx > File > "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/numpy/core/__init__.py", > line 5, in > import multiarray > ImportError: No module named multiarray > > I use the latest 1.6.0 version of numpy and 1.0.0 one of matplotlib. > > My so little experience tells me that this is a deep numpy issue > rather than a matplotlib one. > > Any hint welcome, thanks. > > JB Marquette > > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy -- Thomas Boch CDS/Observatoire Astronomique Phone : 33 (0)3 68 85 24 42 11, rue de l'Universite Fax : 33 (0)3 68 85 24 17 F-67000 Strasbourg Email : thomas.boch at astro.unistra.fr France http://cdsweb.u-strasbg.fr/~boch -------------- next part -------------- An HTML attachment was scrubbed... URL: From derek at astro.physik.uni-goettingen.de Mon May 16 11:55:59 2011 From: derek at astro.physik.uni-goettingen.de (Derek Homeier) Date: Mon, 16 May 2011 17:55:59 +0200 Subject: [AstroPy] import problem (matplotlib ? numpy ?) on Mac Intel, python 2.6 In-Reply-To: References: Message-ID: <66CD2FDD-337A-4F57-AE5B-D195C0830EBE@astro.physik.uni-goettingen.de> On 16 May 2011, at 17:02, Jean-Baptiste Marquette wrote: > File "/Library/Frameworks/Python.framework/Versions/2.6/lib/ > python2.6/site-packages/numpy/core/__init__.py", line 5, in > import multiarray > ImportError: No module named multiarray > > I use the latest 1.6.0 version of numpy and 1.0.0 one of matplotlib. > > My so little experience tells me that this is a deep numpy issue > rather than a matplotlib one. > If you have a complete installation of numpy, multiarray should be present there in ./numpy/core as well - this might point to a version problem (like 32 bit vs. 64 bit)... Could you get more information on the architecture e.g. with file /Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/ site-packages/numpy/core/multiarray.so file $(which jython) HTH, Derek From marquett at iap.fr Mon May 16 12:25:30 2011 From: marquett at iap.fr (Jean-Baptiste Marquette) Date: Mon, 16 May 2011 18:25:30 +0200 Subject: [AstroPy] import problem (matplotlib ? numpy ?) on Mac Intel, python 2.6 In-Reply-To: <4DD142BF.2040208@astro.unistra.fr> References: <4DD142BF.2040208@astro.unistra.fr> Message-ID: Hi Thomas, AtPy looks great. My only concern with starjava was to use stilts functionalities for managing ascii tables and coordinates transformations from degrees to sexagesimal and reverse. But I guess that AtPy deals with headers having a leading "#" and names of columns, like # id ra_min ra_max dec_min dec_max and that pywcs can do transformations. So I can switch to pure python scripts. Cheers, JB > Hi Jean-Baptiste, > > I don't think numpy is currently supported by Jython. > > I do not know exactly what is your use of starjava, but you might want to have a look at AtPy (http://atpy.github.com/) which does a pretty good job at manipulating (astronomical) tables. > > Cheers, > > Thomas Boch > > On 16/05/11 17:02, Jean-Baptiste Marquette wrote: >> >> Dear matplotlib/numpy gurus, >> >> This is my first attempt to use matplotlib & numpy, with the following small piece of code: >> >> #!/usr/bin/env jython **** jython because of the import of stilts from starjava **** >> >> # Fabrique les figures des champs du papier Catalogue >> >> >> __author__="marquett" >> __date__ ="$16 mai 2011 12:17:08$" >> >> import sys >> sys.path.append('/star/starjava/etc/ttools') >> sys.path.append('/Library/Frameworks/Python.framework//Versions/2.6/lib/python2.6/site-packages') >> import stilts >> import glob >> import matplotlib.pyplot as plt >> >> I got the following trace: >> >> Traceback (most recent call last): >> File "/Users/marquett/Downloads/FigCat/src/figcat.py", line 14, in >> import matplotlib.pyplot as plt >> File "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/matplotlib/__init__.py", line 135, in >> from matplotlib.rcsetup import (defaultParams, >> File "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/matplotlib/rcsetup.py", line 19, in >> from matplotlib.colors import is_color_like >> File "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/matplotlib/colors.py", line 52, in >> import numpy as np >> File "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/numpy/__init__.py", line 137, in >> import add_newdocs >> File "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/numpy/add_newdocs.py", line 9, in >> from numpy.lib import add_newdoc >> File "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/numpy/lib/__init__.py", line 4, in >> from type_check import * >> File "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/numpy/lib/type_check.py", line 8, in >> import numpy.core.numeric as _nx >> File "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/numpy/core/__init__.py", line 5, in >> import multiarray >> ImportError: No module named multiarray >> >> I use the latest 1.6.0 version of numpy and 1.0.0 one of matplotlib. >> >> My so little experience tells me that this is a deep numpy issue rather than a matplotlib one. >> >> Any hint welcome, thanks. >> >> JB Marquette >> >> >> _______________________________________________ >> AstroPy mailing list >> AstroPy at scipy.org >> http://mail.scipy.org/mailman/listinfo/astropy > > > -- > Thomas Boch > CDS/Observatoire Astronomique Phone : 33 (0)3 68 85 24 42 > 11, rue de l'Universite Fax : 33 (0)3 68 85 24 17 > F-67000 Strasbourg Email : thomas.boch at astro.unistra.fr > France http://cdsweb.u-strasbg.fr/~boch -------------- next part -------------- An HTML attachment was scrubbed... URL: From marquett at iap.fr Mon May 16 12:37:00 2011 From: marquett at iap.fr (Jean-Baptiste Marquette) Date: Mon, 16 May 2011 18:37:00 +0200 Subject: [AstroPy] import problem (matplotlib ? numpy ?) on Mac Intel, python 2.6 In-Reply-To: <66CD2FDD-337A-4F57-AE5B-D195C0830EBE@astro.physik.uni-goettingen.de> References: <66CD2FDD-337A-4F57-AE5B-D195C0830EBE@astro.physik.uni-goettingen.de> Message-ID: <9630F6C1-F380-4360-9D32-B4939B7D3E9E@iap.fr> Hi Derek, Thank you for your suggestion. The file command yields: file /Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/numpy/core/multiarray.so /Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/numpy/core/multiarray.so: Mach-O universal binary with 2 architectures /Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/numpy/core/multiarray.so (for architecture ppc): Mach-O bundle ppc /Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/numpy/core/multiarray.so (for architecture i386): Mach-O bundle i386 file $(which jython) /usr/local/bin/jython: a bash script text executable jython Jython 2.5.2 (Release_2_5_2:7206, Mar 2 2011, 23:12:06) [Java HotSpot(TM) 64-Bit Server VM (Apple Inc.)] on java1.6.0_24 Maybe a problem between i386 and 64-Bit Server VM ?... Cheers, JB Le 16 mai 2011 ? 17:55, Derek Homeier a ?crit : > On 16 May 2011, at 17:02, Jean-Baptiste Marquette wrote: > >> File "/Library/Frameworks/Python.framework/Versions/2.6/lib/ >> python2.6/site-packages/numpy/core/__init__.py", line 5, in >> import multiarray >> ImportError: No module named multiarray >> >> I use the latest 1.6.0 version of numpy and 1.0.0 one of matplotlib. >> >> My so little experience tells me that this is a deep numpy issue >> rather than a matplotlib one. >> > If you have a complete installation of numpy, multiarray should be > present there > in ./numpy/core as well - this might point to a version problem (like > 32 bit vs. 64 bit)... > Could you get more information on the architecture e.g. with > > file /Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/ > site-packages/numpy/core/multiarray.so > file $(which jython) > > HTH, > Derek > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy From marquett at iap.fr Mon May 16 13:03:47 2011 From: marquett at iap.fr (Jean-Baptiste Marquette) Date: Mon, 16 May 2011 19:03:47 +0200 Subject: [AstroPy] import problem (matplotlib ? numpy ?) on Mac Intel, python 2.6 In-Reply-To: <66CD2FDD-337A-4F57-AE5B-D195C0830EBE@astro.physik.uni-goettingen.de> References: <66CD2FDD-337A-4F57-AE5B-D195C0830EBE@astro.physik.uni-goettingen.de> Message-ID: <839D8577-B132-457F-82AF-3E02B6FF9C45@iap.fr> Hi all, I replaced jython by python in my script, and I got the same error from import atpy. I tried the 1.5.1 version of Numpy, without success. It seems this is more serious than previously expected... There is something wrong in the kingdom of numpy... Cheers JB I got the same problem Le 16 mai 2011 ? 17:55, Derek Homeier a ?crit : > On 16 May 2011, at 17:02, Jean-Baptiste Marquette wrote: > >> File "/Library/Frameworks/Python.framework/Versions/2.6/lib/ >> python2.6/site-packages/numpy/core/__init__.py", line 5, in >> import multiarray >> ImportError: No module named multiarray >> >> I use the latest 1.6.0 version of numpy and 1.0.0 one of matplotlib. >> >> My so little experience tells me that this is a deep numpy issue >> rather than a matplotlib one. >> > If you have a complete installation of numpy, multiarray should be > present there > in ./numpy/core as well - this might point to a version problem (like > 32 bit vs. 64 bit)... > Could you get more information on the architecture e.g. with > > file /Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/ > site-packages/numpy/core/multiarray.so > file $(which jython) > > HTH, > Derek > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy From thomas.robitaille at gmail.com Mon May 16 13:25:07 2011 From: thomas.robitaille at gmail.com (Thomas Robitaille) Date: Mon, 16 May 2011 13:25:07 -0400 Subject: [AstroPy] import problem (matplotlib ? numpy ?) on Mac Intel, python 2.6 In-Reply-To: <839D8577-B132-457F-82AF-3E02B6FF9C45@iap.fr> References: <66CD2FDD-337A-4F57-AE5B-D195C0830EBE@astro.physik.uni-goettingen.de> <839D8577-B132-457F-82AF-3E02B6FF9C45@iap.fr> Message-ID: Hi JB, How did you install Numpy? I've seen that kind of error before when there is something wrong with the compilation of the C code for Numpy. Cheers, Tom On May 16, 2011, at 1:03 PM, Jean-Baptiste Marquette wrote: > Hi all, > > I replaced jython by python in my script, and I got the same error from import atpy. > > I tried the 1.5.1 version of Numpy, without success. > > It seems this is more serious than previously expected... There is something wrong in the kingdom of numpy... > > Cheers > JB > > I got the same problem > Le 16 mai 2011 ? 17:55, Derek Homeier a ?crit : > >> On 16 May 2011, at 17:02, Jean-Baptiste Marquette wrote: >> >>> File "/Library/Frameworks/Python.framework/Versions/2.6/lib/ >>> python2.6/site-packages/numpy/core/__init__.py", line 5, in >>> import multiarray >>> ImportError: No module named multiarray >>> >>> I use the latest 1.6.0 version of numpy and 1.0.0 one of matplotlib. >>> >>> My so little experience tells me that this is a deep numpy issue >>> rather than a matplotlib one. >>> >> If you have a complete installation of numpy, multiarray should be >> present there >> in ./numpy/core as well - this might point to a version problem (like >> 32 bit vs. 64 bit)... >> Could you get more information on the architecture e.g. with >> >> file /Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/ >> site-packages/numpy/core/multiarray.so >> file $(which jython) >> >> HTH, >> Derek >> >> _______________________________________________ >> AstroPy mailing list >> AstroPy at scipy.org >> http://mail.scipy.org/mailman/listinfo/astropy > > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy From marquett at iap.fr Mon May 16 14:18:29 2011 From: marquett at iap.fr (Jean-Baptiste Marquette) Date: Mon, 16 May 2011 20:18:29 +0200 Subject: [AstroPy] import problem (matplotlib ? numpy ?) on Mac Intel, python 2.6 In-Reply-To: References: <66CD2FDD-337A-4F57-AE5B-D195C0830EBE@astro.physik.uni-goettingen.de> <839D8577-B132-457F-82AF-3E02B6FF9C45@iap.fr> Message-ID: <1E056F5B-CE04-4E3A-93F4-B312EE7E1B39@iap.fr> Hi Tom, You made my day. I installed from the default Mac disk image file. I recompiled from source archive with -arch x86_64 and fixed the problem. It works from a terminal, not from NetBeans 6.9.1. However, I made a giant leap today ! Thanks, JB > Hi JB, > > How did you install Numpy? I've seen that kind of error before when there is something wrong with the compilation of the C code for Numpy. > > Cheers, > Tom > > On May 16, 2011, at 1:03 PM, Jean-Baptiste Marquette wrote: > >> Hi all, >> >> I replaced jython by python in my script, and I got the same error from import atpy. >> >> I tried the 1.5.1 version of Numpy, without success. >> >> It seems this is more serious than previously expected... There is something wrong in the kingdom of numpy... >> >> Cheers >> JB >> >> I got the same problem >> Le 16 mai 2011 ? 17:55, Derek Homeier a ?crit : >> >>> On 16 May 2011, at 17:02, Jean-Baptiste Marquette wrote: >>> >>>> File "/Library/Frameworks/Python.framework/Versions/2.6/lib/ >>>> python2.6/site-packages/numpy/core/__init__.py", line 5, in >>>> import multiarray >>>> ImportError: No module named multiarray >>>> >>>> I use the latest 1.6.0 version of numpy and 1.0.0 one of matplotlib. >>>> >>>> My so little experience tells me that this is a deep numpy issue >>>> rather than a matplotlib one. >>>> >>> If you have a complete installation of numpy, multiarray should be >>> present there >>> in ./numpy/core as well - this might point to a version problem (like >>> 32 bit vs. 64 bit)... >>> Could you get more information on the architecture e.g. with >>> >>> file /Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/ >>> site-packages/numpy/core/multiarray.so >>> file $(which jython) >>> >>> HTH, >>> Derek >>> >>> _______________________________________________ >>> AstroPy mailing list >>> AstroPy at scipy.org >>> http://mail.scipy.org/mailman/listinfo/astropy >> >> >> _______________________________________________ >> AstroPy mailing list >> AstroPy at scipy.org >> http://mail.scipy.org/mailman/listinfo/astropy > ================================================ Bien cordialement/Very truly yours/Mit freundlichen Gruessen, Jean-Baptiste Marquette Institut d'Astrophysique de Paris CNRS - UMR 7095 Universit? Pierre & Marie Curie 98bis Bd Arago 75014 Paris - France Tel : +33 (0) 1 4432 8196 Fax : +33 (0) 1 4432 8001 Web : http://www2.iap.fr/users/marquett ================================================ From tevaughan at gmail.com Tue May 17 13:12:44 2011 From: tevaughan at gmail.com (Thomas Vaughan) Date: Tue, 17 May 2011 11:12:44 -0600 Subject: [AstroPy] synthetic photometry Message-ID: I'm looking for a convenient package that I can use to do synthetic photometry. I thought that I'd probably write one myself, but I should at least see what's available. Browsing the Web, I came upon astrolib. Can it already be used for synthetic photometry? I need to make a star catalog by starting from a subset of Tycho-2 photometry and spectroscopic classifications. I have the overall response for my instrument. I need to define a magnitude system for the instrument and to predict the instrument magnitude for each star in the catalog. -- Thomas E. Vaughan From erik.tollerud at gmail.com Tue May 17 14:20:37 2011 From: erik.tollerud at gmail.com (Erik Tollerud) Date: Tue, 17 May 2011 11:20:37 -0700 Subject: [AstroPy] synthetic photometry In-Reply-To: References: Message-ID: Have you looked at pysynphot (http://stsdas.stsci.edu/pysynphot/)? I'm not sure if that does everything you want, but it's probably the most complete option right now if you don't want to write your own... On Tue, May 17, 2011 at 10:12 AM, Thomas Vaughan wrote: > I'm looking for a convenient package that I can use to do synthetic photometry. > > I thought that I'd probably write one myself, but I should at least > see what's available. > > Browsing the Web, I came upon astrolib. > > Can it already be used for synthetic photometry? > > I need to make a star catalog by starting from a subset of Tycho-2 > photometry and spectroscopic classifications. I have the overall > response for my instrument. I need to define a magnitude system for > the instrument and to predict the instrument magnitude for each star > in the catalog. > > -- > Thomas E. Vaughan > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy > -- Erik Tollerud From laidler at stsci.edu Tue May 17 14:22:59 2011 From: laidler at stsci.edu (Vicki Laidler) Date: Tue, 17 May 2011 18:22:59 +0000 Subject: [AstroPy] synthetic photometry In-Reply-To: References: Message-ID: <173C0F068F6E49449797E90DF0B59EF8B5899C@EXCHMAIL1.stsci.edu> Hi Thomas, Indeed, astrolib includes pysynphot, which is presently being used under the hood of the HST Exposure Time Calculators. It's intended as a replacement for the IRAF STSDAS synphot package, and is largely complete. I'm the primary developer for pysynphot at this time. You can certainly accomplish the task you describe below with pysynphot. You would need to create a throughput file (wavelength and throughput in dimensionless units between 0 and 1) for each band in your instrument, and a spectrum file or array (wavelength and flux) or black body as a function of temperature. Once you have those, you can simulate an observation and predict the magnitude, something like this: import pysynphot as psyn bp = psyn.FileBandpass('your_throughput_file.fits') sp = psyn.FileSpectrum('spectrum_file.fits') obs = psyn.Observation(sp, bp) obs.effstim('abmag') #gives you the observed magnitude There is some documentation browsable linked from the astrolib wiki at http://stsdas.stsci.edu/pysynphot/ - it is incomplete but should be enough to get you started. Suggestions for improving the documentation are also very welcome. :) You can get more help here or from help at stsci.edu. cheers, Vicki Laidler ________________________________________ From: astropy-bounces at scipy.org [astropy-bounces at scipy.org] on behalf of Thomas Vaughan [tevaughan at gmail.com] Sent: Tuesday, May 17, 2011 1:12 PM To: astropy at scipy.org Subject: [AstroPy] synthetic photometry I'm looking for a convenient package that I can use to do synthetic photometry. I thought that I'd probably write one myself, but I should at least see what's available. Browsing the Web, I came upon astrolib. Can it already be used for synthetic photometry? I need to make a star catalog by starting from a subset of Tycho-2 photometry and spectroscopic classifications. I have the overall response for my instrument. I need to define a magnitude system for the instrument and to predict the instrument magnitude for each star in the catalog. -- Thomas E. Vaughan _______________________________________________ AstroPy mailing list AstroPy at scipy.org http://mail.scipy.org/mailman/listinfo/astropy From eric at depagne.org Mon May 23 09:46:45 2011 From: eric at depagne.org (=?utf-8?q?=C3=89ric_Depagne?=) Date: Mon, 23 May 2011 15:46:45 +0200 Subject: [AstroPy] Access to compressed data in FITS file. Message-ID: <201105231546.46009.eric@depagne.org> Hi all. I have some fits files that contain compressed data. If I open them using pyfits.open, I get the following: In [17]: f Out[17]: [, , , , ] and then : In [18]: f[1].data Out[18]: FITS_rec([ (array([ 0, 0, 0, ..., 11, 6, 0], dtype=uint8), 7.8563242345474249, 81.508781433105469), (array([ 0, 0, 0, ..., 217, 137, 160], dtype=uint8), 6.1135416774841307, -748.53662109375), (array([ 0, 0, 0, ..., 90, 112, 128], dtype=uint8), 6.308497703138733, -115.72266387939453), ..., (array([ 0, 0, 0, ..., 109, 195, 0], dtype=uint8), 6.7486103076454169, -17.52739143371582), (array([ 0, 0, 0, ..., 92, 198, 0], dtype=uint8), 6.7442002691665648, -20.605709075927734), (array([ 0, 0, 0, ..., 210, 137, 232], dtype=uint8), 6.9185699581314086, -33.143478393554688)], dtype=[('COMPRESSED_DATA', '>i4', 2), ('ZSCALE', '>f8'), ('ZZERO', '>f8')]) I have a script that reads both compressed and uncompressed fits file, and of course, it fails with the compressed ones. Does anybody knows, either how to access the data in the compressed case, or know a pointer wher I could find the info. Thanks a lot. ?ric. Un clavier azerty en vaut deux ---------------------------------------------------------- ?ric Depagne eric at depagne.org From taro at ap.smu.ca Mon May 23 10:05:53 2011 From: taro at ap.smu.ca (Taro Sato) Date: Mon, 23 May 2011 11:05:53 -0300 Subject: [AstroPy] Access to compressed data in FITS file. In-Reply-To: <201105231546.46009.eric@depagne.org> References: <201105231546.46009.eric@depagne.org> Message-ID: On Mon, May 23, 2011 at 10:46 AM, ?ric Depagne wrote: > Hi all. > > I have some fits files that contain compressed data. > If I open them using pyfits.open, I get the following: > In [17]: f > Out[17]: > [, > ?, > ?, > ?, > ?] > > and then : > In [18]: f[1].data > Out[18]: > FITS_rec([ (array([ 0, ?0, ?0, ..., 11, ?6, ?0], dtype=uint8), > 7.8563242345474249, 81.508781433105469), > ? ? ? (array([ ?0, ? 0, ? 0, ..., 217, 137, 160], dtype=uint8), > 6.1135416774841307, -748.53662109375), > ? ? ? (array([ ?0, ? 0, ? 0, ..., ?90, 112, 128], dtype=uint8), > 6.308497703138733, -115.72266387939453), > ? ? ? ..., > ? ? ? (array([ ?0, ? 0, ? 0, ..., 109, 195, ? 0], dtype=uint8), > 6.7486103076454169, -17.52739143371582), > ? ? ? (array([ ?0, ? 0, ? 0, ..., ?92, 198, ? 0], dtype=uint8), > 6.7442002691665648, -20.605709075927734), > ? ? ? (array([ ?0, ? 0, ? 0, ..., 210, 137, 232], dtype=uint8), > 6.9185699581314086, -33.143478393554688)], > ? ? ?dtype=[('COMPRESSED_DATA', '>i4', 2), ('ZSCALE', '>f8'), ('ZZERO', > '>f8')]) > > I have a script that reads both compressed and uncompressed fits file, and of > course, it fails with the compressed ones. > > Does anybody knows, either how to access the data in the compressed case, or > know a pointer wher I could find the info. > > Thanks a lot. > > ?ric. > Un clavier azerty en vaut deux > ---------------------------------------------------------- > ?ric Depagne ? ? ? ? ? ? ? ? ? ? ? ? ? ?eric at depagne.org > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy > The PyFITS manual discusses how to deal with compressed data: http://stsdas.stsci.edu/download/wikidocs/The_PyFITS_Handbook.pdf Is this not enough? -- Taro Sato Department of Astronomy & Physics Saint Mary's University? ? ? ? ? ?? email: taro at ap.smu.ca Halifax, NS B3H 3C3? ? ? ? ? ? ? ?? phone: (902)420-5027 Canada? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? web: http://ap.smu.ca/~taro From eric at depagne.org Mon May 23 10:13:31 2011 From: eric at depagne.org (=?utf-8?q?=C3=89ric_Depagne?=) Date: Mon, 23 May 2011 16:13:31 +0200 Subject: [AstroPy] Access to compressed data in FITS file. In-Reply-To: References: <201105231546.46009.eric@depagne.org> Message-ID: <201105231613.31623.eric@depagne.org> Hi Taro, > The PyFITS manual discusses how to deal with compressed data: > > http://stsdas.stsci.edu/download/wikidocs/The_PyFITS_Handbook.pdf > > Is this not enough? I had already looked at it (sect 7.4.1 and 7.4.2 deals with compressed data), but what puzzles me is that the ouput of their series of command has nothing to do with mine. For instance, my f.info() give me the following : In [20]: f.info() Filename: file.fitz No. Name Type Cards Dimensions Format 0 PRIMARY PrimaryHDU 68 (0,) uint8 1 BinTableHDU 36 2032R x 3C ['1PB(1291)', '1D', '1D'] 2 BinTableHDU 36 2032R x 3C ['1PB(1273)', '1D', '1D'] 3 BinTableHDU 36 2032R x 3C [1PB(1264), 1D, 1D] 4 BinTableHDU 36 2032R x 3C [1PB(1271), 1D, 1D] And their f[1].data output is different from mine in the sense that they do not have a FITS_rec (at least that's my understanding) but directly the numpy.array So I wonder what my file is ! ?ric. -- Un clavier azerty en vaut deux ---------------------------------------------------------- ?ric Depagne eric at depagne.org From aldcroft at head.cfa.harvard.edu Mon May 23 10:18:49 2011 From: aldcroft at head.cfa.harvard.edu (Tom Aldcroft) Date: Mon, 23 May 2011 10:18:49 -0400 Subject: [AstroPy] Practical Python for Astronomers web tutorial Message-ID: My colleagues and I would like to announce the availability of a web tutorial aimed at teaching Python to astronomers through a series of interactive workshops: http://python4astronomers.github.com/ Practical Python for Astronomers is a series of hands-on workshops to explore the Python language and the analysis tools it provides. The emphasis is on using Python to solve real-world problems that astronomers are likely to encounter in research. Some features: - Workshops immediately use plotting, analysis, and file reading tools. - Along the way elements of the Python language are introduced. - Workshops are interactive using examples run by participants on their laptops. - Comprehensive instructions a given for installing a full Python environment. There are two goals. First is to provide tutorials suitable for self-study by those wishing to learn Python for astronomy. The greater goal is for those knowledgable in Python to teach the workshop series at their local institutions, adapting the content as desired. To that end we have developed the content in Sphinx RestructuredText and hosted the source on github at https://github.com/python4astronomers/. Anyone interested can clone the repository or download a tarball and make modifications needed to present the material locally. We would also welcome comments, fixes, or suggestions for improvement. This can be done as a Github issue or pull request, or by sending email to aldcroft at head.cfa.harvard.edu. The workshop material here was presented in the Spring of 2011 at the Harvard / Smithsonian Center for Astrophysics. A range of about 25 to 50 people participated in the different workshops, which were 1.5 hours in duration. One key accomplishment was installing a working Python with NumPy, SciPy, and IPython on over 50 laptops (MacOS, linux, and Windows) during a single session. Tom Aldcroft Tom Robitaille Brian Refsdal Gus Muench From eric at depagne.org Mon May 23 10:33:36 2011 From: eric at depagne.org (=?utf-8?q?=C3=89ric_Depagne?=) Date: Mon, 23 May 2011 16:33:36 +0200 Subject: [AstroPy] Access to compressed data in FITS file. In-Reply-To: <201105231546.46009.eric@depagne.org> References: <201105231546.46009.eric@depagne.org> Message-ID: <201105231633.36471.eric@depagne.org> Erik Bray gave me the solution: the installed pyfits version on the machine I'm opening the file is way too old. Doing that on a more updated one give me access to the data as expected. Thanks all for your help. ?ric. -- Un clavier azerty en vaut deux ---------------------------------------------------------- ?ric Depagne eric at depagne.org From jturner at gemini.edu Mon May 23 12:51:20 2011 From: jturner at gemini.edu (James Turner) Date: Mon, 23 May 2011 12:51:20 -0400 Subject: [AstroPy] Practical Python for Astronomers web tutorial In-Reply-To: References: Message-ID: <4DDA9088.1050707@gemini.edu> Hi Tom, Thanks, that sounds great (the link is currently timing out for me though, so I can't look at it). Will post locally. Ah, the link just started working... Cheers, James. On 23/05/11 10:18, Tom Aldcroft wrote: > My colleagues and I would like to announce the availability of a web > tutorial aimed at teaching Python to astronomers through a series of > interactive workshops: > > http://python4astronomers.github.com/ > > Practical Python for Astronomers is a series of hands-on workshops to > explore the Python language and the analysis tools it provides. The > emphasis is on using Python to solve real-world problems that > astronomers are likely to encounter in research. Some features: > > - Workshops immediately use plotting, analysis, and file reading tools. > - Along the way elements of the Python language are introduced. > - Workshops are interactive using examples run by participants on their laptops. > - Comprehensive instructions a given for installing a full Python environment. > > There are two goals. First is to provide tutorials suitable for > self-study by those wishing to learn Python for astronomy. The > greater goal is for those knowledgable in Python to teach the workshop > series at their local institutions, adapting the content as desired. > To that end we have developed the content in Sphinx RestructuredText > and hosted the source on github at > https://github.com/python4astronomers/. Anyone interested can clone > the repository or download a tarball and make modifications needed to > present the material locally. We would also welcome comments, fixes, > or suggestions for improvement. This can be done as a Github issue or > pull request, or by sending email to aldcroft at head.cfa.harvard.edu. > > The workshop material here was presented in the Spring of 2011 at the > Harvard / Smithsonian Center for Astrophysics. A range of about 25 to > 50 people participated in the different workshops, which were 1.5 > hours in duration. One key accomplishment was installing a working > Python with NumPy, SciPy, and IPython on over 50 laptops (MacOS, > linux, and Windows) during a single session. > > Tom Aldcroft > Tom Robitaille > Brian Refsdal > Gus Muench > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy From jturner at gemini.edu Mon May 23 12:51:31 2011 From: jturner at gemini.edu (James Turner) Date: Mon, 23 May 2011 12:51:31 -0400 Subject: [AstroPy] Practical Python for Astronomers web tutorial In-Reply-To: References: Message-ID: <4DDA9093.9000901@gemini.edu> Hi Tom, Thanks, that sounds great (the link is currently timing out for me though, so I can't look at it). Will post locally. Ah, the link just started working... Cheers, James. On 23/05/11 10:18, Tom Aldcroft wrote: > My colleagues and I would like to announce the availability of a web > tutorial aimed at teaching Python to astronomers through a series of > interactive workshops: > > http://python4astronomers.github.com/ > > Practical Python for Astronomers is a series of hands-on workshops to > explore the Python language and the analysis tools it provides. The > emphasis is on using Python to solve real-world problems that > astronomers are likely to encounter in research. Some features: > > - Workshops immediately use plotting, analysis, and file reading tools. > - Along the way elements of the Python language are introduced. > - Workshops are interactive using examples run by participants on their laptops. > - Comprehensive instructions a given for installing a full Python environment. > > There are two goals. First is to provide tutorials suitable for > self-study by those wishing to learn Python for astronomy. The > greater goal is for those knowledgable in Python to teach the workshop > series at their local institutions, adapting the content as desired. > To that end we have developed the content in Sphinx RestructuredText > and hosted the source on github at > https://github.com/python4astronomers/. Anyone interested can clone > the repository or download a tarball and make modifications needed to > present the material locally. We would also welcome comments, fixes, > or suggestions for improvement. This can be done as a Github issue or > pull request, or by sending email to aldcroft at head.cfa.harvard.edu. > > The workshop material here was presented in the Spring of 2011 at the > Harvard / Smithsonian Center for Astrophysics. A range of about 25 to > 50 people participated in the different workshops, which were 1.5 > hours in duration. One key accomplishment was installing a working > Python with NumPy, SciPy, and IPython on over 50 laptops (MacOS, > linux, and Windows) during a single session. > > Tom Aldcroft > Tom Robitaille > Brian Refsdal > Gus Muench > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy From cabanela at mnstate.edu Mon May 23 16:38:29 2011 From: cabanela at mnstate.edu (Dr. Juan E Cabanela Ph.D.) Date: Mon, 23 May 2011 15:38:29 -0500 Subject: [AstroPy] Question about Atpy with MySQL Message-ID: <06336FF0-7C9F-47ED-A510-B3AC7FB81254@mnstate.edu> I am new to python, trying to convert myself from the pain that is the current combination of perl, C, FORTRAN, and MySQL that I use for my research. I have been trying to implement a MySQL query in ATpy to my MySQL 5.5.12 database. I have ATpy 0.9.5.1 and MySQL-Python 1.2.3 installed. Whenever I try to issue a query like the following (in my case in the 'ipython -pylab' environment): import atpy t = atpy.Table('mysql', user='juan', password='XXXX', database='BigGalaxies', \ table='CompareParams', \ query='SELECT * FROM CompareParams;' ) I get the following error: TypeError: 'password' is an invalid keyword argument for this function If I am interpreting the trace error correctly, the ultimate error get traced to MySQLdb/connections.pyc. I suspect this means my MySQL-python installation is bad. Does anyone have any advice/wisdom for resolving this sort of error? Juan -- Dr. Juan Cabanela 218-477-2453 (V) 218-477-2290 (F) Minnesota State University Moorhead WWW: http://www.cabanela.com/ Dept. of Physics and Astronomy Twitter: Juan_Kinda_Guy 1104 Seventh Ave South, Hagen 307B IM: AstroJuanCab (AIM) Moorhead, MN 56563 cabanela at mnstate.edu (MSN) juancab at gmail.com (GTalk) Public PGP Key available at: http://www.cabanela.com/juan_public.asc From thomas.robitaille at gmail.com Mon May 23 18:40:06 2011 From: thomas.robitaille at gmail.com (Thomas Robitaille) Date: Mon, 23 May 2011 18:40:06 -0400 Subject: [AstroPy] Question about Atpy with MySQL In-Reply-To: <06336FF0-7C9F-47ED-A510-B3AC7FB81254@mnstate.edu> References: <06336FF0-7C9F-47ED-A510-B3AC7FB81254@mnstate.edu> Message-ID: Hi Juan, The issue is that the options for MySQL in ATpy are not user/password/database, but user/passwd/db. See the full API here: http://atpy.github.com/format_sql.html So in your case, try: import atpy t = atpy.Table('mysql', user='juan', passwd='XXXX', db='BigGalaxies', \ table='CompareParams', \ query='SELECT * FROM CompareParams;' ) By the way, if the query is just to select all rows from a table, you don't need to specify query="select * from table;" as that is the default. Let me know if this does not solve your issue! Cheers, Tom On Monday, May 23, 2011 at 4:38 PM, Dr. Juan E Cabanela Ph.D. wrote: > I am new to python, trying to convert myself from the pain that is the current combination of perl, C, FORTRAN, and MySQL that I use for my research. > > I have been trying to implement a MySQL query in ATpy to my MySQL 5.5.12 database. I have ATpy 0.9.5.1 and MySQL-Python 1.2.3 installed. > > Whenever I try to issue a query like the following (in my case in the 'ipython -pylab' environment): > > import atpy > t = atpy.Table('mysql', user='juan', password='XXXX', database='BigGalaxies', \ > table='CompareParams', \ > query='SELECT * FROM CompareParams;' ) > > I get the following error: > > TypeError: 'password' is an invalid keyword argument for this function > > If I am interpreting the trace error correctly, the ultimate error get > traced to MySQLdb/connections.pyc. I suspect this means my MySQL-python installation > is bad. > > Does anyone have any advice/wisdom for resolving this sort of error? > > Juan > > -- > Dr. Juan Cabanela 218-477-2453 (V) 218-477-2290 (F) > Minnesota State University Moorhead WWW: http://www.cabanela.com/ > Dept. of Physics and Astronomy Twitter: Juan_Kinda_Guy > 1104 Seventh Ave South, Hagen 307B IM: AstroJuanCab (AIM) > Moorhead, MN 56563 cabanela at mnstate.edu (MSN) > juancab at gmail.com (GTalk) > Public PGP Key available at: http://www.cabanela.com/juan_public.asc > > > > > > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy > From cabanela at mnstate.edu Mon May 23 18:55:29 2011 From: cabanela at mnstate.edu (Dr. Juan E Cabanela Ph.D.) Date: Mon, 23 May 2011 17:55:29 -0500 Subject: [AstroPy] Question about Atpy with MySQL In-Reply-To: References: <06336FF0-7C9F-47ED-A510-B3AC7FB81254@mnstate.edu> Message-ID: Tom, Thank you for catching that, I will try again when I get back to my computer (this is from my iPad). I will note the documentation at http://atpy.github.com/format_sql.html is in error if what you say is correct. It states as an example: >>> t = atpy.TableSet('mysql', user='monty', password='spam', \ database='measurements') and all the other examples on that page state user/password/database, although the API documentation below that does in fact show user/passwd/db. Juan On May 23, 2011, at 5:40 PM, Thomas Robitaille wrote: > Hi Juan, > > The issue is that the options for MySQL in ATpy are not user/password/database, but user/passwd/db. See the full API here: > > http://atpy.github.com/format_sql.html > > So in your case, try: > > import atpy > t = atpy.Table('mysql', user='juan', passwd='XXXX', db='BigGalaxies', \ > table='CompareParams', \ > query='SELECT * FROM CompareParams;' ) > > By the way, if the query is just to select all rows from a table, you don't need to specify query="select * from table;" as that is the default. > > Let me know if this does not solve your issue! > > Cheers, > Tom > > > On Monday, May 23, 2011 at 4:38 PM, Dr. Juan E Cabanela Ph.D. wrote: >> I am new to python, trying to convert myself from the pain that is the current combination of perl, C, FORTRAN, and MySQL that I use for my research. >> >> I have been trying to implement a MySQL query in ATpy to my MySQL 5.5.12 database. I have ATpy 0.9.5.1 and MySQL-Python 1.2.3 installed. >> >> Whenever I try to issue a query like the following (in my case in the 'ipython -pylab' environment): >> >> import atpy >> t = atpy.Table('mysql', user='juan', password='XXXX', database='BigGalaxies', \ >> table='CompareParams', \ >> query='SELECT * FROM CompareParams;' ) >> >> I get the following error: >> >> TypeError: 'password' is an invalid keyword argument for this function >> >> If I am interpreting the trace error correctly, the ultimate error get >> traced to MySQLdb/connections.pyc. I suspect this means my MySQL-python installation >> is bad. >> >> Does anyone have any advice/wisdom for resolving this sort of error? >> >> Juan >> >> -- >> Dr. Juan Cabanela 218-477-2453 (V) 218-477-2290 (F) >> Minnesota State University Moorhead WWW: http://www.cabanela.com/ >> Dept. of Physics and Astronomy Twitter: Juan_Kinda_Guy >> 1104 Seventh Ave South, Hagen 307B IM: AstroJuanCab (AIM) >> Moorhead, MN 56563 cabanela at mnstate.edu (MSN) >> juancab at gmail.com (GTalk) >> Public PGP Key available at: http://www.cabanela.com/juan_public.asc >> >> >> >> >> >> >> _______________________________________________ >> AstroPy mailing list >> AstroPy at scipy.org >> http://mail.scipy.org/mailman/listinfo/astropy >> > -- Dr. Juan Cabanela 218-477-2453 (V) 218-477-2290 (F) Minnesota State University Moorhead WWW: http://www.cabanela.com/ Dept. of Physics and Astronomy Twitter: Juan_Kinda_Guy 1104 Seventh Ave South, Hagen 307B IM: AstroJuanCab (AIM) Moorhead, MN 56563 cabanela at mnstate.edu (MSN) juancab at gmail.com (GTalk) Public PGP Key available at: http://www.cabanela.com/juan_public.asc From erik.tollerud at gmail.com Mon May 23 19:40:55 2011 From: erik.tollerud at gmail.com (Erik Tollerud) Date: Mon, 23 May 2011 16:40:55 -0700 Subject: [AstroPy] Practical Python for Astronomers web tutorial In-Reply-To: References: Message-ID: Are you planning on having this be a "living document"? That is, do you intend to have this document evolve and want others to submit new sections or improvements for future workshops, or just things like small improvements and typo corrections? On Mon, May 23, 2011 at 7:18 AM, Tom Aldcroft wrote: > My colleagues and I would like to announce the availability of a web > tutorial aimed at teaching Python to astronomers through a series of > interactive workshops: > > ?http://python4astronomers.github.com/ > > Practical Python for Astronomers is a series of hands-on workshops to > explore the Python language and the analysis tools it provides. The > emphasis is on using Python to solve real-world problems that > astronomers are likely to encounter in research. ?Some features: > > - Workshops immediately use plotting, analysis, and file reading tools. > - Along the way elements of the Python language are introduced. > - Workshops are interactive using examples run by participants on their laptops. > - Comprehensive instructions a given for installing a full Python environment. > > There are two goals. ?First is to provide tutorials suitable for > self-study by those wishing to learn Python for astronomy. ?The > greater goal is for those knowledgable in Python to teach the workshop > series at their local institutions, adapting the content as desired. > To that end we have developed the content in Sphinx RestructuredText > and hosted the source on github at > https://github.com/python4astronomers/. Anyone interested can clone > the repository or download a tarball and make modifications needed to > present the material locally. ?We would also welcome comments, fixes, > or suggestions for improvement. This can be done as a Github issue or > pull request, or by sending email to aldcroft at head.cfa.harvard.edu. > > The workshop material here was presented in the Spring of 2011 at the > Harvard / Smithsonian Center for Astrophysics. A range of about 25 to > 50 people participated in the different workshops, which were 1.5 > hours in duration. ?One key accomplishment was installing a working > Python with NumPy, SciPy, and IPython on over 50 laptops (MacOS, > linux, and Windows) during a single session. > > Tom Aldcroft > Tom Robitaille > Brian Refsdal > Gus Muench > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy > -- Erik Tollerud From aldcroft at head.cfa.harvard.edu Mon May 23 20:28:01 2011 From: aldcroft at head.cfa.harvard.edu (Tom Aldcroft) Date: Mon, 23 May 2011 20:28:01 -0400 Subject: [AstroPy] Practical Python for Astronomers web tutorial In-Reply-To: References: Message-ID: The goal was definitely to make a living document that would be improved and expanded by contributions from the community. That was a key motivation for hosting the source and web site on github and writing it using Sphinx. Adding content is as easy as forking the repository, writing new material and building locally until you are happy, then submitting a github pull request. If someone became a regular contributor we would be happy to add them as a project member with full commit rights. Other methods for contributing would also work of course, e.g. emailing a tar file with new content. If there is interest along these lines please let me know or talk to me at SciPy. - Tom On Mon, May 23, 2011 at 7:40 PM, Erik Tollerud wrote: > Are you planning on having this be a "living document"? ?That is, do > you intend to have this document evolve and want others to submit new > sections or improvements for future workshops, or just things like > small improvements and typo corrections? > > On Mon, May 23, 2011 at 7:18 AM, Tom Aldcroft > wrote: >> My colleagues and I would like to announce the availability of a web >> tutorial aimed at teaching Python to astronomers through a series of >> interactive workshops: >> >> ?http://python4astronomers.github.com/ >> >> Practical Python for Astronomers is a series of hands-on workshops to >> explore the Python language and the analysis tools it provides. The >> emphasis is on using Python to solve real-world problems that >> astronomers are likely to encounter in research. ?Some features: >> >> - Workshops immediately use plotting, analysis, and file reading tools. >> - Along the way elements of the Python language are introduced. >> - Workshops are interactive using examples run by participants on their laptops. >> - Comprehensive instructions a given for installing a full Python environment. >> >> There are two goals. ?First is to provide tutorials suitable for >> self-study by those wishing to learn Python for astronomy. ?The >> greater goal is for those knowledgable in Python to teach the workshop >> series at their local institutions, adapting the content as desired. >> To that end we have developed the content in Sphinx RestructuredText >> and hosted the source on github at >> https://github.com/python4astronomers/. Anyone interested can clone >> the repository or download a tarball and make modifications needed to >> present the material locally. ?We would also welcome comments, fixes, >> or suggestions for improvement. This can be done as a Github issue or >> pull request, or by sending email to aldcroft at head.cfa.harvard.edu. >> >> The workshop material here was presented in the Spring of 2011 at the >> Harvard / Smithsonian Center for Astrophysics. A range of about 25 to >> 50 people participated in the different workshops, which were 1.5 >> hours in duration. ?One key accomplishment was installing a working >> Python with NumPy, SciPy, and IPython on over 50 laptops (MacOS, >> linux, and Windows) during a single session. >> >> Tom Aldcroft >> Tom Robitaille >> Brian Refsdal >> Gus Muench >> _______________________________________________ >> AstroPy mailing list >> AstroPy at scipy.org >> http://mail.scipy.org/mailman/listinfo/astropy >> > > > > -- > Erik Tollerud > > From astropy at liska.ath.cx Tue May 24 15:47:41 2011 From: astropy at liska.ath.cx (Ole Streicher) Date: Tue, 24 May 2011 21:47:41 +0200 Subject: [AstroPy] DATAMD5 calculation Message-ID: <4DDC0B5D.9050604@liska.ath.cx> Hi, is there a method in pyfits to calc or verify the MD5 checksum (keyword DATAMD5 in the primary header) for a HDUList? As far as I understand, the writeto() method just calcs some other checksums. I only found some method that calcs this for a file on the disk; however this uses the internal hdu._datLoc and hdu._datSpan methods, and I would like to apply it also to a newly created pyfits.HDUList (or to one that is changed after reading). As far as I understand, it just takes the MD5sum of all data for all HDUs from _datLoc to _datLoc+_datSpan. How would I do that "legally" with the different HDU types? Best regards Ole From embray at stsci.edu Tue May 24 18:08:04 2011 From: embray at stsci.edu (Erik Bray) Date: Tue, 24 May 2011 18:08:04 -0400 Subject: [AstroPy] DATAMD5 calculation In-Reply-To: <4DDC0B5D.9050604@liska.ath.cx> References: <4DDC0B5D.9050604@liska.ath.cx> Message-ID: <4DDC2C44.6070509@stsci.edu> On 05/24/2011 03:47 PM, Ole Streicher wrote: > Hi, > > is there a method in pyfits to calc or verify the MD5 checksum (keyword > DATAMD5 in the primary header) for a HDUList? As far as I understand, > the writeto() method just calcs some other checksums. > > I only found some method that calcs this for a file on the disk; however > this uses the internal hdu._datLoc and hdu._datSpan methods, and I would > like to apply it also to a newly created pyfits.HDUList (or to one that > is changed after reading). As far as I understand, it just takes the > MD5sum of all data for all HDUs from _datLoc to _datLoc+_datSpan. How > would I do that "legally" with the different HDU types? > > Best regards > > Ole Hello, I don't think pyfits has anything built in for handling MD5 sums. Is there some particular standard this relates to, such that it would be a good feature to add? At any rate, in the meantime you can use hashlib to generate a checksum on hdu.data--no need to use any internal attributes: import hashlib md5sum = hashlib.md5() md5sum.update(hdu.data.tostring()) hdu.header['DATAMD5'] = md5sum.hexdigest() My version of hashlib seems to know how to efficiently handle objects that implement the buffer interface (i.e. numpy arrays), so this should be pretty fast. Erik From embray at stsci.edu Tue May 24 18:10:34 2011 From: embray at stsci.edu (Erik Bray) Date: Tue, 24 May 2011 18:10:34 -0400 Subject: [AstroPy] DATAMD5 calculation In-Reply-To: <4DDC286C.5060504@liska.ath.cx> References: <4DDC0B5D.9050604@liska.ath.cx> <4DDC1F77.7060907@stsci.edu> <4DDC286C.5060504@liska.ath.cx> Message-ID: <4DDC2CDA.6070207@stsci.edu> On 05/24/2011 05:51 PM, Ole Streicher wrote: > Hi Erik, > > (I didn't get your mail through the mailing list -- on purpose?) Yes, that was a mistake; I meant to reply to the list. > Am 24.05.2011 23:13, schrieb Erik Bray: >> I don't think pyfits has anything built in for handling MD5 sums. Is >> there some particular standard this relates to, such that it would be a >> good feature to add? > > Maybe. At least, I know this from the ESO pipelines that they add a > DATAMD5 keyword to the primary header. And since this keyword is not > prefixed with "HIERARCH ESO", I guessed that it is some standard. > However, google dit not point me to more information. I'll look into it... >> At any rate, in the meantime you can use hashlib to generate a checksum >> on hdu.data--no need to use any internal attributes: > > This is unfortunately not compatible to the files I already have (from > an ESO pipeline). Strange... could it be that they're using the header plus the data for the MD5 sum? Erik From jh at physics.ucf.edu Tue May 24 20:42:37 2011 From: jh at physics.ucf.edu (Joe Harrington) Date: Tue, 24 May 2011 20:42:37 -0400 Subject: [AstroPy] Practical Python for Astronomers web tutorial In-Reply-To: (astropy-request@scipy.org) References: Message-ID: > http://python4astronomers.github.com/ First, this is a great idea and I encourage your work in it! Second, I've been teaching a class in Python (the one for which I started the SciPy Doc Project). Feel free to look at the class web site for additional ideas and demos. The demos are in the lecture directories, organized as numbered subdirectories. For solutions and large datasets, we can talk offline (I have to be careful about circulating solutions for obvious reasons). http://planets.ucf.edu/academics/ast5765/ (Follow the tiny files link in the first block of normal type.) Third, comments on the workshops. 1. Don't use: from whatever import * or any shortcuts like 'ipython -pylab' that do that. It's poor style and against the practice of the numpy community in all its docs and demos. Remember that the numpy, scipy, and matplotlib developers can in the future put anything they want in their namespaces, and a new release could break your working code because of a namespace conflict that doesn't exist when you write it. While that may not bite you for your workshops, it will eventually bite anyone who writes enough code like that. Those problems can be hard to find. In my experience, the "import *" habit is a very tough one to break, so it's particularly important never to teach it. I categorically do not use in my course any books or other resources that do "import *". Using: from whatever import foo, bar, baz is ok, though not great just because of the bookkeeping required at the top of every file. But, it's a low-impact way of rescuing old code. 2. Unless you are specifically teaching how to use the features of ipython in a particular demo, make sure all demos run in plain python, so they can be used in programs. At the very least, give exactly your startup options (e.g., "ipython -pylab") and imports so the user can run your demo. Also make sure you test them with no startup files (see #5 below for bug report). 3. Provide the data you use in your demos as a hyperlink above the demo (referring to "Reading a table and plotting" demo). In general, I've found it simpler just to generate data or embed a simple dataset, except when teaching how to read from a table. 4. The "Reading a table" demo is far too complicated for a first table demo. Remember that most of your audience will want to copy the demo and edit for their own purposes, so give them something short, like a simple ASCII file with two columns, read with a numpy function and plot. Then go for the asciitable, fancy histogram, real data thing. They will be more impressed with the simplicity of plotting in 3 lines than with the fanciness of 23 lines. 5. In this demo, random.uniform() isn't imported by pylab: $ ipython -pylab Python 2.6.5 (r265:79063, Apr 16 2010, 13:57:41) Type "copyright", "credits" or "license" for more information. IPython 0.10 -- An enhanced Interactive Python. ? -> Introduction and overview of IPython's features. %quickref -> Quick reference. help -> Python's own help system. object? -> Details about 'object'. ?object also works, ?? prints more. Welcome to pylab, a matplotlib-based Python environment. For more information, type 'help(pylab)'. In [1]: n = 100 In [2]: x = random.uniform(-10., 10., n) --------------------------------------------------------------------------- AttributeError Traceback (most recent call last) /home/jh/ in () AttributeError: 'builtin_function_or_method' object has no attribute 'uniform' In [3]: Do you really want to exit ([y]/n)? y 6. In the "Curve fitting with SciPy", don't forget the "2" in the denominator of the Gaussian exponent. In your example, sigma should be sqrt(2), not 2 (since 2 * sigma^2 = 4 in the input). Add a comment that you're passing a function in the fit and that it queries the function for how many args there are after the x data to find out how many parameters to fit. 7. In "Synthetic images", put a note in the for loop that there are more efficient, vector-based ways to do this without loops or conditionals (e.g., with np.where() or making a bigger image and then cropping it). Why make a FITS rather than a matplotlib image? 8. In "Running existing compiled codes", put in a note that there are straightforward ways to get compiled code to execute directly as Python functions, which is what most people call "glue". Why write the data to a file at all? It would be simpler to write the output to a file with shell redirection and then read the file, if you must deal with a file here. Close the file either way. Far as I got... Btw, I use that xkcd in my class as well. Note that it is the one ON THE GROUND who says the other guy is flying. So it's not a medicine-cabinet-induced hallucination! Also, the title of each workshop has the first word on the end of a breadcrumb trail and the rest on a second line. Maybe put in a line break before the title. Alsoalso, in the table of workshops, give an estimate of the time each takes. Again, this is a fantastic resource, nicely put together! --jh-- From aldcroft at head.cfa.harvard.edu Wed May 25 00:55:39 2011 From: aldcroft at head.cfa.harvard.edu (Tom Aldcroft) Date: Wed, 25 May 2011 00:55:39 -0400 Subject: [AstroPy] Practical Python for Astronomers web tutorial In-Reply-To: References: Message-ID: Hi Joe, Thanks for the detailed review and comments on the tutorial site, this is very helpful! In general I agree with your points and we'll plan to tweak the content accordingly. > 1. Don't use: > > from whatever import * > > or any shortcuts like 'ipython -pylab' that do that. ?It's poor style > and against the practice of the numpy community in all its docs and > demos. ?Remember that the numpy, scipy, and matplotlib developers can > in the future put anything they want in their namespaces, and a new > release could break your working code because of a namespace conflict > that doesn't exist when you write it. ?While that may not bite you for > your workshops, it will eventually bite anyone who writes enough code > like that. ?Those problems can be hard to find. Now I'm going risk being branded as a heretic or lazy or a bad influence, or all of the above... The decision to use "pylab" was a deliberate one based on the desire to present Python as a simple and easy tool for the research astronomers that participated in the workshops. For a first look at Python for analysis there is a real advantage to reducing the visual and conceptual noise in the form of package prefix names everywhere. For astronomers who have been using IDL or Matlab for 20 years this would make Python seem far less attractive. In our workshops the majority of people were already fluent in some analysis environment but few had much experience with Python. On the topic of rigorously avoiding the implicit "import *", the workshops tend to follow the Zen of "practicality beats purity". Most astronomers I know write only simple programs and are highly focused on getting an answer quickly so they can have results for their talk next week. In that regime I think a "pylab" approach is not only acceptable but the most efficient. Of course there is no question that for a package module or longer program that will get re-used later, one needs to worry about namespaces and following best practices. And this would be a perfect topic for a new workshop chapter "Beyond one-off scripts: writing real programs in Python" Volunteers? :-). You're right that interactive analysis is not the whole story. I should note that I never use "import *" in any scripts and do not endorse ever using that construct explicitly in a script. For interactive plotting and analysis I use "ipython -pylab" and typing plus execfile(), otherwise always explicit import statements. > Also, the title of each workshop has the first word on the end of a > breadcrumb trail and the rest on a second line. ?Maybe put in a line > break before the title. I don't understand what this means. Is the HTML rendering differently for you or ?? Thanks, Tom From astropy at liska.ath.cx Wed May 25 01:25:44 2011 From: astropy at liska.ath.cx (Ole Streicher) Date: Wed, 25 May 2011 07:25:44 +0200 Subject: [AstroPy] DATAMD5 calculation In-Reply-To: <4DDC2CDA.6070207@stsci.edu> References: <4DDC0B5D.9050604@liska.ath.cx> <4DDC1F77.7060907@stsci.edu> <4DDC286C.5060504@liska.ath.cx> <4DDC2CDA.6070207@stsci.edu> Message-ID: <4DDC92D8.3030706@liska.ath.cx> Am 25.05.2011 00:10, schrieb Erik Bray: > On 05/24/2011 05:51 PM, Ole Streicher wrote: >> Am 24.05.2011 23:13, schrieb Erik Bray: >>> I don't think pyfits has anything built in for handling MD5 sums. Is >>> there some particular standard this relates to, such that it would be a >>> good feature to add? >> >> Maybe. At least, I know this from the ESO pipelines that they add a >> DATAMD5 keyword to the primary header. And since this keyword is not >> prefixed with "HIERARCH ESO", I guessed that it is some standard. >> However, google dit not point me to more information. > > I'll look into it... It seems to come from the "qfits" package. There is a program "fitsmd5" from this library which seems to compute it. > >>> At any rate, in the meantime you can use hashlib to generate a checksum >>> on hdu.data--no need to use any internal attributes: >> >> This is unfortunately not compatible to the files I already have (from >> an ESO pipeline). > > Strange... could it be that they're using the header plus the data for > the MD5 sum? The code I found calcs for every HDU the data from hdu._datLoc to hdu._datLoc + hdu._datSpan. Since hdu._datSpan is different from hdu.data.nbytes (they differ by 832 bytes), the results must be different. Unfortunately, I dont know the meaning of _datLoc and _datSpan with respect to the data, so I dont know how to reconstruct the data without a file. Best Ole From laidler at stsci.edu Wed May 25 01:35:29 2011 From: laidler at stsci.edu (Vicki Laidler) Date: Wed, 25 May 2011 05:35:29 +0000 Subject: [AstroPy] Practical Python for Astronomers web tutorial In-Reply-To: References: , Message-ID: <173C0F068F6E49449797E90DF0B59EF8B67373@EXCHMAIL1.stsci.edu> Hi Tom, I'm going to second Joe's emphatic prohibition on "from whatever import *", for two reasons: not only as a matter of purity, but also as a practical matter. Using pylab specifically, I have more than once clobbered a few of my own variables when I did "from pylab import *" after doing a bunch of data analysis. Or, conversely, I accidentally clobber a pylab function with one of my own variables. I recommend the idiom "import pylab as P". I use this with our local scientists all the time, explaining to them that this sets P as an abbreviation for pylab. Then we use P.plot and so on. Prepending a "P." in front of the pylab commands is really very low typing overhead and visual clutter, and I haven't heard or seen any resistance to it. We import numpy in a similar way. (...and while we're on the subject of visual clutter, I may be the only person in the world who utterly loathes iPython, but I'd like to second the request to make sure your demos work in plain vanilla python as well.) I do agree that pylab is very valuable for this kind of tutorial and for astronomers' Python use in general. In real life I do "from pylab import *" for quick&dirty interactive work, but I would never teach with it and I don't do it in a session where I'm doing any non-trivial amount of data manipulation because I've been burned. cheers, Vicki Laidler ________________________________________ From: astropy-bounces at scipy.org [astropy-bounces at scipy.org] on behalf of Tom Aldcroft [aldcroft at head.cfa.harvard.edu] Sent: Wednesday, May 25, 2011 12:55 AM To: jh at physics.ucf.edu Cc: astropy at scipy.org Subject: Re: [AstroPy] Practical Python for Astronomers web tutorial Hi Joe, Thanks for the detailed review and comments on the tutorial site, this is very helpful! In general I agree with your points and we'll plan to tweak the content accordingly. > 1. Don't use: > > from whatever import * > > or any shortcuts like 'ipython -pylab' that do that. It's poor style > and against the practice of the numpy community in all its docs and > demos. Remember that the numpy, scipy, and matplotlib developers can > in the future put anything they want in their namespaces, and a new > release could break your working code because of a namespace conflict > that doesn't exist when you write it. While that may not bite you for > your workshops, it will eventually bite anyone who writes enough code > like that. Those problems can be hard to find. Now I'm going risk being branded as a heretic or lazy or a bad influence, or all of the above... The decision to use "pylab" was a deliberate one based on the desire to present Python as a simple and easy tool for the research astronomers that participated in the workshops. For a first look at Python for analysis there is a real advantage to reducing the visual and conceptual noise in the form of package prefix names everywhere. For astronomers who have been using IDL or Matlab for 20 years this would make Python seem far less attractive. In our workshops the majority of people were already fluent in some analysis environment but few had much experience with Python. On the topic of rigorously avoiding the implicit "import *", the workshops tend to follow the Zen of "practicality beats purity". Most astronomers I know write only simple programs and are highly focused on getting an answer quickly so they can have results for their talk next week. In that regime I think a "pylab" approach is not only acceptable but the most efficient. Of course there is no question that for a package module or longer program that will get re-used later, one needs to worry about namespaces and following best practices. And this would be a perfect topic for a new workshop chapter "Beyond one-off scripts: writing real programs in Python" Volunteers? :-). You're right that interactive analysis is not the whole story. I should note that I never use "import *" in any scripts and do not endorse ever using that construct explicitly in a script. For interactive plotting and analysis I use "ipython -pylab" and typing plus execfile(), otherwise always explicit import statements. > Also, the title of each workshop has the first word on the end of a > breadcrumb trail and the rest on a second line. Maybe put in a line > break before the title. I don't understand what this means. Is the HTML rendering differently for you or ?? Thanks, Tom _______________________________________________ AstroPy mailing list AstroPy at scipy.org http://mail.scipy.org/mailman/listinfo/astropy From ubutsu at gmail.com Wed May 25 02:01:07 2011 From: ubutsu at gmail.com (Taro Sato) Date: Wed, 25 May 2011 03:01:07 -0300 Subject: [AstroPy] Practical Python for Astronomers web tutorial In-Reply-To: <173C0F068F6E49449797E90DF0B59EF8B67373@EXCHMAIL1.stsci.edu> References: <173C0F068F6E49449797E90DF0B59EF8B67373@EXCHMAIL1.stsci.edu> Message-ID: Hi there. I also hope whoever will be writing tutorials and teaching materials actually try to preach good practices. Namespace is there for a reason as are other conventions and coding standards. I recently had to customize codes written by astronomers and it was an absolute nightmare dealing with all the global variables defined in different include files.... Yes the code worked but why don't you make them break the rules only after they know what they are doing? Taro On Wed, May 25, 2011 at 2:35 AM, Vicki Laidler wrote: > Hi Tom, > > I'm going to second Joe's emphatic prohibition on "from whatever import *", for two reasons: > not only as a matter of purity, but also as a practical matter. Using pylab specifically, I have > more than once clobbered a few of my own variables when I did "from pylab import *" after > doing a bunch of data analysis. Or, conversely, I accidentally clobber a pylab function with > one of my own variables. > > I recommend the idiom "import pylab as P". I use this with our local > scientists all the time, explaining to them that this sets P as an abbreviation for pylab. > Then we use P.plot and so on. Prepending a "P." in front of the pylab commands is really > very low typing overhead and visual clutter, and I haven't heard or seen any resistance to it. > We import numpy in a similar way. > > (...and while we're on the subject of visual clutter, I may be the only person in the world who > utterly loathes iPython, but I'd like to second the request to make sure your demos work > in plain vanilla python as well.) > > I do agree that pylab is very valuable for this kind of tutorial and for astronomers' Python use > in general. In real life I do "from pylab import *" for quick&dirty interactive work, but I would > never teach with it and I don't do it in a session where I'm doing any non-trivial amount of > data manipulation because I've been burned. > > cheers, > Vicki Laidler > > > ________________________________________ > From: astropy-bounces at scipy.org [astropy-bounces at scipy.org] on behalf of Tom Aldcroft [aldcroft at head.cfa.harvard.edu] > Sent: Wednesday, May 25, 2011 12:55 AM > To: jh at physics.ucf.edu > Cc: astropy at scipy.org > Subject: Re: [AstroPy] Practical Python for Astronomers web tutorial > > Hi Joe, > > Thanks for the detailed review and comments on the tutorial site, this > is very helpful! ?In general I agree with your points and we'll plan > to tweak the content accordingly. > >> 1. Don't use: >> >> from whatever import * >> >> or any shortcuts like 'ipython -pylab' that do that. ?It's poor style >> and against the practice of the numpy community in all its docs and >> demos. ?Remember that the numpy, scipy, and matplotlib developers can >> in the future put anything they want in their namespaces, and a new >> release could break your working code because of a namespace conflict >> that doesn't exist when you write it. ?While that may not bite you for >> your workshops, it will eventually bite anyone who writes enough code >> like that. ?Those problems can be hard to find. > > Now I'm going risk being branded as a heretic or lazy or a bad > influence, or all of the above... > > The decision to use "pylab" was a deliberate one based on the desire > to present Python as a simple and easy tool for the research > astronomers that participated in the workshops. ?For a first look at > Python for analysis there is a real advantage to reducing the visual > and conceptual noise in the form of package prefix names everywhere. > For astronomers who have been using IDL or Matlab for 20 years this > would make Python seem far less attractive. ?In our workshops the > majority of people were already fluent in some analysis environment > but few had much experience with Python. > > On the topic of rigorously avoiding the implicit "import *", the > workshops tend to follow the Zen of "practicality beats purity". ?Most > astronomers I know write only simple programs and are highly focused > on getting an answer quickly so they can have results for their talk > next week. ?In that regime I think a "pylab" approach is not only > acceptable but the most efficient. ?Of course there is no question > that for a package module or longer program that will get re-used > later, one needs to worry about namespaces and following best > practices. ?And this would be a perfect topic for a new workshop > chapter "Beyond one-off scripts: writing real programs in Python" > Volunteers? :-). ?You're right that interactive analysis is not the > whole story. > > I should note that I never use "import *" in any scripts and do not > endorse ever using that construct explicitly in a script. ?For > interactive plotting and analysis I use "ipython -pylab" and typing > plus execfile(), otherwise always explicit import statements. > >> Also, the title of each workshop has the first word on the end of a >> breadcrumb trail and the rest on a second line. ?Maybe put in a line >> break before the title. > > I don't understand what this means. ?Is the HTML rendering differently > for you or ?? > > Thanks, Tom > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy > -- Taro Sato Department of Astronomy & Physics Saint Mary's University? ? ? ? ? ?? email: taro at ap.smu.ca Halifax, NS B3H 3C3? ? ? ? ? ? ? ?? phone: (902)420-5027 Canada? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? web: http://ap.smu.ca/~taro From gmuench at cfa.harvard.edu Wed May 25 03:31:27 2011 From: gmuench at cfa.harvard.edu (August (Gus) Muench) Date: Wed, 25 May 2011 03:31:27 -0400 Subject: [AstroPy] Practical Python for Astronomers web tutorial Message-ID: Hi, Sadly, Joe, despite all the docs work, the assertion that numpy docs are clean of 'from numpy import *' excepts immediately. A new user following this reasonable link path has namespace garbage heaps before they even get past example #1: http://numpy.scipy.org/ => "NumPy Tutorial" http://www.scipy.org/Tentative_NumPy_Tutorial => "The Basics: An example" http://www.scipy.org/Tentative_NumPy_Tutorial#head-c5f4ceae0ab4b1313de41aba9104d0d7648e35cc :) I also think that audience matters a lot. IRAF namespaces are obfuscated away and its driven me crazy that CIAO/Sherpa are devoid of namespace but if those users are in the audience then short of preaching (which is always pedagogically inappropriate) what is one to do? [Heretic warning] Finally, who cares if someone else can't read your code? Its about audience. The audience for my reduction script is me (and maybe I should use namespaces so I can retrace what I did). If I wish to share it then I'm either going to be ignored on github, voted down on stackoverflow or run into collaborators anger or experience pedagogical problems if its crappy (and maybe even namespace free). - Gus > Date: Tue, 24 May 2011 20:42:37 -0400 > From: Joe Harrington > Subject: Re: [AstroPy] Practical Python for Astronomers web tutorial > To: astropy at scipy.org > Cc: jh at physics.ucf.edu > Message-ID: > >> http://python4astronomers.github.com/ > > First, this is a great idea and I encourage your work in it! > > Second, I've been teaching a class in Python (the one for which I > started the SciPy Doc Project). ?Feel free to look at the class web > site for additional ideas and demos. ?The demos are in the lecture > directories, organized as numbered subdirectories. ?For solutions and > large datasets, we can talk offline (I have to be careful about > circulating solutions for obvious reasons). > > http://planets.ucf.edu/academics/ast5765/ > (Follow the tiny files link in the first block of normal type.) > > Third, comments on the workshops. > > 1. Don't use: > > from whatever import * > > or any shortcuts like 'ipython -pylab' that do that. ?It's poor style > and against the practice of the numpy community in all its docs and > demos. ?Remember that the numpy, scipy, and matplotlib developers can > in the future put anything they want in their namespaces, and a new > release could break your working code because of a namespace conflict > that doesn't exist when you write it. ?While that may not bite you for > your workshops, it will eventually bite anyone who writes enough code > like that. ?Those problems can be hard to find. > > In my experience, the "import *" habit is a very tough one to break, > so it's particularly important never to teach it. ?I categorically do > not use in my course any books or other resources that do "import *". > Using: > > from whatever import foo, bar, baz > > is ok, though not great just because of the bookkeeping required at > the top of every file. ?But, it's a low-impact way of rescuing old > code. > > 2. Unless you are specifically teaching how to use the features of > ipython in a particular demo, make sure all demos run in plain python, > so they can be used in programs. ?At the very least, give exactly your > startup options (e.g., "ipython -pylab") and imports so the user can > run your demo. ?Also make sure you test them with no startup files > (see #5 below for bug report). > > 3. Provide the data you use in your demos as a hyperlink above the > demo (referring to "Reading a table and plotting" demo). ?In general, > I've found it simpler just to generate data or embed a simple dataset, > except when teaching how to read from a table. > > 4. The "Reading a table" demo is far too complicated for a first table > demo. ?Remember that most of your audience will want to copy the demo > and edit for their own purposes, so give them something short, like a > simple ASCII file with two columns, read with a numpy function and > plot. ?Then go for the asciitable, fancy histogram, real data > thing. ?They will be more impressed with the simplicity of plotting in > 3 lines than with the fanciness of 23 lines. > > 5. In this demo, random.uniform() isn't imported by pylab: > > $ ipython -pylab > Python 2.6.5 (r265:79063, Apr 16 2010, 13:57:41) > Type "copyright", "credits" or "license" for more information. > > IPython 0.10 -- An enhanced Interactive Python. > ? ? ? ? ? -> Introduction and overview of IPython's features. > %quickref -> Quick reference. > help ? ? ?-> Python's own help system. > object? ? -> Details about 'object'. ?object also works, ?? prints more. > > ?Welcome to pylab, a matplotlib-based Python environment. > ?For more information, type 'help(pylab)'. > > In [1]: n = 100 > > In [2]: x = random.uniform(-10., 10., n) > --------------------------------------------------------------------------- > AttributeError ? ? ? ? ? ? ? ? ? ? ? ? ? ?Traceback (most recent call last) > > /home/jh/ in () > > AttributeError: 'builtin_function_or_method' object has no attribute 'uniform' > > In [3]: > Do you really want to exit ([y]/n)? y > > 6. In the "Curve fitting with SciPy", don't forget the "2" in the > denominator of the Gaussian exponent. ?In your example, sigma should > be sqrt(2), not 2 (since 2 * sigma^2 = 4 in the input). ?Add a comment > that you're passing a function in the fit and that it queries the > function for how many args there are after the x data to find out how > many parameters to fit. > > 7. In "Synthetic images", put a note in the for loop that there are > more efficient, vector-based ways to do this without loops or > conditionals (e.g., with np.where() or making a bigger image and then > cropping it). ?Why make a FITS rather than a matplotlib image? > > 8. In "Running existing compiled codes", put in a note that there are > straightforward ways to get compiled code to execute directly as > Python functions, which is what most people call "glue". ?Why write > the data to a file at all? ?It would be simpler to write the output to > a file with shell redirection and then read the file, if you must deal > with a file here. ?Close the file either way. > > Far as I got... > > Btw, I use that xkcd in my class as well. ?Note that it is the one ON > THE GROUND who says the other guy is flying. ?So it's not a > medicine-cabinet-induced hallucination! > > Also, the title of each workshop has the first word on the end of a > breadcrumb trail and the rest on a second line. ?Maybe put in a line > break before the title. > > Alsoalso, in the table of workshops, give an estimate of the time each > takes. > > Again, this is a fantastic resource, nicely put together! > > --jh-- > > > > Message: 7 > Date: Wed, 25 May 2011 05:35:29 +0000 > From: Vicki Laidler > Subject: Re: [AstroPy] Practical Python for Astronomers web tutorial > To: Tom Aldcroft , "jh at physics.ucf.edu" > ? ? ? ? > Cc: "astropy at scipy.org" > Message-ID: > ? ? ? ?<173C0F068F6E49449797E90DF0B59EF8B67373 at EXCHMAIL1.stsci.edu> > Content-Type: text/plain; charset="us-ascii" > > Hi Tom, > > I'm going to second Joe's emphatic prohibition on "from whatever import *", for two reasons: > not only as a matter of purity, but also as a practical matter. Using pylab specifically, I have > more than once clobbered a few of my own variables when I did "from pylab import *" after > doing a bunch of data analysis. Or, conversely, I accidentally clobber a pylab function with > one of my own variables. > > I recommend the idiom "import pylab as P". I use this with our local > scientists all the time, explaining to them that this sets P as an abbreviation for pylab. > Then we use P.plot and so on. Prepending a "P." in front of the pylab commands is really > very low typing overhead and visual clutter, and I haven't heard or seen any resistance to it. > We import numpy in a similar way. > > (...and while we're on the subject of visual clutter, I may be the only person in the world who > utterly loathes iPython, but I'd like to second the request to make sure your demos work > in plain vanilla python as well.) > > I do agree that pylab is very valuable for this kind of tutorial and for astronomers' Python use > in general. In real life I do "from pylab import *" for quick&dirty interactive work, but I would > never teach with it and I don't do it in a session where I'm doing any non-trivial amount of > data manipulation because I've been burned. > > cheers, > Vicki Laidler > > > > Message: 8 > Date: Wed, 25 May 2011 03:01:07 -0300 > From: Taro Sato > Subject: Re: [AstroPy] Practical Python for Astronomers web tutorial > To: Vicki Laidler > Cc: "astropy at scipy.org" , ? ?"jh at physics.ucf.edu" > ? ? ? ? > Message-ID: > Content-Type: text/plain; charset=UTF-8 > > Hi there. ?I also hope whoever will be writing tutorials and teaching > materials actually try to preach good practices. ?Namespace is there > for a reason as are other conventions and coding standards. ?I > recently had to customize codes written by astronomers and it was an > absolute nightmare dealing with all the global variables defined in > different include files.... ?Yes the code worked but why don't you > make them break the rules only after they know what they are doing? > > Taro > > -- August Muench Astrophysicist & Friend of the Virtual Observatory Smithsonian Astrophysical Observatory gmuench at cfa.harvard.edu (617) 495-7979 @augustmuench From embray at stsci.edu Wed May 25 10:10:23 2011 From: embray at stsci.edu (Erik Bray) Date: Wed, 25 May 2011 10:10:23 -0400 Subject: [AstroPy] DATAMD5 calculation In-Reply-To: <4DDC2C44.6070509@stsci.edu> References: <4DDC0B5D.9050604@liska.ath.cx> <4DDC2C44.6070509@stsci.edu> Message-ID: <4DDD0DCF.3000103@stsci.edu> On 05/24/2011 06:08 PM, Erik Bray wrote: > At any rate, in the meantime you can use hashlib to generate a checksum > on hdu.data--no need to use any internal attributes: > > import hashlib > md5sum = hashlib.md5() > md5sum.update(hdu.data.tostring()) I should add, the tostring() here was a typo--I don't know why I thought to insert that. just md5sum.update(hdu.data) is sufficient. Calling tostring() is a bad idea, since it actually causes the array to be copied. > hdu.header['DATAMD5'] = md5sum.hexdigest() > > My version of hashlib seems to know how to efficiently handle objects > that implement the buffer interface (i.e. numpy arrays), so this should > be pretty fast. > > Erik > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy From embray at stsci.edu Wed May 25 10:14:44 2011 From: embray at stsci.edu (Erik Bray) Date: Wed, 25 May 2011 10:14:44 -0400 Subject: [AstroPy] DATAMD5 calculation In-Reply-To: <4DDC92D8.3030706@liska.ath.cx> References: <4DDC0B5D.9050604@liska.ath.cx> <4DDC1F77.7060907@stsci.edu> <4DDC286C.5060504@liska.ath.cx> <4DDC2CDA.6070207@stsci.edu> <4DDC92D8.3030706@liska.ath.cx> Message-ID: <4DDD0ED4.1030905@stsci.edu> On 05/25/2011 01:25 AM, Ole Streicher wrote: > Am 25.05.2011 00:10, schrieb Erik Bray: >> On 05/24/2011 05:51 PM, Ole Streicher wrote: >>> Am 24.05.2011 23:13, schrieb Erik Bray: >>>> I don't think pyfits has anything built in for handling MD5 sums. Is >>>> there some particular standard this relates to, such that it would be a >>>> good feature to add? >>> >>> Maybe. At least, I know this from the ESO pipelines that they add a >>> DATAMD5 keyword to the primary header. And since this keyword is not >>> prefixed with "HIERARCH ESO", I guessed that it is some standard. >>> However, google dit not point me to more information. >> >> I'll look into it... > > It seems to come from the "qfits" package. There is a program "fitsmd5" > from this library which seems to compute it. >> >>>> At any rate, in the meantime you can use hashlib to generate a checksum >>>> on hdu.data--no need to use any internal attributes: >>> >>> This is unfortunately not compatible to the files I already have (from >>> an ESO pipeline). >> >> Strange... could it be that they're using the header plus the data for >> the MD5 sum? > > The code I found calcs for every HDU the data from hdu._datLoc to > hdu._datLoc + hdu._datSpan. Since hdu._datSpan is different from > hdu.data.nbytes (they differ by 832 bytes), the results must be different. > > Unfortunately, I dont know the meaning of _datLoc and _datSpan with > respect to the data, so I dont know how to reconstruct the data without > a file. > > Best > > Ole I took a look at the fitsmd5 utility, and as you said it's the MD5 sum of all the data sections. Where PyFITS is concerned, _datLoc is just the offset within the file where that HDU's data section begins, and _datSpan is the length of the data section. Since FITS files are read in 2880 byte blocks, there can be some padding at the end of the data, which is why it may not be exactly the same as hdu.data.nbytes. But it looks like fitsmd5 includes that padding in the sum. The padding is just null bytes, so you can still add that yourself. For example: FITS_BLOCK_SIZE = 2880 md5sum.update(hdu.data) pad = '\0' * (FITS_BLOCK_SIZE - hdu.data.nbytes % FITS_BLOCK_SIZE) md5sum.update(pad) And do that for each HDU. I think that would do it. Erik From jeanluc.menut at free.fr Wed May 25 10:39:47 2011 From: jeanluc.menut at free.fr (Jean-Luc Menut) Date: Wed, 25 May 2011 16:39:47 +0200 Subject: [AstroPy] Practical Python for Astronomers web tutorial In-Reply-To: References: Message-ID: <4DDD14B3.3020904@free.fr> > The decision to use "pylab" was a deliberate one based on the desire > to present Python as a simple and easy tool for the research > astronomers that participated in the workshops. I agree with you. If you have to teach python as a replacement for idl/matlab/whatever, it as to be as simple as the tool it replace. So using ipython -pylab is a good start. In a second step you can introduce the concept of namespaces (and give a good example for why this is really important). J.L. From aldcroft at head.cfa.harvard.edu Wed May 25 10:52:22 2011 From: aldcroft at head.cfa.harvard.edu (Tom Aldcroft) Date: Wed, 25 May 2011 10:52:22 -0400 Subject: [AstroPy] Practical Python for Astronomers web tutorial In-Reply-To: References: <173C0F068F6E49449797E90DF0B59EF8B67373@EXCHMAIL1.stsci.edu> Message-ID: I strongly endorse teaching good practices at the undergraduate and early graduate level, and I wish that a year or more of programming courses were a universal requirement for a PhD in astronomy. Sadly this is not the case and as you note many astronomers have terrible coding practices. Even more sad is my experience (repeated over and over) that by the time people get to be postdocs and beyond they are mostly set in their ways. Those who appreciate programming as an art will naturally adopt good practices, but the majority just do whatever it takes to quickly get a result and are simply not interested in coding practices. You can preach all you want but it won't help. I have one colleague who has successfully been doing data analysis using AWK for 20 years, so just getting him to look at Python has been a challenge. - Tom On Wed, May 25, 2011 at 2:01 AM, Taro Sato wrote: > Hi there. ?I also hope whoever will be writing tutorials and teaching > materials actually try to preach good practices. ?Namespace is there > for a reason as are other conventions and coding standards. ?I > recently had to customize codes written by astronomers and it was an > absolute nightmare dealing with all the global variables defined in > different include files.... ?Yes the code worked but why don't you > make them break the rules only after they know what they are doing? > > Taro > > > On Wed, May 25, 2011 at 2:35 AM, Vicki Laidler wrote: >> Hi Tom, >> >> I'm going to second Joe's emphatic prohibition on "from whatever import *", for two reasons: >> not only as a matter of purity, but also as a practical matter. Using pylab specifically, I have >> more than once clobbered a few of my own variables when I did "from pylab import *" after >> doing a bunch of data analysis. Or, conversely, I accidentally clobber a pylab function with >> one of my own variables. >> >> I recommend the idiom "import pylab as P". I use this with our local >> scientists all the time, explaining to them that this sets P as an abbreviation for pylab. >> Then we use P.plot and so on. Prepending a "P." in front of the pylab commands is really >> very low typing overhead and visual clutter, and I haven't heard or seen any resistance to it. >> We import numpy in a similar way. >> >> (...and while we're on the subject of visual clutter, I may be the only person in the world who >> utterly loathes iPython, but I'd like to second the request to make sure your demos work >> in plain vanilla python as well.) >> >> I do agree that pylab is very valuable for this kind of tutorial and for astronomers' Python use >> in general. In real life I do "from pylab import *" for quick&dirty interactive work, but I would >> never teach with it and I don't do it in a session where I'm doing any non-trivial amount of >> data manipulation because I've been burned. >> >> cheers, >> Vicki Laidler >> >> >> ________________________________________ >> From: astropy-bounces at scipy.org [astropy-bounces at scipy.org] on behalf of Tom Aldcroft [aldcroft at head.cfa.harvard.edu] >> Sent: Wednesday, May 25, 2011 12:55 AM >> To: jh at physics.ucf.edu >> Cc: astropy at scipy.org >> Subject: Re: [AstroPy] Practical Python for Astronomers web tutorial >> >> Hi Joe, >> >> Thanks for the detailed review and comments on the tutorial site, this >> is very helpful! ?In general I agree with your points and we'll plan >> to tweak the content accordingly. >> >>> 1. Don't use: >>> >>> from whatever import * >>> >>> or any shortcuts like 'ipython -pylab' that do that. ?It's poor style >>> and against the practice of the numpy community in all its docs and >>> demos. ?Remember that the numpy, scipy, and matplotlib developers can >>> in the future put anything they want in their namespaces, and a new >>> release could break your working code because of a namespace conflict >>> that doesn't exist when you write it. ?While that may not bite you for >>> your workshops, it will eventually bite anyone who writes enough code >>> like that. ?Those problems can be hard to find. >> >> Now I'm going risk being branded as a heretic or lazy or a bad >> influence, or all of the above... >> >> The decision to use "pylab" was a deliberate one based on the desire >> to present Python as a simple and easy tool for the research >> astronomers that participated in the workshops. ?For a first look at >> Python for analysis there is a real advantage to reducing the visual >> and conceptual noise in the form of package prefix names everywhere. >> For astronomers who have been using IDL or Matlab for 20 years this >> would make Python seem far less attractive. ?In our workshops the >> majority of people were already fluent in some analysis environment >> but few had much experience with Python. >> >> On the topic of rigorously avoiding the implicit "import *", the >> workshops tend to follow the Zen of "practicality beats purity". ?Most >> astronomers I know write only simple programs and are highly focused >> on getting an answer quickly so they can have results for their talk >> next week. ?In that regime I think a "pylab" approach is not only >> acceptable but the most efficient. ?Of course there is no question >> that for a package module or longer program that will get re-used >> later, one needs to worry about namespaces and following best >> practices. ?And this would be a perfect topic for a new workshop >> chapter "Beyond one-off scripts: writing real programs in Python" >> Volunteers? :-). ?You're right that interactive analysis is not the >> whole story. >> >> I should note that I never use "import *" in any scripts and do not >> endorse ever using that construct explicitly in a script. ?For >> interactive plotting and analysis I use "ipython -pylab" and typing >> plus execfile(), otherwise always explicit import statements. >> >>> Also, the title of each workshop has the first word on the end of a >>> breadcrumb trail and the rest on a second line. ?Maybe put in a line >>> break before the title. >> >> I don't understand what this means. ?Is the HTML rendering differently >> for you or ?? >> >> Thanks, Tom >> _______________________________________________ >> AstroPy mailing list >> AstroPy at scipy.org >> http://mail.scipy.org/mailman/listinfo/astropy >> _______________________________________________ >> AstroPy mailing list >> AstroPy at scipy.org >> http://mail.scipy.org/mailman/listinfo/astropy >> > > > > -- > Taro Sato > Department of Astronomy & Physics > Saint Mary's University? ? ? ? ? ?? email: taro at ap.smu.ca > Halifax, NS B3H 3C3? ? ? ? ? ? ? ?? phone: (902)420-5027 > Canada? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? web: http://ap.smu.ca/~taro > > From aldcroft at head.cfa.harvard.edu Wed May 25 11:03:49 2011 From: aldcroft at head.cfa.harvard.edu (Tom Aldcroft) Date: Wed, 25 May 2011 11:03:49 -0400 Subject: [AstroPy] Practical Python for Astronomers web tutorial In-Reply-To: <173C0F068F6E49449797E90DF0B59EF8B67373@EXCHMAIL1.stsci.edu> References: <173C0F068F6E49449797E90DF0B59EF8B67373@EXCHMAIL1.stsci.edu> Message-ID: > I'm going to second Joe's emphatic prohibition on "from whatever import *", for two reasons: > not only as a matter of purity, but also as a practical matter. Using pylab specifically, I have > more than once clobbered a few of my own variables when I did "from pylab import *" after > doing a bunch of data analysis. Or, conversely, I accidentally clobber a pylab function with > one of my own variables. We are basically in agreement and I totally discourage use of "from whatever import *" at the Python prompt. The people who go through the workshops will not even know such a construct exists! (OK, it is mentioned exactly once in a parenthetical side note). But clobbering imported functions or even built-in functions is always possible with Python, and that is considered a feature by many. > I recommend the idiom "import pylab as P". I use this with our local > scientists all the time, explaining to them that this sets P as an abbreviation for pylab. > Then we use P.plot and so on. Prepending a "P." in front of the pylab commands is really > very low typing overhead and visual clutter, and I haven't heard or seen any resistance to it. > We import numpy in a similar way. This is a workable option but as a style issue it's not my preference. I also note that "import pylab as P" violates a number of PEP8 recommendations and I do take PEP8 pretty seriously, although practicality beats purity. :-) > (...and while we're on the subject of visual clutter, I may be the only person in the world who > utterly loathes iPython, but I'd like to second the request to make sure your demos work > in plain vanilla python as well.) That would require from pylab import *, or reworking them all to use package prefixes everywhere. I understand your position but at least for now will leave the examples as they are. **However**, based on feedback from this list we will definitely add substantial new content with a discussion of namespaces, what's going on with pylab and import *, and how to translate the interactive code snippets into standalone python programs. > I do agree that pylab is very valuable for this kind of tutorial and for astronomers' Python use > in general. In real life I do "from pylab import *" for quick&dirty interactive work, but I would > never teach with it and I don't do it in a session where I'm doing any non-trivial amount of > data manipulation because I've been burned. I'm sure some will disagree, but I would argue it is reasonable to "Preach (or teach) what you practice", and that is exactly the perspective of Practical Python for Astronomers. Pylab or derived IPython variants are ubiquitous in interactive data analysis software for astronomy, and for good reason. It makes life easy for users. Witness CIAO, CASA, PyRaf, HIPE - they feature environments with a rich set of preloaded functions, magic imports, or else the dreaded "from package import *" within examples. For interactive analysis this is what astronomers demand and it's what developers provide. > cheers, > Vicki Laidler > > > ________________________________________ > From: astropy-bounces at scipy.org [astropy-bounces at scipy.org] on behalf of Tom Aldcroft [aldcroft at head.cfa.harvard.edu] > Sent: Wednesday, May 25, 2011 12:55 AM > To: jh at physics.ucf.edu > Cc: astropy at scipy.org > Subject: Re: [AstroPy] Practical Python for Astronomers web tutorial > > Hi Joe, > > Thanks for the detailed review and comments on the tutorial site, this > is very helpful! ?In general I agree with your points and we'll plan > to tweak the content accordingly. > >> 1. Don't use: >> >> from whatever import * >> >> or any shortcuts like 'ipython -pylab' that do that. ?It's poor style >> and against the practice of the numpy community in all its docs and >> demos. ?Remember that the numpy, scipy, and matplotlib developers can >> in the future put anything they want in their namespaces, and a new >> release could break your working code because of a namespace conflict >> that doesn't exist when you write it. ?While that may not bite you for >> your workshops, it will eventually bite anyone who writes enough code >> like that. ?Those problems can be hard to find. > > Now I'm going risk being branded as a heretic or lazy or a bad > influence, or all of the above... > > The decision to use "pylab" was a deliberate one based on the desire > to present Python as a simple and easy tool for the research > astronomers that participated in the workshops. ?For a first look at > Python for analysis there is a real advantage to reducing the visual > and conceptual noise in the form of package prefix names everywhere. > For astronomers who have been using IDL or Matlab for 20 years this > would make Python seem far less attractive. ?In our workshops the > majority of people were already fluent in some analysis environment > but few had much experience with Python. > > On the topic of rigorously avoiding the implicit "import *", the > workshops tend to follow the Zen of "practicality beats purity". ?Most > astronomers I know write only simple programs and are highly focused > on getting an answer quickly so they can have results for their talk > next week. ?In that regime I think a "pylab" approach is not only > acceptable but the most efficient. ?Of course there is no question > that for a package module or longer program that will get re-used > later, one needs to worry about namespaces and following best > practices. ?And this would be a perfect topic for a new workshop > chapter "Beyond one-off scripts: writing real programs in Python" > Volunteers? :-). ?You're right that interactive analysis is not the > whole story. > > I should note that I never use "import *" in any scripts and do not > endorse ever using that construct explicitly in a script. ?For > interactive plotting and analysis I use "ipython -pylab" and typing > plus execfile(), otherwise always explicit import statements. > >> Also, the title of each workshop has the first word on the end of a >> breadcrumb trail and the rest on a second line. ?Maybe put in a line >> break before the title. > > I don't understand what this means. ?Is the HTML rendering differently > for you or ?? > > Thanks, Tom > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy > > From sransom at nrao.edu Wed May 25 11:16:15 2011 From: sransom at nrao.edu (Scott Ransom) Date: Wed, 25 May 2011 11:16:15 -0400 Subject: [AstroPy] Practical Python for Astronomers web tutorial In-Reply-To: References: <173C0F068F6E49449797E90DF0B59EF8B67373@EXCHMAIL1.stsci.edu> Message-ID: <4DDD1D3F.3070603@nrao.edu> Hi All, I'm going to chime in here... I've been programming astro stuff heavily in python for about 13 years now. And I have definitely been burned before with namespace issues. So basically all of my scripts use imports rather than "from XXX import *". However, for quick and dirty calculator-like work in ipython, using -pylab or some amount of "from XXX import *" is _incredibly_ useful. And not being able to occasionally do things that way will (and does!) keep away matlab or IDL users (I've heard that many times over the years). We are dealing with smart people. If we teach them from day one that namespace issues really matter (and show them why that is), that using "import XXX" or variants is absolutely the right way to write programs and/or scripts, and "from XXX import *" is _only_ for quick and dirty interactive calculation, then they will do that. When I teach students, my rule of thumb is: if you are using an editor, use explicit imports. We really don't want to turn people off from python by a large amount of extra typing during their interactive, command-line tutorials. They'll think (and they do) -- "How clunky... this is much simpler in IDL." It really is possible to pound all of this into people's brains if you do it from day one. Scott On 05/25/2011 11:03 AM, Tom Aldcroft wrote: >> I'm going to second Joe's emphatic prohibition on "from whatever import *", for two reasons: >> not only as a matter of purity, but also as a practical matter. Using pylab specifically, I have >> more than once clobbered a few of my own variables when I did "from pylab import *" after >> doing a bunch of data analysis. Or, conversely, I accidentally clobber a pylab function with >> one of my own variables. > > We are basically in agreement and I totally discourage use of "from > whatever import *" at the Python prompt. The people who go through the > workshops will not even know such a construct exists! (OK, it is > mentioned exactly once in a parenthetical side note). But clobbering > imported functions or even built-in functions is always possible with > Python, and that is considered a feature by many. > >> I recommend the idiom "import pylab as P". I use this with our local >> scientists all the time, explaining to them that this sets P as an abbreviation for pylab. >> Then we use P.plot and so on. Prepending a "P." in front of the pylab commands is really >> very low typing overhead and visual clutter, and I haven't heard or seen any resistance to it. >> We import numpy in a similar way. > > This is a workable option but as a style issue it's not my preference. > I also note that "import pylab as P" violates a number of PEP8 > recommendations and I do take PEP8 pretty seriously, although > practicality beats purity. :-) > >> (...and while we're on the subject of visual clutter, I may be the only person in the world who >> utterly loathes iPython, but I'd like to second the request to make sure your demos work >> in plain vanilla python as well.) > > That would require from pylab import *, or reworking them all to use > package prefixes everywhere. I understand your position but at least > for now will leave the examples as they are. > > **However**, based on feedback from this list we will definitely add > substantial new content with a discussion of namespaces, what's going > on with pylab and import *, and how to translate the interactive code > snippets into standalone python programs. -- Scott M. Ransom Address: NRAO Phone: (434) 296-0320 520 Edgemont Rd. email: sransom at nrao.edu Charlottesville, VA 22903 USA GPG Fingerprint: 06A9 9553 78BE 16DB 407B FFCA 9BFA B6FF FFD3 2989 From laidler at stsci.edu Wed May 25 11:31:25 2011 From: laidler at stsci.edu (Victoria G. Laidler) Date: Wed, 25 May 2011 11:31:25 -0400 Subject: [AstroPy] Practical Python for Astronomers web tutorial In-Reply-To: References: <173C0F068F6E49449797E90DF0B59EF8B67373@EXCHMAIL1.stsci.edu> Message-ID: <4DDD20CD.1010205@stsci.edu> Hi Tom, Tom Aldcroft wrote: >> I'm going to second Joe's emphatic prohibition on "from whatever import *", for two reasons: >> not only as a matter of purity, but also as a practical matter. Using pylab specifically, I have >> more than once clobbered a few of my own variables when I did "from pylab import *" after >> doing a bunch of data analysis. Or, conversely, I accidentally clobber a pylab function with >> one of my own variables. > We are basically in agreement and I totally discourage use of "from > whatever import *" at the Python prompt. The people who go through the > workshops will not even know such a construct exists! (OK, it is > mentioned exactly once in a parenthetical side note). But clobbering > imported functions or even built-in functions is always possible with > Python, and that is considered a feature by many. > Clobbering imported or built-in functions with local functions is definitely a feature. Accidentally clobbering a function with a _variable_ of the same name is the case I'm talking about. Oops, I can't call the std() function anymore because I read a column out of a table and assigned it to a variable named std -- that sort of thing. >> I recommend the idiom "import pylab as P". I use this with our local >> scientists all the time, explaining to them that this sets P as an abbreviation for pylab. >> Then we use P.plot and so on. Prepending a "P." in front of the pylab commands is really >> very low typing overhead and visual clutter, and I haven't heard or seen any resistance to it. >> We import numpy in a similar way. >> > > This is a workable option but as a style issue it's not my preference. > I also note that "import pylab as P" violates a number of PEP8 > recommendations and I do take PEP8 pretty seriously, although > practicality beats purity. :-) > Ah, right. I'm slowly trying to wean myself to a more compliant "import pylab as pl" or something similar, but "pl" reads as "power law" to my IRAF.synphot-saturated brain. ;) >> (...and while we're on the subject of visual clutter, I may be the only person in the world who >> utterly loathes iPython, but I'd like to second the request to make sure your demos work >> in plain vanilla python as well.) >> > > That would require from pylab import *, or reworking them all to use > package prefixes everywhere. I understand your position but at least > for now will leave the examples as they are. > Understood. > **However**, based on feedback from this list we will definitely add > substantial new content with a discussion of namespaces, what's going > on with pylab and import *, and how to translate the interactive code > snippets into standalone python programs. > Great! >> I do agree that pylab is very valuable for this kind of tutorial and for astronomers' Python use >> in general. In real life I do "from pylab import *" for quick&dirty interactive work, but I would >> never teach with it and I don't do it in a session where I'm doing any non-trivial amount of >> data manipulation because I've been burned. >> > > I'm sure some will disagree, but I would argue it is reasonable to > "Preach (or teach) what you practice", and that is exactly the > perspective of Practical Python for Astronomers. Pylab or derived > IPython variants are ubiquitous in interactive data analysis software > for astronomy, and for good reason. It makes life easy for users. > Witness CIAO, CASA, PyRaf, HIPE - they feature environments with a > rich set of preloaded functions, magic imports, or else the dreaded > "from package import *" within examples. For interactive analysis > this is what astronomers demand and it's what developers provide. That's a fair point. cheers, Vicki From astropy at liska.ath.cx Wed May 25 11:56:10 2011 From: astropy at liska.ath.cx (Ole Streicher) Date: Wed, 25 May 2011 17:56:10 +0200 Subject: [AstroPy] DATAMD5 calculation In-Reply-To: <4DDD0ED4.1030905@stsci.edu> References: <4DDC0B5D.9050604@liska.ath.cx> <4DDC1F77.7060907@stsci.edu> <4DDC286C.5060504@liska.ath.cx> <4DDC2CDA.6070207@stsci.edu> <4DDC92D8.3030706@liska.ath.cx> <4DDD0ED4.1030905@stsci.edu> Message-ID: <4DDD269A.3080703@liska.ath.cx> Am 25.05.2011 16:14, schrieb Erik Bray: > I took a look at the fitsmd5 utility, and as you said it's the MD5 sum > of all the data sections. Where PyFITS is concerned, _datLoc is just > the offset within the file where that HDU's data section begins, and > _datSpan is the length of the data section. Since FITS files are read > in 2880 byte blocks, there can be some padding at the end of the data, > which is why it may not be exactly the same as hdu.data.nbytes. > > But it looks like fitsmd5 includes that padding in the sum. The padding > is just null bytes, so you can still add that yourself. For example: > > FITS_BLOCK_SIZE = 2880 > md5sum.update(hdu.data) > pad = '\0' * (FITS_BLOCK_SIZE - hdu.data.nbytes % FITS_BLOCK_SIZE) > md5sum.update(pad) > > And do that for each HDU. I think that would do it. That works. Thank you very much for your help. Best regards Ole From cabanela at mnstate.edu Wed May 25 15:29:09 2011 From: cabanela at mnstate.edu (Dr. Juan E Cabanela Ph.D.) Date: Wed, 25 May 2011 14:29:09 -0500 Subject: [AstroPy] Practical Python for Astronomers web tutorial In-Reply-To: <4DDD20CD.1010205@stsci.edu> References: <173C0F068F6E49449797E90DF0B59EF8B67373@EXCHMAIL1.stsci.edu> <4DDD20CD.1010205@stsci.edu> Message-ID: <529889C4-96BF-45AE-8CD1-821BDAA4D0F2@mnstate.edu> As a note on Python for Astronomers website, I stumbled across it while the seminar was in session and followed it. It was an excellent introduction to python for me. There were a couple of errors on the site, but the only one I can definitely track down right now is that on the VO webmodules page: http://python4astronomers.github.com/vo/webmodules.html?highlight=null in the solution to the Exercise importing WISE data and making a color-color diagram, the call t5 = atpy.Table(raw,type='ascii', fill_values=fill_values) fails for me. In fact, there appears to be no way to get atpy.Table to fill values automatically as it triggers an error TypeError: read() got an unexpected keyword argument 'fill_values' I hope this can be fixed and if someone can tell me how to use fill_values within ATpy that would be appreciated. Cheers, Juan -- Dr. Juan Cabanela 218-477-2453 (V) 218-477-2290 (F) Minnesota State University Moorhead WWW: http://www.cabanela.com/ Dept. of Physics and Astronomy Twitter: Juan_Kinda_Guy 1104 Seventh Ave South, Hagen 307B IM: AstroJuanCab (AIM) Moorhead, MN 56563 cabanela at mnstate.edu (MSN) juancab at gmail.com (GTalk) Public PGP Key available at: http://www.cabanela.com/juan_public.asc From embray at stsci.edu Wed May 25 19:11:57 2011 From: embray at stsci.edu (Erik Bray) Date: Wed, 25 May 2011 19:11:57 -0400 Subject: [AstroPy] A quick survey of PyFITS users Message-ID: <4DDD8CBD.4080904@stsci.edu> Dear AstroPy community, Obviously, if you're not a PyFITS user you can skip this :) As some of you know I've been working on the next version of PyFITS, which will hopefully be released within the next couple months. I've called it version 3.0 for two primary reasons: It's the first version to support Python 3, and it contains a pretty large overhaul to the source code. Despite the large changes, I've tried, at least for this release, to maintain as much backwards-compatibility as possible with the last release (2.4.0). So most users hopefully shouldn't even notice much of a difference. I've already had a handful of unofficial beta-testers. They've been very helpful in helping me squash some bugs, but haven't otherwise pointed out any serious API incompatibilities. However, there are some large changes to the 'private' API (anything prefixed with an underscore), as well as to a few nominally 'public' methods, but that aren't of much use to most users. In addition, there are a few interfaces that are still supported, but that I would like to deprecate. So I just wanted to try to get a sense of what interfaces users are actually using, and if anybody is likely to experience significant difficulties with this new version. Here are my questions: 1) Are you using any 'private' variables, classes, functions, etc. from PyFITS? That is, is your code using anything imported from pyfits that begins with an underscore? 2) Do you use the classExtensions feature at all? 3) Do you have any custom subclasses of any classes in pyfits? 4) Do you use the _ValidHDU.req_cards() method or the Card.fromstring() method at all? 5) Are there any other odd PyFITS interfaces you directly access that are not documented in the PyFITS Users' Manual? I hope to deal with as many problems as I can now, so as to not get inundated with "bug" reports once users switch to this new version. Of course, not everybody who uses PyFITS is on this mailing list, and that will probably happen anyways. But any information I can gather to mitigate that will be greatly helpful. Thanks, Erik From fred.grollier at gmail.com Thu May 26 03:20:03 2011 From: fred.grollier at gmail.com (Frederic Grollier) Date: Thu, 26 May 2011 09:20:03 +0200 Subject: [AstroPy] Practical Python for Astronomers web tutorial In-Reply-To: <529889C4-96BF-45AE-8CD1-821BDAA4D0F2@mnstate.edu> References: <173C0F068F6E49449797E90DF0B59EF8B67373@EXCHMAIL1.stsci.edu> <4DDD20CD.1010205@stsci.edu> <529889C4-96BF-45AE-8CD1-821BDAA4D0F2@mnstate.edu> Message-ID: <20110526072003.GA5479@free.fr> On Wed, May 25, 2011 at 02:29:09PM -0500, Dr. Juan E Cabanela Ph.D. wrote: > > t5 = atpy.Table(raw,type='ascii', fill_values=fill_values) > > fails for me. In fact, there appears to be no way to get atpy.Table to fill > values automatically as it triggers an error > > TypeError: read() got an unexpected keyword argument 'fill_values' > > I hope this can be fixed and if someone can tell me how to use fill_values > within ATpy that would be appreciated. This code snippet works for me here, using ATpy.0.9.5.1 and asciitable.0.5.2. In the case of type='asciitable', the read() method delegates most of its work to asciitable, and so 'fill_values' should be a feature of this module. As far as I can tell, the 'fill_values' parameter was introduced in version 0.4.0 of asciitable, can you check your version info for this ? Cheers, Fred. From sergiopr at fis.ucm.es Thu May 26 06:12:57 2011 From: sergiopr at fis.ucm.es (Sergio Pascual) Date: Thu, 26 May 2011 12:12:57 +0200 Subject: [AstroPy] ATpy accepted in Fedora Message-ID: Hello, I'm happy to inform you that an ATpy package has been accepted in the Fedora Package Collection. ATpy has been built for fedora releases 15, 14 and 13 and for EPEL 6 and 5 https://admin.fedoraproject.org/updates/ATpy-0.9.5.1-1.fc13 https://admin.fedoraproject.org/updates/ATpy-0.9.5.1-1.fc14 https://admin.fedoraproject.org/updates/ATpy-0.9.5.1-1.fc15 https://admin.fedoraproject.org/updates/ATpy-0.9.5.1-1.el5 https://admin.fedoraproject.org/updates/ATpy-0.9.5.1-1.el6 For the moment, packages are in the testing repository. If you don't have it enabled, you can do # yum install ATpy --enablerepo=updates-testing If you test the package and it works as intended, please return feedback in the updates page. For Fedora 15 and EL 6, I have built also a parallel installable python3 package called python3-ATpy To install it: # yum install python3-ATpy --enablerepo=updates-testing Regards, Sergio -- Sergio Pascual ? ? ? ? ? ? ? ? ? ? ? ?http://guaix.fis.ucm.es/~spr gpg fingerprint: 5203 B42D 86A0 5649 410A F4AC A35F D465 F263 BCCC Departamento de Astrof?sica -- Universidad Complutense de Madrid (Spain) From aldcroft at head.cfa.harvard.edu Thu May 26 07:37:53 2011 From: aldcroft at head.cfa.harvard.edu (Tom Aldcroft) Date: Thu, 26 May 2011 07:37:53 -0400 Subject: [AstroPy] asciitable 0.6.0 Message-ID: Version 0.6.0 of asciitable is now available. The main code change is fixing a bug that showed up only on Windows. From this version forward asciitable will be supported and tested on linux, MacOS and Windows XP. Details: - Fixed issue reading a table from a string under Windows. - Added testing using Python 2.7 for Windows XP and MacOS 10.6. - Changed Python 3 testing on linux from 3.1 to 3.2. - Added pip and easy_install instructions to the installation documentation. - Added __version__ variable to module. - Removed download_url from setup.py since tar file is hosted on PyPI now. - Tom p.s. Juan and Frederic, regarding the fill_values issue: you need asciitable >= 0.4.0. From aldcroft at head.cfa.harvard.edu Thu May 26 08:54:33 2011 From: aldcroft at head.cfa.harvard.edu (Tom Aldcroft) Date: Thu, 26 May 2011 08:54:33 -0400 Subject: [AstroPy] A quick survey of PyFITS users In-Reply-To: <4DDD8CBD.4080904@stsci.edu> References: <4DDD8CBD.4080904@stsci.edu> Message-ID: Hi Erik, Congratulations on the new 3.0 release and the Python 3 compatibility! This is good news for PyFITS users and the Python community. I just have some documentation requests: - Could you put a link to the API docs on the main page or somewhere obvious? I never even knew they existed online until I just googled "pyfits api". - The current online API docs refer to version 2.2.2, not 2.4.0. Since this is a widely used package, it might be worth the effort to host API docs for all releases, e.g. http://stsdas.stsci.edu/stsci_python_epydoc/pyfits/2.4.0/index.html. - Could you put a version number in the PyFITS Handbook in addition to a release date? - Is there any chance of making an online HTML version of the PyFITS Handbook instead of only PDF? For me, HTML would be much easier to use and I wouldn't end up with 10 copies of the manual sitting in my downloads directory. Thanks, Tom On Wed, May 25, 2011 at 7:11 PM, Erik Bray wrote: > Dear AstroPy community, > > ? ? Obviously, if you're not a PyFITS user you can skip this :) ?As > some of you know I've been working on the next version of PyFITS, which > will hopefully be released within the next couple months. ?I've called > it version 3.0 for two primary reasons: ?It's the first version to > support Python 3, and it contains a pretty large overhaul to the source > code. > > ? ? Despite the large changes, I've tried, at least for this release, > to maintain as much backwards-compatibility as possible with the last > release (2.4.0). ?So most users hopefully shouldn't even notice much of > a difference. ?I've already had a handful of unofficial beta-testers. > They've been very helpful in helping me squash some bugs, but haven't > otherwise pointed out any serious API incompatibilities. > > ? ? However, there are some large changes to the 'private' API > (anything prefixed with an underscore), as well as to a few nominally > 'public' methods, but that aren't of much use to most users. ?In > addition, there are a few interfaces that are still supported, but that > I would like to deprecate. > > ? ? So I just wanted to try to get a sense of what interfaces users are > actually using, and if anybody is likely to experience significant > difficulties with this new version. ?Here are my questions: > > 1) Are you using any 'private' variables, classes, functions, etc. from > PyFITS? ?That is, is your code using anything imported from pyfits that > begins with an underscore? > > 2) Do you use the classExtensions feature at all? > > 3) Do you have any custom subclasses of any classes in pyfits? > > 4) Do you use the _ValidHDU.req_cards() method or the Card.fromstring() > method at all? > > 5) Are there any other odd PyFITS interfaces you directly access that > are not documented in the PyFITS Users' Manual? > > > I hope to deal with as many problems as I can now, so as to not get > inundated with "bug" reports once users switch to this new version. ?Of > course, not everybody who uses PyFITS is on this mailing list, and that > will probably happen anyways. ?But any information I can gather to > mitigate that will be greatly helpful. > > Thanks, > > Erik > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy > > From cabanela at mnstate.edu Thu May 26 09:40:29 2011 From: cabanela at mnstate.edu (Dr. Juan E Cabanela Ph.D.) Date: Thu, 26 May 2011 08:40:29 -0500 Subject: [AstroPy] Practical Python for Astronomers web tutorial In-Reply-To: <20110526072003.GA5479@free.fr> References: <173C0F068F6E49449797E90DF0B59EF8B67373@EXCHMAIL1.stsci.edu> <4DDD20CD.1010205@stsci.edu> <529889C4-96BF-45AE-8CD1-821BDAA4D0F2@mnstate.edu> <20110526072003.GA5479@free.fr> Message-ID: On May 26, 2011, at 2:20 AM, Frederic Grollier wrote: > On Wed, May 25, 2011 at 02:29:09PM -0500, Dr. Juan E Cabanela Ph.D. wrote: >> >> t5 = atpy.Table(raw,type='ascii', fill_values=fill_values) >> >> fails for me. In fact, there appears to be no way to get atpy.Table to fill >> values automatically as it triggers an error >> >> TypeError: read() got an unexpected keyword argument 'fill_values' >> >> I hope this can be fixed and if someone can tell me how to use fill_values >> within ATpy that would be appreciated. > > This code snippet works for me here, using ATpy.0.9.5.1 and > asciitable.0.5.2. In the case of type='asciitable', the read() method > delegates most of its work to asciitable, and so 'fill_values' should > be a feature of this module. As far as I can tell, the 'fill_values' > parameter was introduced in version 0.4.0 of asciitable, can you check > your version info for this ? I am running ATpy 0.9.5.1 and asciitable 0.5.2 as well, running on Python 2.7.1 -- EPD 7.0-2 (64-bit) on MacOS 10.6.7. In my script test.py I do the following: =========================================================================== import numpy import pylab import atpy filler = [('null',numpy.nan),('null',-999,'cntr_u','nb','na','ext_flg','w1nm','w1m','w2nm','w2m','w3nm','w3m','w4nm','w4m','tmass_key')] data = atpy.Table('wise_matches.tbl',fill_values=filler) =========================================================================== and that last line triggers the following error (with traceback): Traceback (most recent call last): File "test.py", line 14, in data = atpy.Table('wise_matches.tbl',fill_values=filler) File "/Users/juan/Library/Python/2.7/lib/python/site-packages/ATpy-0.9.5.1-py2.7.egg/atpy/basetable.py", line 167, in __init__ self.read(*args, **kwargs) File "/Users/juan/Library/Python/2.7/lib/python/site-packages/ATpy-0.9.5.1-py2.7.egg/atpy/basetable.py", line 213, in read atpy._readers[table_type](self, *args, **kwargs) TypeError: read() got an unexpected keyword argument 'fill_values' And so I have the same modules, but a different result. Juan -- Dr. Juan Cabanela 218-477-2453 (V) 218-477-2290 (F) Minnesota State University Moorhead WWW: http://www.cabanela.com/ Dept. of Physics and Astronomy Twitter: Juan_Kinda_Guy 1104 Seventh Ave South, Hagen 307B IM: AstroJuanCab (AIM) Moorhead, MN 56563 cabanela at mnstate.edu (MSN) juancab at gmail.com (GTalk) Public PGP Key available at: http://www.cabanela.com/juan_public.asc From aldcroft at head.cfa.harvard.edu Thu May 26 09:58:50 2011 From: aldcroft at head.cfa.harvard.edu (Tom Aldcroft) Date: Thu, 26 May 2011 09:58:50 -0400 Subject: [AstroPy] Practical Python for Astronomers web tutorial In-Reply-To: References: <173C0F068F6E49449797E90DF0B59EF8B67373@EXCHMAIL1.stsci.edu> <4DDD20CD.1010205@stsci.edu> <529889C4-96BF-45AE-8CD1-821BDAA4D0F2@mnstate.edu> <20110526072003.GA5479@free.fr> Message-ID: Hi Juan, In your example ATpy is looking at the table and inferring that it is an IPAC table. So by default it uses the internal ATpy IPAC table reader. This does not have the fill_values keyword, but it automatically deals with the missing values in an acceptable way. So either do: data = atpy.Table('wise_matches.tbl') # OR data = atpy.Table('wise_matches.tbl', type='ascii', fill_values=filler) The latter specifically tells atpy to use the asciitable IPAC reader instead of the atpy IPAC reader. This is one case that is a little confusing because there is overlapping functionality. I would recommend using the ATpy internal reader in most cases since it automatically deals with common missing value cases. - tom On Thu, May 26, 2011 at 9:40 AM, Dr. Juan E Cabanela Ph.D. wrote: > On May 26, 2011, at 2:20 AM, Frederic Grollier wrote: >> On Wed, May 25, 2011 at 02:29:09PM -0500, Dr. Juan E Cabanela Ph.D. wrote: >>> >>> t5 = atpy.Table(raw,type='ascii', fill_values=fill_values) >>> >>> fails for me. ?In fact, there appears to be no way to get atpy.Table to fill >>> values automatically as it triggers an error >>> >>> TypeError: read() got an unexpected keyword argument 'fill_values' >>> >>> I hope this can be fixed and if someone can tell me how to use fill_values >>> within ATpy that would be appreciated. >> >> This code snippet works for me here, using ATpy.0.9.5.1 and >> asciitable.0.5.2. In the case of type='asciitable', the read() method >> delegates most of its work to asciitable, and so 'fill_values' should >> be a feature of this module. As far as I can tell, the 'fill_values' >> parameter was introduced in version 0.4.0 of asciitable, can you check >> your version info for this ? > > I am running ATpy 0.9.5.1 and asciitable 0.5.2 as well, running on Python 2.7.1 -- EPD 7.0-2 (64-bit) on MacOS 10.6.7. > > In my script test.py I do the following: > =========================================================================== > import numpy > import pylab > import atpy > > filler = [('null',numpy.nan),('null',-999,'cntr_u','nb','na','ext_flg','w1nm','w1m','w2nm','w2m','w3nm','w3m','w4nm','w4m','tmass_key')] > data = atpy.Table('wise_matches.tbl',fill_values=filler) > =========================================================================== > > and that last line triggers the following error (with traceback): > > Traceback (most recent call last): > ?File "test.py", line 14, in > ? data = atpy.Table('wise_matches.tbl',fill_values=filler) > ?File "/Users/juan/Library/Python/2.7/lib/python/site-packages/ATpy-0.9.5.1-py2.7.egg/atpy/basetable.py", line 167, in __init__ > ? self.read(*args, **kwargs) > ?File "/Users/juan/Library/Python/2.7/lib/python/site-packages/ATpy-0.9.5.1-py2.7.egg/atpy/basetable.py", line 213, in read > ? atpy._readers[table_type](self, *args, **kwargs) > TypeError: read() got an unexpected keyword argument 'fill_values' > > And so I have the same modules, but a different result. > > Juan > -- > Dr. Juan Cabanela ? ? ? ? ? ? ? ? ? ?218-477-2453 (V) ?218-477-2290 (F) > Minnesota State University Moorhead ? ? WWW: http://www.cabanela.com/ > Dept. of Physics and Astronomy ? ? ?Twitter: Juan_Kinda_Guy > 1104 Seventh Ave South, Hagen 307B ? ? ? IM: AstroJuanCab (AIM) > Moorhead, MN 56563 ? ? ? ? ? ? ? ? ? ? ? ? ? cabanela at mnstate.edu (MSN) > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?juancab at gmail.com (GTalk) > ?Public PGP Key available at: http://www.cabanela.com/juan_public.asc > > > > > > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy > > From embray at stsci.edu Thu May 26 10:07:49 2011 From: embray at stsci.edu (Erik Bray) Date: Thu, 26 May 2011 10:07:49 -0400 Subject: [AstroPy] A quick survey of PyFITS users In-Reply-To: References: <4DDD8CBD.4080904@stsci.edu> Message-ID: <4DDE5EB5.1020904@stsci.edu> Thanks Tom, Regarding documentation, I've been working on sphinx versions of the users guide (the Handbook) and the the API docs. It still needs some work, but it's mostly usable :) I have the latest version of the docs uploaded at http://packages.python.org/pyfits/, which will be kept up to date. It will also be hosted a the URL you pointed out, but I don't know how to put anything on that web server yet. (Worth mentioning also is that pyfits itself can be downloaded from PyPI now: http://pypi.python.org/pypi/pyfits). In the future I can try to make these docs easier to find. And I agree, it would be good to add the release date to the docs. Erik On 05/26/2011 08:54 AM, Tom Aldcroft wrote: > Hi Erik, > > Congratulations on the new 3.0 release and the Python 3 compatibility! > This is good news for PyFITS users and the Python community. I just > have some documentation requests: > > - Could you put a link to the API docs on the main page or somewhere > obvious? I never even knew they existed online until I just googled > "pyfits api". > > - The current online API docs refer to version 2.2.2, not 2.4.0. > Since this is a widely used package, it might be worth the effort to > host API docs for all releases, e.g. > http://stsdas.stsci.edu/stsci_python_epydoc/pyfits/2.4.0/index.html. > > - Could you put a version number in the PyFITS Handbook in addition to > a release date? > > - Is there any chance of making an online HTML version of the PyFITS > Handbook instead of only PDF? For me, HTML would be much easier to > use and I wouldn't end up with 10 copies of the manual sitting in my > downloads directory. > > Thanks, > Tom > > On Wed, May 25, 2011 at 7:11 PM, Erik Bray wrote: >> Dear AstroPy community, >> >> Obviously, if you're not a PyFITS user you can skip this :) As >> some of you know I've been working on the next version of PyFITS, which >> will hopefully be released within the next couple months. I've called >> it version 3.0 for two primary reasons: It's the first version to >> support Python 3, and it contains a pretty large overhaul to the source >> code. >> >> Despite the large changes, I've tried, at least for this release, >> to maintain as much backwards-compatibility as possible with the last >> release (2.4.0). So most users hopefully shouldn't even notice much of >> a difference. I've already had a handful of unofficial beta-testers. >> They've been very helpful in helping me squash some bugs, but haven't >> otherwise pointed out any serious API incompatibilities. >> >> However, there are some large changes to the 'private' API >> (anything prefixed with an underscore), as well as to a few nominally >> 'public' methods, but that aren't of much use to most users. In >> addition, there are a few interfaces that are still supported, but that >> I would like to deprecate. >> >> So I just wanted to try to get a sense of what interfaces users are >> actually using, and if anybody is likely to experience significant >> difficulties with this new version. Here are my questions: >> >> 1) Are you using any 'private' variables, classes, functions, etc. from >> PyFITS? That is, is your code using anything imported from pyfits that >> begins with an underscore? >> >> 2) Do you use the classExtensions feature at all? >> >> 3) Do you have any custom subclasses of any classes in pyfits? >> >> 4) Do you use the _ValidHDU.req_cards() method or the Card.fromstring() >> method at all? >> >> 5) Are there any other odd PyFITS interfaces you directly access that >> are not documented in the PyFITS Users' Manual? >> >> >> I hope to deal with as many problems as I can now, so as to not get >> inundated with "bug" reports once users switch to this new version. Of >> course, not everybody who uses PyFITS is on this mailing list, and that >> will probably happen anyways. But any information I can gather to >> mitigate that will be greatly helpful. >> >> Thanks, >> >> Erik >> _______________________________________________ >> AstroPy mailing list >> AstroPy at scipy.org >> http://mail.scipy.org/mailman/listinfo/astropy >> >> From cabanela at mnstate.edu Thu May 26 10:25:28 2011 From: cabanela at mnstate.edu (Dr. Juan E Cabanela Ph.D.) Date: Thu, 26 May 2011 09:25:28 -0500 Subject: [AstroPy] Practical Python for Astronomers web tutorial In-Reply-To: References: <173C0F068F6E49449797E90DF0B59EF8B67373@EXCHMAIL1.stsci.edu> <4DDD20CD.1010205@stsci.edu> <529889C4-96BF-45AE-8CD1-821BDAA4D0F2@mnstate.edu> <20110526072003.GA5479@free.fr> Message-ID: <5E485821-026B-4DB4-AF5C-466166E7AD72@mnstate.edu> Tom, Thanks for the clarification as to the cause of the error. Makes sense. The reason I was attempting my own fill was precisely because the default filling of missing values seemed odd. Specifically, while replacing 'null' with 'nan' makes sense to me, there were several cases where: WARNING: null value for column ***** changed from null to -9223372036854775807 which seemed an odd choice to me. It appears to be the smallest signed 64-bit integer, I would have preferred more control over it. But your alternative solution works for me. Juan On May 26, 2011, at 8:58 AM, Tom Aldcroft wrote: > Hi Juan, > > In your example ATpy is looking at the table and inferring that it is > an IPAC table. So by default it uses the internal ATpy IPAC table > reader. This does not have the fill_values keyword, but it > automatically deals with the missing values in an acceptable way. So > either do: > > data = atpy.Table('wise_matches.tbl') > # OR > data = atpy.Table('wise_matches.tbl', type='ascii', fill_values=filler) > > The latter specifically tells atpy to use the asciitable IPAC reader > instead of the atpy IPAC reader. This is one case that is a little > confusing because there is overlapping functionality. I would > recommend using the ATpy internal reader in most cases since it > automatically deals with common missing value cases. > > - tom > > On Thu, May 26, 2011 at 9:40 AM, Dr. Juan E Cabanela Ph.D. > wrote: >> On May 26, 2011, at 2:20 AM, Frederic Grollier wrote: >>> On Wed, May 25, 2011 at 02:29:09PM -0500, Dr. Juan E Cabanela Ph.D. wrote: >>>> >>>> t5 = atpy.Table(raw,type='ascii', fill_values=fill_values) >>>> >>>> fails for me. In fact, there appears to be no way to get atpy.Table to fill >>>> values automatically as it triggers an error >>>> >>>> TypeError: read() got an unexpected keyword argument 'fill_values' >>>> >>>> I hope this can be fixed and if someone can tell me how to use fill_values >>>> within ATpy that would be appreciated. >>> >>> This code snippet works for me here, using ATpy.0.9.5.1 and >>> asciitable.0.5.2. In the case of type='asciitable', the read() method >>> delegates most of its work to asciitable, and so 'fill_values' should >>> be a feature of this module. As far as I can tell, the 'fill_values' >>> parameter was introduced in version 0.4.0 of asciitable, can you check >>> your version info for this ? >> >> I am running ATpy 0.9.5.1 and asciitable 0.5.2 as well, running on Python 2.7.1 -- EPD 7.0-2 (64-bit) on MacOS 10.6.7. >> >> In my script test.py I do the following: >> =========================================================================== >> import numpy >> import pylab >> import atpy >> >> filler = [('null',numpy.nan),('null',-999,'cntr_u','nb','na','ext_flg','w1nm','w1m','w2nm','w2m','w3nm','w3m','w4nm','w4m','tmass_key')] >> data = atpy.Table('wise_matches.tbl',fill_values=filler) >> =========================================================================== >> >> and that last line triggers the following error (with traceback): >> >> Traceback (most recent call last): >> File "test.py", line 14, in >> data = atpy.Table('wise_matches.tbl',fill_values=filler) >> File "/Users/juan/Library/Python/2.7/lib/python/site-packages/ATpy-0.9.5.1-py2.7.egg/atpy/basetable.py", line 167, in __init__ >> self.read(*args, **kwargs) >> File "/Users/juan/Library/Python/2.7/lib/python/site-packages/ATpy-0.9.5.1-py2.7.egg/atpy/basetable.py", line 213, in read >> atpy._readers[table_type](self, *args, **kwargs) >> TypeError: read() got an unexpected keyword argument 'fill_values' >> >> And so I have the same modules, but a different result. -- Dr. Juan Cabanela 218-477-2453 (V) 218-477-2290 (F) Minnesota State University Moorhead WWW: http://www.cabanela.com/ Dept. of Physics and Astronomy Twitter: Juan_Kinda_Guy 1104 Seventh Ave South, Hagen 307B IM: AstroJuanCab (AIM) Moorhead, MN 56563 cabanela at mnstate.edu (MSN) juancab at gmail.com (GTalk) Public PGP Key available at: http://www.cabanela.com/juan_public.asc From kellecruz at gmail.com Fri May 27 11:50:05 2011 From: kellecruz at gmail.com (Kelle Cruz) Date: Fri, 27 May 2011 11:50:05 -0400 Subject: [AstroPy] Practical Python for Astronomers web tutorial In-Reply-To: References: Message-ID: Such a great resource, thanks guys!! We've decided to do the workshops during the first two weeks of our CUNY/AMNH REU program and so far, I anticipate one problem: Scisoft. There's a bit in the Installation workshop about handling packages with multiple python installations ( http://python4astronomers.github.com/installation/packages.html#multiple-pythons-on-your-computer), but I would like some specific advice on how to handle existing Scisoft installations. Specifically, if we want to keep Scisoft installed for the IRAF installation (which we need for pyraf, right?), but NOT for the Python install, what should we do? I figure there's some combo of environment variables that need setting/unsetting in the .tcshrc and if someone can point us in the right direction, that would help get us started as smoothly as possible. Are there any other conflicts that I should anticipate with installing the EPD on computers that have existing Python installations? Thanks for your advice! kelle On Mon, May 23, 2011 at 10:18 AM, Tom Aldcroft < aldcroft at head.cfa.harvard.edu> wrote: > My colleagues and I would like to announce the availability of a web > tutorial aimed at teaching Python to astronomers through a series of > interactive workshops: > > http://python4astronomers.github.com/ > > Practical Python for Astronomers is a series of hands-on workshops to > explore the Python language and the analysis tools it provides. The > emphasis is on using Python to solve real-world problems that > astronomers are likely to encounter in research. Some features: > > - Workshops immediately use plotting, analysis, and file reading tools. > - Along the way elements of the Python language are introduced. > - Workshops are interactive using examples run by participants on their > laptops. > - Comprehensive instructions a given for installing a full Python > environment. > > There are two goals. First is to provide tutorials suitable for > self-study by those wishing to learn Python for astronomy. The > greater goal is for those knowledgable in Python to teach the workshop > series at their local institutions, adapting the content as desired. > To that end we have developed the content in Sphinx RestructuredText > and hosted the source on github at > https://github.com/python4astronomers/. Anyone interested can clone > the repository or download a tarball and make modifications needed to > present the material locally. We would also welcome comments, fixes, > or suggestions for improvement. This can be done as a Github issue or > pull request, or by sending email to aldcroft at head.cfa.harvard.edu. > > The workshop material here was presented in the Spring of 2011 at the > Harvard / Smithsonian Center for Astrophysics. A range of about 25 to > 50 people participated in the different workshops, which were 1.5 > hours in duration. One key accomplishment was installing a working > Python with NumPy, SciPy, and IPython on over 50 laptops (MacOS, > linux, and Windows) during a single session. > > Tom Aldcroft > Tom Robitaille > Brian Refsdal > Gus Muench > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy > -- Kelle Cruz, PhD ? http://kellecruz.com/ 917.725.1334 ? Hunter ext: 16486 ? AMNH ext: 3404 -------------- next part -------------- An HTML attachment was scrubbed... URL: From mservillat at cfa.harvard.edu Fri May 27 12:19:50 2011 From: mservillat at cfa.harvard.edu (Mathieu Servillat) Date: Fri, 27 May 2011 12:19:50 -0400 Subject: [AstroPy] Practical Python for Astronomers web tutorial In-Reply-To: References: Message-ID: <4DDFCF26.1050006@cfa.harvard.edu> Hi Kelle, not sure I'm the best person to answer, but I indeed had to face conflicts between Scisoft and EPD (on Mac). I want to use pyraf, but also the latest python distribution and packages. Basically, if you install EPD after scisoft, the different PATH are set for EPD and pyraf won't start. I have both Scisoft (2010.10.2b) and EPD (7.0-2) installed, and my dirty solution to be able to use both is in the .bash_profile: #import my python scripts export PYTHONPATH=$PYTHONPATH:/.../scripts/python/... alias setpyraf="export PYTHONPATH=${PYTHONPATH}:/usr/local/scisoft/packages/python/lib/python2.6/site-packages/" #setup scisoft (as recommended) . /usr/local/scisoft/bin/Setup.bash #Setting PATH for EPD-7.0-2 (automatically added) PATH="/Library/Frameworks/Python.framework/Versions/Current/bin:${PATH}" export PATH So when I want to run pyraf I simply have to type setpyraf before and it works! (however I'm not sure I understand everything, and maybe someone can develop this with a cleaner solution). I believe the conversion of this to csh or tcsh will work in the same way. Cheers, Mat On 27/05/11 11:50 AM, Kelle Cruz wrote: > Such a great resource, thanks guys!! > > We've decided to do the workshops during the first two weeks of our > CUNY/AMNH REU program and so far, I anticipate one problem: Scisoft. > There's a bit in the Installation workshop about handling packages > with multiple python installations > (http://python4astronomers.github.com/installation/packages.html#multiple-pythons-on-your-computer), > but I would like some specific advice on how to handle existing > Scisoft installations. > > Specifically, if we want to keep Scisoft installed for the IRAF > installation (which we need for pyraf, right?), but NOT for the Python > install, what should we do? I figure there's some combo of environment > variables that need setting/unsetting in the .tcshrc and if someone > can point us in the right direction, that would help get us started as > smoothly as possible. > > Are there any other conflicts that I should anticipate with installing > the EPD on computers that have existing Python installations? > > Thanks for your advice! > kelle > > On Mon, May 23, 2011 at 10:18 AM, Tom Aldcroft > > > wrote: > > My colleagues and I would like to announce the availability of a web > tutorial aimed at teaching Python to astronomers through a series of > interactive workshops: > > http://python4astronomers.github.com/ > > Practical Python for Astronomers is a series of hands-on workshops to > explore the Python language and the analysis tools it provides. The > emphasis is on using Python to solve real-world problems that > astronomers are likely to encounter in research. Some features: > > - Workshops immediately use plotting, analysis, and file reading > tools. > - Along the way elements of the Python language are introduced. > - Workshops are interactive using examples run by participants on > their laptops. > - Comprehensive instructions a given for installing a full Python > environment. > > There are two goals. First is to provide tutorials suitable for > self-study by those wishing to learn Python for astronomy. The > greater goal is for those knowledgable in Python to teach the workshop > series at their local institutions, adapting the content as desired. > To that end we have developed the content in Sphinx RestructuredText > and hosted the source on github at > https://github.com/python4astronomers/. Anyone interested can clone > the repository or download a tarball and make modifications needed to > present the material locally. We would also welcome comments, fixes, > or suggestions for improvement. This can be done as a Github issue or > pull request, or by sending email to aldcroft at head.cfa.harvard.edu > . > > The workshop material here was presented in the Spring of 2011 at the > Harvard / Smithsonian Center for Astrophysics. A range of about 25 to > 50 people participated in the different workshops, which were 1.5 > hours in duration. One key accomplishment was installing a working > Python with NumPy, SciPy, and IPython on over 50 laptops (MacOS, > linux, and Windows) during a single session. > > Tom Aldcroft > Tom Robitaille > Brian Refsdal > Gus Muench > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy > > > > > -- > Kelle Cruz, PhD ? http://kellecruz.com/ > 917.725.1334 ? Hunter ext: 16486 ? AMNH ext: 3404 > > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy -- Dr. Mathieu Servillat Harvard-Smithsonian Center for Astrophysics 60 Garden Street, MS-67 Cambridge, MA 02138, USA +1 617 495 7275 mservillat at cfa.harvard.edu -- -------------- next part -------------- An HTML attachment was scrubbed... URL: From mservillat at cfa.harvard.edu Fri May 27 12:23:55 2011 From: mservillat at cfa.harvard.edu (Mathieu Servillat) Date: Fri, 27 May 2011 12:23:55 -0400 Subject: [AstroPy] Practical Python for Astronomers web tutorial In-Reply-To: <4DDFCF26.1050006@cfa.harvard.edu> References: <4DDFCF26.1050006@cfa.harvard.edu> Message-ID: <4DDFD01B.6090701@cfa.harvard.edu> I'm afraid the solution is even dirtier: I commented in /usr/local/scisoft/bin/Setup.bash the lines that set the PYTHONPATH variable to make this work. ++ Mat On 27/05/11 12:19 PM, Mathieu Servillat wrote: > Hi Kelle, > > not sure I'm the best person to answer, but I indeed had to face > conflicts between Scisoft and EPD (on Mac). I want to use pyraf, but > also the latest python distribution and packages. Basically, if you > install EPD after scisoft, the different PATH are set for EPD and > pyraf won't start. > > I have both Scisoft (2010.10.2b) and EPD (7.0-2) installed, and my > dirty solution to be able to use both is in the .bash_profile: > > #import my python scripts > export PYTHONPATH=$PYTHONPATH:/.../scripts/python/... > > alias setpyraf="export > PYTHONPATH=${PYTHONPATH}:/usr/local/scisoft/packages/python/lib/python2.6/site-packages/" > > #setup scisoft (as recommended) > . /usr/local/scisoft/bin/Setup.bash > > #Setting PATH for EPD-7.0-2 (automatically added) > PATH="/Library/Frameworks/Python.framework/Versions/Current/bin:${PATH}" > export PATH > > > So when I want to run pyraf I simply have to type setpyraf before and > it works! (however I'm not sure I understand everything, and maybe > someone can develop this with a cleaner solution). I believe the > conversion of this to csh or tcsh will work in the same way. > > Cheers, > Mat > > > On 27/05/11 11:50 AM, Kelle Cruz wrote: >> Such a great resource, thanks guys!! >> >> We've decided to do the workshops during the first two weeks of our >> CUNY/AMNH REU program and so far, I anticipate one problem: Scisoft. >> There's a bit in the Installation workshop about handling packages >> with multiple python installations >> (http://python4astronomers.github.com/installation/packages.html#multiple-pythons-on-your-computer), >> but I would like some specific advice on how to handle existing >> Scisoft installations. >> >> Specifically, if we want to keep Scisoft installed for the IRAF >> installation (which we need for pyraf, right?), but NOT for the >> Python install, what should we do? I figure there's some combo of >> environment variables that need setting/unsetting in the .tcshrc and >> if someone can point us in the right direction, that would help get >> us started as smoothly as possible. >> >> Are there any other conflicts that I should anticipate with >> installing the EPD on computers that have existing Python installations? >> >> Thanks for your advice! >> kelle >> >> On Mon, May 23, 2011 at 10:18 AM, Tom Aldcroft >> > > wrote: >> >> My colleagues and I would like to announce the availability of a web >> tutorial aimed at teaching Python to astronomers through a series of >> interactive workshops: >> >> http://python4astronomers.github.com/ >> >> Practical Python for Astronomers is a series of hands-on workshops to >> explore the Python language and the analysis tools it provides. The >> emphasis is on using Python to solve real-world problems that >> astronomers are likely to encounter in research. Some features: >> >> - Workshops immediately use plotting, analysis, and file reading >> tools. >> - Along the way elements of the Python language are introduced. >> - Workshops are interactive using examples run by participants on >> their laptops. >> - Comprehensive instructions a given for installing a full Python >> environment. >> >> There are two goals. First is to provide tutorials suitable for >> self-study by those wishing to learn Python for astronomy. The >> greater goal is for those knowledgable in Python to teach the >> workshop >> series at their local institutions, adapting the content as desired. >> To that end we have developed the content in Sphinx RestructuredText >> and hosted the source on github at >> https://github.com/python4astronomers/. Anyone interested can clone >> the repository or download a tarball and make modifications needed to >> present the material locally. We would also welcome comments, fixes, >> or suggestions for improvement. This can be done as a Github issue or >> pull request, or by sending email to >> aldcroft at head.cfa.harvard.edu . >> >> The workshop material here was presented in the Spring of 2011 at the >> Harvard / Smithsonian Center for Astrophysics. A range of about 25 to >> 50 people participated in the different workshops, which were 1.5 >> hours in duration. One key accomplishment was installing a working >> Python with NumPy, SciPy, and IPython on over 50 laptops (MacOS, >> linux, and Windows) during a single session. >> >> Tom Aldcroft >> Tom Robitaille >> Brian Refsdal >> Gus Muench >> _______________________________________________ >> AstroPy mailing list >> AstroPy at scipy.org >> http://mail.scipy.org/mailman/listinfo/astropy >> >> >> >> >> -- >> Kelle Cruz, PhD ? http://kellecruz.com/ >> 917.725.1334 ? Hunter ext: 16486 ? AMNH ext: 3404 >> >> >> _______________________________________________ >> AstroPy mailing list >> AstroPy at scipy.org >> http://mail.scipy.org/mailman/listinfo/astropy > > > -- > Dr. Mathieu Servillat > Harvard-Smithsonian Center for Astrophysics > 60 Garden Street, MS-67 > Cambridge, MA 02138, USA > +1 617 495 7275 > mservillat at cfa.harvard.edu > -- > > > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy -- Dr. Mathieu Servillat Harvard-Smithsonian Center for Astrophysics 60 Garden Street, MS-67 Cambridge, MA 02138, USA +1 617 495 7275 mservillat at cfa.harvard.edu -- -------------- next part -------------- An HTML attachment was scrubbed... URL: From matthewturk at gmail.com Fri May 27 12:24:37 2011 From: matthewturk at gmail.com (Matthew Turk) Date: Fri, 27 May 2011 12:24:37 -0400 Subject: [AstroPy] Practical Python for Astronomers web tutorial In-Reply-To: <4DDFCF26.1050006@cfa.harvard.edu> References: <4DDFCF26.1050006@cfa.harvard.edu> Message-ID: Hi Kelle, One issue I've run into is that EPD supplies its own versions of libraries such as HDF5, but not the development headers. This typically results in both a difficulty of linking any external packages that directly use those libraries against them, as well as the broader issue of building them in the first place. To be perfectly honest, I've never seen the utility of a framework build of Python in the first place unless you intend to a huge amount of native widget interaction; I spent quite a lot of effort trying to maintain a framework build but ultimately found it did not provide terribly much benefit. The best cross-platform GUI nowadays is just an embedded web server (such as rocket, with bottle as a framework) and a javascript library. -Matt On Fri, May 27, 2011 at 12:19 PM, Mathieu Servillat wrote: > Hi Kelle, > > not sure I'm the best person to answer, but I indeed had to face conflicts > between Scisoft and EPD (on Mac). I want to use pyraf, but also the latest > python distribution and packages. Basically, if you install EPD after > scisoft, the different PATH are set for EPD and pyraf won't start. > > I have both Scisoft (2010.10.2b) and EPD (7.0-2) installed, and my dirty > solution to be able to use both is in the .bash_profile: > > ? #import my python scripts > ? export PYTHONPATH=$PYTHONPATH:/.../scripts/python/... > > ? alias setpyraf="export > PYTHONPATH=${PYTHONPATH}:/usr/local/scisoft/packages/python/lib/python2.6/site-packages/" > > ? #setup scisoft (as recommended) > ? . /usr/local/scisoft/bin/Setup.bash > > ? #Setting PATH for EPD-7.0-2 (automatically added) > ? PATH="/Library/Frameworks/Python.framework/Versions/Current/bin:${PATH}" > ? export PATH > > > So when I want to run pyraf I simply have to type setpyraf before and it > works! (however I'm not sure I understand everything, and maybe someone can > develop this with a cleaner solution). I believe the conversion of this to > csh or tcsh will work in the same way. > > Cheers, > Mat > > > On 27/05/11 11:50 AM, Kelle Cruz wrote: > > Such a great resource, thanks guys!! > We've decided to do the workshops during the first two weeks of our > CUNY/AMNH REU program and so far, I anticipate one problem: Scisoft. There's > a bit in the Installation workshop about handling packages with multiple > python installations > (http://python4astronomers.github.com/installation/packages.html#multiple-pythons-on-your-computer), > but I would like some specific advice on how to handle existing Scisoft > installations. > Specifically, if we want to keep Scisoft installed for the IRAF installation > (which we need for pyraf, right?), but NOT for the Python install, what > should we do? I figure there's some combo of environment variables that need > setting/unsetting in the .tcshrc and if someone can point us in the right > direction, that would help get us started as smoothly as possible. > Are there any other conflicts that I should anticipate with installing the > EPD on computers that have existing Python installations? > Thanks for your advice! > kelle > > On Mon, May 23, 2011 at 10:18 AM, Tom Aldcroft > wrote: >> >> My colleagues and I would like to announce the availability of a web >> tutorial aimed at teaching Python to astronomers through a series of >> interactive workshops: >> >> ?http://python4astronomers.github.com/ >> >> Practical Python for Astronomers is a series of hands-on workshops to >> explore the Python language and the analysis tools it provides. The >> emphasis is on using Python to solve real-world problems that >> astronomers are likely to encounter in research. ?Some features: >> >> - Workshops immediately use plotting, analysis, and file reading tools. >> - Along the way elements of the Python language are introduced. >> - Workshops are interactive using examples run by participants on their >> laptops. >> - Comprehensive instructions a given for installing a full Python >> environment. >> >> There are two goals. ?First is to provide tutorials suitable for >> self-study by those wishing to learn Python for astronomy. ?The >> greater goal is for those knowledgable in Python to teach the workshop >> series at their local institutions, adapting the content as desired. >> To that end we have developed the content in Sphinx RestructuredText >> and hosted the source on github at >> https://github.com/python4astronomers/. Anyone interested can clone >> the repository or download a tarball and make modifications needed to >> present the material locally. ?We would also welcome comments, fixes, >> or suggestions for improvement. This can be done as a Github issue or >> pull request, or by sending email to aldcroft at head.cfa.harvard.edu. >> >> The workshop material here was presented in the Spring of 2011 at the >> Harvard / Smithsonian Center for Astrophysics. A range of about 25 to >> 50 people participated in the different workshops, which were 1.5 >> hours in duration. ?One key accomplishment was installing a working >> Python with NumPy, SciPy, and IPython on over 50 laptops (MacOS, >> linux, and Windows) during a single session. >> >> Tom Aldcroft >> Tom Robitaille >> Brian Refsdal >> Gus Muench >> _______________________________________________ >> AstroPy mailing list >> AstroPy at scipy.org >> http://mail.scipy.org/mailman/listinfo/astropy > > > > -- > Kelle Cruz, PhD ??http://kellecruz.com/ > 917.725.1334 ??Hunter ext: 16486 ? AMNH ext: 3404 > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy > > > -- > Dr. Mathieu Servillat > Harvard-Smithsonian Center for Astrophysics > 60 Garden Street, MS-67 > Cambridge, MA 02138, USA > +1 617 495 7275 > mservillat at cfa.harvard.edu > -- > > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy > > From sontag at stsci.edu Fri May 27 12:26:35 2011 From: sontag at stsci.edu (Chris Sontag) Date: Fri, 27 May 2011 12:26:35 -0400 Subject: [AstroPy] Practical Python for Astronomers web tutorial In-Reply-To: References: Message-ID: <4DDFD0BB.6060006@stsci.edu> Hi all, I'm jumping in just for the PyRAF/IRAF question. If you are running PyRAF in order to run IRAF tasks (which is very likely), then you will want IRAF installed. IRAF can be installed separately from Scisoft, however, if that is your wish. It can be downloaded from iraf.net, or binary installers for OSX are also available from: http://www.stsci.edu/resources/software_hardware/pyraf/stsci_python/current/download Lastly, for PyRAF, there is a stand-alone installation download (IRAF not included): http://www.stsci.edu/resources/software_hardware/pyraf/current/download, which includes the latest version of PyRAF not available within Scisoft. If you are having difficulty starting/running PyRAF, please send any questions (with any output or errors found) to help at stsci.edu. Chris On 5/27/11 11:50 AM, Kelle Cruz wrote: > Specifically, if we want to keep Scisoft installed for the IRAF installation (which we need for pyraf, right?), but NOT for the Python install, what should we do? -------------- next part -------------- An HTML attachment was scrubbed... URL: From jturner at gemini.edu Fri May 27 12:53:12 2011 From: jturner at gemini.edu (James Turner) Date: Fri, 27 May 2011 12:53:12 -0400 Subject: [AstroPy] Practical Python for Astronomers web tutorial In-Reply-To: References: Message-ID: <4DDFD6F8.1080909@gemini.edu> I think you could get started by allowing EPD to set up its environment as normal and then setting "iraf" and "IMTOOLRC" (and possibly "host") to point to scisoft (compare with what they're currently set to). You might need to set "tmp" too (don't ask why). Of course you'll also need to add the "bin" directory with all the symbolic links to "cl" etc. in your "PATH". Disclaimer: I have no real experience with SciSoft and hear it's IRAF setup is quite quirky, but this kind of thing works for my own source installations. Following Chris's suggestions might be safest. When you install PyRAF, I think you'll need the EPD environment set up first. Cheers, James. On 27/05/11 11:50, Kelle Cruz wrote: > Such a great resource, thanks guys!! > > We've decided to do the workshops during the first two weeks of our CUNY/AMNH > REU program and so far, I anticipate one problem: Scisoft. There's a bit in the > Installation workshop about handling packages with multiple python installations > (http://python4astronomers.github.com/installation/packages.html#multiple-pythons-on-your-computer), > but I would like some specific advice on how to handle existing Scisoft > installations. > > Specifically, if we want to keep Scisoft installed for the IRAF installation > (which we need for pyraf, right?), but NOT for the Python install, what should > we do? I figure there's some combo of environment variables that need > setting/unsetting in the .tcshrc and if someone can point us in the right > direction, that would help get us started as smoothly as possible. > > Are there any other conflicts that I should anticipate with installing the EPD > on computers that have existing Python installations? > > Thanks for your advice! > kelle > > On Mon, May 23, 2011 at 10:18 AM, Tom Aldcroft > wrote: > > My colleagues and I would like to announce the availability of a web > tutorial aimed at teaching Python to astronomers through a series of > interactive workshops: > > http://python4astronomers.github.com/ > > Practical Python for Astronomers is a series of hands-on workshops to > explore the Python language and the analysis tools it provides. The > emphasis is on using Python to solve real-world problems that > astronomers are likely to encounter in research. Some features: > > - Workshops immediately use plotting, analysis, and file reading tools. > - Along the way elements of the Python language are introduced. > - Workshops are interactive using examples run by participants on their laptops. > - Comprehensive instructions a given for installing a full Python environment. > > There are two goals. First is to provide tutorials suitable for > self-study by those wishing to learn Python for astronomy. The > greater goal is for those knowledgable in Python to teach the workshop > series at their local institutions, adapting the content as desired. > To that end we have developed the content in Sphinx RestructuredText > and hosted the source on github at > https://github.com/python4astronomers/. Anyone interested can clone > the repository or download a tarball and make modifications needed to > present the material locally. We would also welcome comments, fixes, > or suggestions for improvement. This can be done as a Github issue or > pull request, or by sending email to aldcroft at head.cfa.harvard.edu > . > > The workshop material here was presented in the Spring of 2011 at the > Harvard / Smithsonian Center for Astrophysics. A range of about 25 to > 50 people participated in the different workshops, which were 1.5 > hours in duration. One key accomplishment was installing a working > Python with NumPy, SciPy, and IPython on over 50 laptops (MacOS, > linux, and Windows) during a single session. > > Tom Aldcroft > Tom Robitaille > Brian Refsdal > Gus Muench > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy > > > > > -- > Kelle Cruz, PhD ? http://kellecruz.com/ > 917.725.1334 ? Hunter ext: 16486 ? AMNH ext: 3404 > > > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy From aldcroft at head.cfa.harvard.edu Fri May 27 13:26:31 2011 From: aldcroft at head.cfa.harvard.edu (Tom Aldcroft) Date: Fri, 27 May 2011 13:26:31 -0400 Subject: [AstroPy] Practical Python for Astronomers web tutorial In-Reply-To: <4DDFD6F8.1080909@gemini.edu> References: <4DDFD6F8.1080909@gemini.edu> Message-ID: As people have noted, getting SciSoft and EPD to play together and allowing easy switching between environments could be tricky. For Kelle's original need to set up a bunch of summer REUs to go through the tutorial and then use PyRAF, I would recommend uninstalling (or ignoring) the SciSoft installation and installing IRAF / PyRAF from the base of EPD. This requires possibly more work up front, but if you have 10 new students and start playing around with paths and links on each laptop I think there will be trouble. Also, do they *all* have Macs? If not there need to instructions for linux as well. If anybody can provide a specific IRAF / PyRAF installation guide assuming a clean EPD7.0 install on Mac and/or linux, that would tremendous! This would be immediately included in the Practical Python Workshops. I'm a bit on the busy side this week and have travel impending so I don't think I can do this right now... Thanks, Tom On Fri, May 27, 2011 at 12:53 PM, James Turner wrote: > I think you could get started by allowing EPD to set up its environment > as normal and then setting "iraf" and "IMTOOLRC" (and possibly "host") > to point to scisoft (compare with what they're currently set to). You > might need to set "tmp" too (don't ask why). Of course you'll also need > to add the "bin" directory with all the symbolic links to "cl" etc. in > your "PATH". Disclaimer: I have no real experience with SciSoft and hear > it's IRAF setup is quite quirky, but this kind of thing works for my own > source installations. Following Chris's suggestions might be safest. > > When you install PyRAF, I think you'll need the EPD environment set up > first. > > Cheers, > > James. > > > On 27/05/11 11:50, Kelle Cruz wrote: >> Such a great resource, thanks guys!! >> >> We've decided to do the workshops during the first two weeks of our CUNY/AMNH >> REU program and so far, I anticipate one problem: Scisoft. There's a bit in the >> Installation workshop about handling packages with multiple python installations >> (http://python4astronomers.github.com/installation/packages.html#multiple-pythons-on-your-computer), >> but I would like some specific advice on how to handle existing Scisoft >> installations. >> >> Specifically, if we want to keep Scisoft installed for the IRAF installation >> (which we need for pyraf, right?), but NOT for the Python install, what should >> we do? I figure there's some combo of environment variables that need >> setting/unsetting in the .tcshrc and if someone can point us in the right >> direction, that would help get us started as smoothly as possible. >> >> Are there any other conflicts that I should anticipate with installing the EPD >> on computers that have existing Python installations? >> >> Thanks for your advice! >> kelle >> >> On Mon, May 23, 2011 at 10:18 AM, Tom Aldcroft > > wrote: >> >> ? ? My colleagues and I would like to announce the availability of a web >> ? ? tutorial aimed at teaching Python to astronomers through a series of >> ? ? interactive workshops: >> >> ? ? http://python4astronomers.github.com/ >> >> ? ? Practical Python for Astronomers is a series of hands-on workshops to >> ? ? explore the Python language and the analysis tools it provides. The >> ? ? emphasis is on using Python to solve real-world problems that >> ? ? astronomers are likely to encounter in research. Some features: >> >> ? ? - Workshops immediately use plotting, analysis, and file reading tools. >> ? ? - Along the way elements of the Python language are introduced. >> ? ? - Workshops are interactive using examples run by participants on their laptops. >> ? ? - Comprehensive instructions a given for installing a full Python environment. >> >> ? ? There are two goals. First is to provide tutorials suitable for >> ? ? self-study by those wishing to learn Python for astronomy. The >> ? ? greater goal is for those knowledgable in Python to teach the workshop >> ? ? series at their local institutions, adapting the content as desired. >> ? ? To that end we have developed the content in Sphinx RestructuredText >> ? ? and hosted the source on github at >> ? ? https://github.com/python4astronomers/. Anyone interested can clone >> ? ? the repository or download a tarball and make modifications needed to >> ? ? present the material locally. We would also welcome comments, fixes, >> ? ? or suggestions for improvement. This can be done as a Github issue or >> ? ? pull request, or by sending email to aldcroft at head.cfa.harvard.edu >> ? ? . >> >> ? ? The workshop material here was presented in the Spring of 2011 at the >> ? ? Harvard / Smithsonian Center for Astrophysics. A range of about 25 to >> ? ? 50 people participated in the different workshops, which were 1.5 >> ? ? hours in duration. One key accomplishment was installing a working >> ? ? Python with NumPy, SciPy, and IPython on over 50 laptops (MacOS, >> ? ? linux, and Windows) during a single session. >> >> ? ? Tom Aldcroft >> ? ? Tom Robitaille >> ? ? Brian Refsdal >> ? ? Gus Muench >> ? ? _______________________________________________ >> ? ? AstroPy mailing list >> ? ? AstroPy at scipy.org >> ? ? http://mail.scipy.org/mailman/listinfo/astropy >> >> >> >> >> -- >> Kelle Cruz, PhD ? http://kellecruz.com/ >> 917.725.1334 ? Hunter ext: 16486 ? AMNH ext: 3404 >> >> >> >> _______________________________________________ >> AstroPy mailing list >> AstroPy at scipy.org >> http://mail.scipy.org/mailman/listinfo/astropy > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy > > From jturner at gemini.edu Fri May 27 13:32:47 2011 From: jturner at gemini.edu (James Turner) Date: Fri, 27 May 2011 13:32:47 -0400 Subject: [AstroPy] Practical Python for Astronomers web tutorial In-Reply-To: References: <4DDFD6F8.1080909@gemini.edu> Message-ID: <4DDFE03F.7070201@gemini.edu> Yes, starting with vanilla IRAF might save work overall. Now if only I could find the time to finish up the Python/IRAF installation we've been working on, for wider distribution... James. On 27/05/11 13:26, Tom Aldcroft wrote: > As people have noted, getting SciSoft and EPD to play together and > allowing easy switching between environments could be tricky. For > Kelle's original need to set up a bunch of summer REUs to go through > the tutorial and then use PyRAF, I would recommend uninstalling (or > ignoring) the SciSoft installation and installing IRAF / PyRAF from > the base of EPD. This requires possibly more work up front, but if > you have 10 new students and start playing around with paths and links > on each laptop I think there will be trouble. Also, do they *all* > have Macs? If not there need to instructions for linux as well. > > If anybody can provide a specific IRAF / PyRAF installation guide > assuming a clean EPD7.0 install on Mac and/or linux, that would > tremendous! This would be immediately included in the Practical > Python Workshops. I'm a bit on the busy side this week and have > travel impending so I don't think I can do this right now... > > Thanks, Tom > > On Fri, May 27, 2011 at 12:53 PM, James Turner wrote: >> I think you could get started by allowing EPD to set up its environment >> as normal and then setting "iraf" and "IMTOOLRC" (and possibly "host") >> to point to scisoft (compare with what they're currently set to). You >> might need to set "tmp" too (don't ask why). Of course you'll also need >> to add the "bin" directory with all the symbolic links to "cl" etc. in >> your "PATH". Disclaimer: I have no real experience with SciSoft and hear >> it's IRAF setup is quite quirky, but this kind of thing works for my own >> source installations. Following Chris's suggestions might be safest. >> >> When you install PyRAF, I think you'll need the EPD environment set up >> first. >> >> Cheers, >> >> James. >> >> >> On 27/05/11 11:50, Kelle Cruz wrote: >>> Such a great resource, thanks guys!! >>> >>> We've decided to do the workshops during the first two weeks of our CUNY/AMNH >>> REU program and so far, I anticipate one problem: Scisoft. There's a bit in the >>> Installation workshop about handling packages with multiple python installations >>> (http://python4astronomers.github.com/installation/packages.html#multiple-pythons-on-your-computer), >>> but I would like some specific advice on how to handle existing Scisoft >>> installations. >>> >>> Specifically, if we want to keep Scisoft installed for the IRAF installation >>> (which we need for pyraf, right?), but NOT for the Python install, what should >>> we do? I figure there's some combo of environment variables that need >>> setting/unsetting in the .tcshrc and if someone can point us in the right >>> direction, that would help get us started as smoothly as possible. >>> >>> Are there any other conflicts that I should anticipate with installing the EPD >>> on computers that have existing Python installations? >>> >>> Thanks for your advice! >>> kelle >>> >>> On Mon, May 23, 2011 at 10:18 AM, Tom Aldcroft>> > wrote: >>> >>> My colleagues and I would like to announce the availability of a web >>> tutorial aimed at teaching Python to astronomers through a series of >>> interactive workshops: >>> >>> http://python4astronomers.github.com/ >>> >>> Practical Python for Astronomers is a series of hands-on workshops to >>> explore the Python language and the analysis tools it provides. The >>> emphasis is on using Python to solve real-world problems that >>> astronomers are likely to encounter in research. Some features: >>> >>> - Workshops immediately use plotting, analysis, and file reading tools. >>> - Along the way elements of the Python language are introduced. >>> - Workshops are interactive using examples run by participants on their laptops. >>> - Comprehensive instructions a given for installing a full Python environment. >>> >>> There are two goals. First is to provide tutorials suitable for >>> self-study by those wishing to learn Python for astronomy. The >>> greater goal is for those knowledgable in Python to teach the workshop >>> series at their local institutions, adapting the content as desired. >>> To that end we have developed the content in Sphinx RestructuredText >>> and hosted the source on github at >>> https://github.com/python4astronomers/. Anyone interested can clone >>> the repository or download a tarball and make modifications needed to >>> present the material locally. We would also welcome comments, fixes, >>> or suggestions for improvement. This can be done as a Github issue or >>> pull request, or by sending email to aldcroft at head.cfa.harvard.edu >>> . >>> >>> The workshop material here was presented in the Spring of 2011 at the >>> Harvard / Smithsonian Center for Astrophysics. A range of about 25 to >>> 50 people participated in the different workshops, which were 1.5 >>> hours in duration. One key accomplishment was installing a working >>> Python with NumPy, SciPy, and IPython on over 50 laptops (MacOS, >>> linux, and Windows) during a single session. >>> >>> Tom Aldcroft >>> Tom Robitaille >>> Brian Refsdal >>> Gus Muench >>> _______________________________________________ >>> AstroPy mailing list >>> AstroPy at scipy.org >>> http://mail.scipy.org/mailman/listinfo/astropy >>> >>> >>> >>> >>> -- >>> Kelle Cruz, PhD ? http://kellecruz.com/ >>> 917.725.1334 ? Hunter ext: 16486 ? AMNH ext: 3404 >>> >>> >>> >>> _______________________________________________ >>> AstroPy mailing list >>> AstroPy at scipy.org >>> http://mail.scipy.org/mailman/listinfo/astropy >> _______________________________________________ >> AstroPy mailing list >> AstroPy at scipy.org >> http://mail.scipy.org/mailman/listinfo/astropy >> >> -- James E.H. Turner Gemini Observatory Southern Operations Centre, Casilla 603, Tel. (+56) 51 205609 La Serena, Chile. Fax. (+56) 51 205650 From favilac at astro.uson.mx Fri May 27 13:43:09 2011 From: favilac at astro.uson.mx (Fernando Avila Castro) Date: Fri, 27 May 2011 10:43:09 -0700 Subject: [AstroPy] Practical Python for Astronomers web tutorial In-Reply-To: <4DDFE03F.7070201@gemini.edu> References: <4DDFD6F8.1080909@gemini.edu> <4DDFE03F.7070201@gemini.edu> Message-ID: <4DDFE2AD.8040700@astro.uson.mx> Hi. I have an installer for IRAF 2.14.1 with STSCI Python for Ubuntu LTS and Debian Stable. It uses the following distro packages to keep it a stock as possible. python-pmw python-urwid ipython python-numpy python-scitools python-tk python-dev python-mpmath python-scientific python-scipy python-matplotlib python-matplotlib-data The installer is here: http://www.astro.uson.mx/favilac/linastro.html On 05/27/2011 10:32 AM, James Turner wrote: > Yes, starting with vanilla IRAF might save work overall. > > Now if only I could find the time to finish up the Python/IRAF > installation we've been working on, for wider distribution... > > James. > -- P.Dr. Fernando A. ?vila-Castro ?rea de Astronom?a, UNISON http://www.astro.uson.mx/favilac Cel 6621 99 48 45 From sienkiew at stsci.edu Fri May 27 14:46:15 2011 From: sienkiew at stsci.edu (Mark Sienkiewicz) Date: Fri, 27 May 2011 14:46:15 -0400 Subject: [AstroPy] Practical Python for Astronomers web tutorial In-Reply-To: References: <4DDFD6F8.1080909@gemini.edu> Message-ID: <4DDFF177.2080109@stsci.edu> Tom Aldcroft wrote: > As people have noted, getting SciSoft and EPD to play together and > allowing easy switching between environments could be tricky. There is an easy answer: Don't switch between environments. - start a terminal window - select an environment - work in that environment - close that terminal window For your default login configuration, do not select either environment. Once you select an environment, just live with the constraint that you can't select the other one in the same terminal window. Since you can have arbitrarily many terminal windows, it should not be that much of a hardship. Mark From aldcroft at head.cfa.harvard.edu Fri May 27 16:31:04 2011 From: aldcroft at head.cfa.harvard.edu (Tom Aldcroft) Date: Fri, 27 May 2011 16:31:04 -0400 Subject: [AstroPy] Python installation: erratum Message-ID: Please pardon me for one more mail about the Practical Python Workshops. I was horrified to just discover that an extra ".. " in the installation source RST file wiped out the *key* links to the installation instructions for Mac, Linux, and Windows. They have been restored: http://python4astronomers.github.com/installation/python_install.html#install-core-python - Tom From thomas.robitaille at gmail.com Fri May 27 17:37:32 2011 From: thomas.robitaille at gmail.com (Thomas Robitaille) Date: Fri, 27 May 2011 17:37:32 -0400 Subject: [AstroPy] Question about Atpy with MySQL In-Reply-To: References: <06336FF0-7C9F-47ED-A510-B3AC7FB81254@mnstate.edu> Message-ID: Hi Juan, Thanks for spotting the typos in the documentation. It should now be fixed: http://atpy.github.com/format_sql.html Let me know if you run into any other issues! Cheers, Tom On Monday, May 23, 2011 at 6:55 PM, Dr. Juan E Cabanela Ph.D. wrote: > Tom, > Thank you for catching that, I will try again when I get back to my computer (this is from my iPad). > > I will note the documentation at > > http://atpy.github.com/format_sql.html > > is in error if what you say is correct. It states as an example: > > > > > t = atpy.TableSet('mysql', user='monty', password='spam', \ > database='measurements') > > and all the other examples on that page state user/password/database, although the API documentation below that does in fact show user/passwd/db. > > Juan > > > On May 23, 2011, at 5:40 PM, Thomas Robitaille wrote: > > Hi Juan, > > > > The issue is that the options for MySQL in ATpy are not user/password/database, but user/passwd/db. See the full API here: > > > > http://atpy.github.com/format_sql.html > > > > So in your case, try: > > > > import atpy > > t = atpy.Table('mysql', user='juan', passwd='XXXX', db='BigGalaxies', \ > > table='CompareParams', \ > > query='SELECT * FROM CompareParams;' ) > > > > By the way, if the query is just to select all rows from a table, you don't need to specify query="select * from table;" as that is the default. > > > > Let me know if this does not solve your issue! > > > > Cheers, > > Tom > > > > > > On Monday, May 23, 2011 at 4:38 PM, Dr. Juan E Cabanela Ph.D. wrote: > > > I am new to python, trying to convert myself from the pain that is the current combination of perl, C, FORTRAN, and MySQL that I use for my research. > > > > > > I have been trying to implement a MySQL query in ATpy to my MySQL 5.5.12 database. I have ATpy 0.9.5.1 and MySQL-Python 1.2.3 installed. > > > > > > Whenever I try to issue a query like the following (in my case in the 'ipython -pylab' environment): > > > > > > import atpy > > > t = atpy.Table('mysql', user='juan', password='XXXX', database='BigGalaxies', \ > > > table='CompareParams', \ > > > query='SELECT * FROM CompareParams;' ) > > > > > > I get the following error: > > > > > > TypeError: 'password' is an invalid keyword argument for this function > > > > > > If I am interpreting the trace error correctly, the ultimate error get > > > traced to MySQLdb/connections.pyc. I suspect this means my MySQL-python installation > > > is bad. > > > > > > Does anyone have any advice/wisdom for resolving this sort of error? > > > > > > Juan > > > > > > -- > > > Dr. Juan Cabanela 218-477-2453 (V) 218-477-2290 (F) > > > Minnesota State University Moorhead WWW: http://www.cabanela.com/ > > > Dept. of Physics and Astronomy Twitter: Juan_Kinda_Guy > > > 1104 Seventh Ave South, Hagen 307B IM: AstroJuanCab (AIM) > > > Moorhead, MN 56563 cabanela at mnstate.edu (mailto:cabanela at mnstate.edu) (MSN) > > > juancab at gmail.com (mailto:juancab at gmail.com) (GTalk) > > > Public PGP Key available at: http://www.cabanela.com/juan_public.asc > > > > > > > > > > > > > > > > > > > > > _______________________________________________ > > > AstroPy mailing list > > > AstroPy at scipy.org (mailto:AstroPy at scipy.org) > > > http://mail.scipy.org/mailman/listinfo/astropy > > > > -- > Dr. Juan Cabanela 218-477-2453 (V) 218-477-2290 (F) > Minnesota State University Moorhead WWW: http://www.cabanela.com/ > Dept. of Physics and Astronomy Twitter: Juan_Kinda_Guy > 1104 Seventh Ave South, Hagen 307B IM: AstroJuanCab (AIM) > Moorhead, MN 56563 cabanela at mnstate.edu (mailto:cabanela at mnstate.edu) (MSN) > juancab at gmail.com (mailto:juancab at gmail.com) (GTalk) > Public PGP Key available at: http://www.cabanela.com/juan_public.asc From cabanela at mnstate.edu Fri May 27 17:54:07 2011 From: cabanela at mnstate.edu (Dr. Juan E Cabanela Ph.D.) Date: Fri, 27 May 2011 16:54:07 -0500 Subject: [AstroPy] Question about Atpy with MySQL In-Reply-To: References: <06336FF0-7C9F-47ED-A510-B3AC7FB81254@mnstate.edu> Message-ID: On May 27, 2011, at 4:37 PM, Thomas Robitaille wrote: > Hi Juan, > > Thanks for spotting the typos in the documentation. It should now be fixed: > > http://atpy.github.com/format_sql.html > > Let me know if you run into any other issues! Hi Tom, The only other thing that isn't clear form the documentation is that to write to the MySQL database, I found you need to declare the table name before the call to create the database: >>> t.table_name = "velocities" >>> t.write('mysql', user='monty', passwd='spam', db='measurements') Passing "table='velocities'" as an argument for the write command didn't appear to work for me, so I had to declare the table_name value first. If this isn't the way it is supposed to work, please let me know. There also doesn't appear to be a clear way to _overwrite_ an existing MySQL table. I imagine if I dig around the documentation, I would find it, but it would be nice if the scheme was documented. I DO like the fact protect exists against automatically overwriting MySQL tables. Juan -- Dr. Juan Cabanela 218-477-2453 (V) 218-477-2290 (F) Minnesota State University Moorhead WWW: http://www.cabanela.com/ Dept. of Physics and Astronomy Twitter: Juan_Kinda_Guy 1104 Seventh Ave South, Hagen 307B IM: AstroJuanCab (AIM) Moorhead, MN 56563 cabanela at mnstate.edu (MSN) juancab at gmail.com (GTalk) Public PGP Key available at: http://www.cabanela.com/juan_public.asc From oneaufs at gmail.com Sat May 28 05:31:48 2011 From: oneaufs at gmail.com (Prasanth) Date: Sat, 28 May 2011 15:01:48 +0530 Subject: [AstroPy] Practical Python for Astronomers web tutorial In-Reply-To: <4DDFF177.2080109@stsci.edu> References: <4DDFD6F8.1080909@gemini.edu> <4DDFF177.2080109@stsci.edu> Message-ID: Hello, I thought I will share the methods I use while dealing with Python packages. I use Linux though most, but not all, of this should be relevant for Mac and Windows users. Before I do that, I have a suggestion specifically regarding the REU program mentioned in a previous email: after teaching students the basics of Python, shouldn't they be exposed to the manual installation process? We don't want new users to start with installing packages, but say after 1 or 2 week of playing with Python and astronomy specific packages, perhaps they should try installing packages on their own, without relying on, say EPD. At some point we do need people to be able to do this. They may get frustrated early on, but it is better to get frustrated, and over come it, during summer than during the academic year! I would also like to suggest that, students who haven't programmed before should be asked to go through some basic Python material, such as the official tutorial or an intensive 1 or 2 day "boot camp" like class, before taking them through "python4astronomers". With regard to the same email, installing IRAF is now pretty easy, at-least on linux machines. The latest version can do automatic installation, and dependency handling, of many packages such as STSDAS. So there is no need for using Scisoft just for getting IRAF. Getting back to installing Python packages. + I use the Linux package manager to install all "heavy duty" non-Python and Python packages. These include C/C++ libraries such as HDF5, QT4, and Python packages such as wxPython, PyQt, etc., I make it a point to install the "dev" or "devel" packages along with the main package. So, these will be shared by all Python interpreters. + I set PYTHONPATH="". + I use virtualenv and virtualenvwrapper for managing all my Python installations. I won't go into how to install and setup these; it is given in the websites for these packages and is very easy. These tools are widely used by the web development community. I haven't come across any serious criticisms of these tools from this community. The latter is built on top of the former and my interactions are limited to the latter. This tool allows one to select the Python interpreter to use, gives the option to use or not use the root "site-packages" directory of the chosen interpreter, and completely separates one Python interpreter from another, except for the "~/.local" path (?). If I want to create an environment with the default Python then I issue a command like: $ mkvirtualenv --no-site-packages --distribute name_of_env The above command will create a virtual environment that uses the default Python, that excludes the contents of the default Python's "site-packages" directory, and will use the distribute package for handling installations. It also installs "pip". I usually do not use the "--no-site-packages" options, since till now I haven't managed to manually install the wxPython/wxWidgets package! I rely on the version supplied with Ubuntu, which of-course gets installed in the root "site-packages" directory. This doesn't need root privileges. The virtual environment will be created in the location specified by environment variables that are read by virtualenv and virtualenvwrapper. If I want to have an environment that uses another Python interpreter then I would download the Python interpreter and install it in some folder, say "~/bin/python2.7", and then use the option "--python" to point to the binary executable inside the "~/bin/python2.7/bin" directory. Also, one can create multiple environments that use the same interpreter. I think this can be done for EPD too: just specify path to EPD Python executable. This would be another situation, where I would not use the "--no-site-packages" options; if I were using EPD I would like to use all the packages that come with it. When I want to work in a given environment, I type: $ workon name_of_env The "name_of_env" string will be displayed in the command line prompt and so it is easy to see which environment one is working in. All Python related commands will now use the Python that corresponds to this environment. To leave the environment I type: $ deactivate The "name_of_env" string will be removed from the prompt, and the default Python will be used when needed. If I want to delete an environment I use, say because I want to start fresh: $ rmvirtualenv name_of_env There are command line completions available for these commands. These are the only commands that I have used so far, and they have served me well. + Installing packages I download the source code for the package, and run "python setup.py install", while inside an environment. This doesn't need root privileges, since the package will be installed somewhere in the user's home directory, as specified while setting up virtualenv and virtualenvwrapper. I do not delete the source code. When I want to install the same in another environment, I switch to that environment and then simply repeat the process. Just to make sure that everything gets compiled again, I delete the "build" directory before running setup.py. For packages that work with "pip" or "easy_install", I run the appropriate command while inside the environment. Anything that I do inside an environment, will affect only that environment. In fact, virtualenv will refuse to install packages that try to modify items outside the current environment(?). I try not to use the "--user" option with setup.py and pip. If I need to use the same package with another Python, I simply install it again, after switching to the concerned environment. + These steps have, till now, worked very well for me. The only complication that I can anticipate is when I will need a version of the "heavy duty" C/C++ and Python packages, that is not supplied by the linux distribution. In short, the very first command that I type in my terminal, when starting to work with Python, is "workon some_name". Then I just stay in that environment. Only on few occasions do I need to switch to a different environment, say when trying out a new version of a package, or testing code using a different version of Python. Even though this is a long description and sounds complicated, in practice it is very easy. Hope this doesn't add to the confusion! I would really like to hear from others who have used these tools. Perhaps someone has run into problems that I haven't? Does this work on Mac OS X and Windows? Thanks, Prasanth On Sat, May 28, 2011 at 12:16 AM, Mark Sienkiewicz wrote: > Tom Aldcroft wrote: > > As people have noted, getting SciSoft and EPD to play together and > > allowing easy switching between environments could be tricky. > > > There is an easy answer: Don't switch between environments. > > - start a terminal window > - select an environment > - work in that environment > - close that terminal window > > For your default login configuration, do not select either environment. > Once you select an environment, just live with the constraint that you > can't select the other one in the same terminal window. Since you can > have arbitrarily many terminal windows, it should not be that much of a > hardship. > > Mark > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy > -------------- next part -------------- An HTML attachment was scrubbed... URL: From kellecruz at gmail.com Sat May 28 07:01:25 2011 From: kellecruz at gmail.com (Kelle Cruz) Date: Sat, 28 May 2011 07:01:25 -0400 Subject: [AstroPy] Practical Python for Astronomers web tutorial In-Reply-To: References: <4DDFD6F8.1080909@gemini.edu> <4DDFF177.2080109@stsci.edu> Message-ID: Wow, thanks for all the pointers. It will take me a while to parse it all, but I think I have a general idea of which direction to go. Some quick comments/responses: - They will all have Intel Macs running 10.5 or 10.6. - I really like the idea of doing a non-Scisoft related Pyraf installation. Thanks to Chris and others for those links! shouldn't they be exposed to the manual installation process? During an 8 week program where we want them to get some science done to present at AAS? No, I don't think so. Especially if they don't have to. I'd really, *really* rather they struggle with something else...like the interpretation of their data. thanks again! kelle -- Kelle Cruz, PhD ? http://kellecruz.com/ 917.725.1334 ? Hunter ext: 16486 ? AMNH ext: 3404 -------------- next part -------------- An HTML attachment was scrubbed... URL: From oneaufs at gmail.com Sun May 29 02:33:20 2011 From: oneaufs at gmail.com (Prasanth) Date: Sun, 29 May 2011 12:03:20 +0530 Subject: [AstroPy] Practical Python for Astronomers web tutorial In-Reply-To: References: <4DDFD6F8.1080909@gemini.edu> <4DDFF177.2080109@stsci.edu> Message-ID: Ah, only if we had 96 hours a day! On Sat, May 28, 2011 at 4:31 PM, Kelle Cruz wrote: > Wow, thanks for all the pointers. It will take me a while to parse it all, > but I think I have a general idea of which direction to go. Some quick > comments/responses: > > - They will all have Intel Macs running 10.5 or 10.6. > > - I really like the idea of doing a non-Scisoft related Pyraf installation. > Thanks to Chris and others for those links! > > shouldn't they be exposed to the manual installation process? > > > During an 8 week program where we want them to get some science done to > present at AAS? No, I don't think so. Especially if they don't have to. I'd > really, *really* rather they struggle with something else...like the > interpretation of their data. > > thanks again! > kelle > > -- > Kelle Cruz, PhD ? http://kellecruz.com/ > 917.725.1334 ? Hunter ext: 16486 ? AMNH ext: 3404 > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From marquett at iap.fr Mon May 30 06:51:11 2011 From: marquett at iap.fr (Marquette Jean-Baptiste) Date: Mon, 30 May 2011 12:51:11 +0200 Subject: [AstroPy] Oblique rectangles in APLpy Message-ID: Dear APLpy gurus, I have observational rectangle fields with J2000 coordinates I would like to plot on images in galactic coordinates. But the show_rectangles routine doesn't allow to plot those rectangles obliquely. Any hint welcome. Cheers Jean-Baptiste Marquette From thomas.robitaille at gmail.com Mon May 30 12:57:20 2011 From: thomas.robitaille at gmail.com (Thomas Robitaille) Date: Mon, 30 May 2011 12:57:20 -0400 Subject: [AstroPy] Oblique rectangles in APLpy In-Reply-To: References: Message-ID: Hi Jean-Baptiste, The easiest way to do this would be to figure out the coordinates of the corners of the rectangles in Galactic coordinates, and use show_polygons to plot them. Note that you can use the fk52gal function in aplpy.wcs_util to facilitate the conversion from J2000 to Galactic coordinates. Let me know if this does not work! Cheers, Tom On Monday, May 30, 2011 at 6:51 AM, Marquette Jean-Baptiste wrote: > Dear APLpy gurus, > > I have observational rectangle fields with J2000 coordinates I would > like to plot on images in galactic coordinates. But the > show_rectangles routine doesn't allow to plot those rectangles > obliquely. > Any hint welcome. > > Cheers > Jean-Baptiste Marquette > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org (mailto:AstroPy at scipy.org) > http://mail.scipy.org/mailman/listinfo/astropy From akshar.bhosale at gmail.com Sun May 29 00:33:53 2011 From: akshar.bhosale at gmail.com (akshar bhosale) Date: Sun, 29 May 2011 10:03:53 +0530 Subject: [AstroPy] numpy,scipy installation using mkl Message-ID: Hi, Is this right forum for doubts abt numpy/scipy installation? Please find our issue below .. we have machine having intel xeon x7350 processors(8 nos) and RHEL 5.2 x86_64 with kernel 2.6.18-92.el5. We have following configuration : /opt/intel/Compiler/11.0/069/ mkl/lib/em64t Now we want to install numpy and scipy as an user in my home directory. Following are the libraries build inside MKL. libfftw2x_cdft_DOUBLE.a libmkl_blacs_sgimpt_ilp64.a libmkl_intel_ilp64.a libmkl_pgi_thread.so libmkl_vml_mc2.so libfftw2xc_intel.a libmkl_blacs_sgimpt_lp64.a libmkl_intel_ilp64.so libmkl_scalapack.a libmkl_vml_mc3.so libfftw2xf_intel.a libmkl_blas95.a libmkl_intel_lp64.a libmkl_scalapack_ilp64.a libmkl_vml_mc.so libfftw3xc_intel.a libmkl_cdft.a libmkl_intel_lp64.so libmkl_scalapack_ilp64.so libmkl_vml_p4n.so libfftw3xf_intel.a libmkl_cdft_core.a libmkl_intel_sp2dp.a libmkl_scalapack_lp64.a locale libmkl_blacs_ilp64.a libmkl_core.a libmkl_intel_sp2dp.so libmkl_scalapack_lp64.so mkl77_blas.mod libmkl_blacs_intelmpi20_ilp64.a libmkl_core.so libmkl_intel_thread.a libmkl_sequential.a mkl77_lapack1.mod libmkl_blacs_intelmpi20_lp64.a libmkl_def.so libmkl_intel_thread.so libmkl_sequential.so mkl77_lapack.mod libmkl_blacs_intelmpi_ilp64.a libmkl_em64t.a libmkl_lapack95.a libmkl.so mkl95_blas.mod libmkl_blacs_intelmpi_ilp64.so libmkl_gf_ilp64.a libmkl_lapack.a libmkl_solver.a mkl95_lapack.mod libmkl_blacs_intelmpi_lp64.a libmkl_gf_ilp64.so libmkl_lapack.so libmkl_solver_ilp64.a mkl95_precision.mod libmkl_blacs_intelmpi_lp64.so libmkl_gf_lp64.a libmkl_mc3.so libmkl_solver_ilp64_sequential.a libmkl_blacs_lp64.a libmkl_gf_lp64.so libmkl_mc.so libmkl_solver_lp64.a libmkl_blacs_openmpi_ilp64.a libmkl_gnu_thread.a libmkl_p4n.so libmkl_solver_lp64_sequential.a libmkl_blacs_openmpi_lp64.a libmkl_gnu_thread.so libmkl_pgi_thread.a libmkl_vml_def.so Version we are trying to build of numpy and scipy are : numpy : 1.6.0b2 and scipy : 0.9.0 We have configured python as a user in my home directory with version : 2.6.6 our machine has python : 2.4.3 We want to know the exact procedure for installing it using MKL as we are facing lot of issues while installing the same. Please help us. -------------- next part -------------- An HTML attachment was scrubbed... URL: From ianc at ucla.edu Tue May 31 10:26:44 2011 From: ianc at ucla.edu (Ian Crossfield) Date: Tue, 31 May 2011 07:26:44 -0700 Subject: [AstroPy] Q re. function minimization Message-ID: <4DE4FAA4.6020604@ucla.edu> What functions or wrappers do people use for minimizing a function in multidimensional parameter space? I have some experience with the scipy.optimize subfunctions but I've never been entirely happy with them; there also appear to be options in sage and a Python MPFIT. What I'd like is a method that lets one minimize a function but also allows things like passed keyword arguments (unsupported in scipy.optimize) and above all a way to minimize while holding specified parameters fixed -- so that one can minimize over all parameters, and then repeat with some parameters held constant without having to write an entirely new function for each peculiar set of parameters. Is anything that offers these features this out there? How do people deal with this? Thanks, -- Ian Crossfield UCLA Astronomy KH 3-145J http://www.astro.ucla.edu/~ianc/ From neilcrighton at gmail.com Tue May 31 10:31:10 2011 From: neilcrighton at gmail.com (Neil Crighton) Date: Tue, 31 May 2011 16:31:10 +0200 Subject: [AstroPy] Q re. function minimization In-Reply-To: <4DE4FAA4.6020604@ucla.edu> References: <4DE4FAA4.6020604@ucla.edu> Message-ID: I've used pyminuit: http://code.google.com/p/pyminuit/ Occasionally it's a bit flaky, but it mostly seems to work well and the documentation is very good. Neil On 31 May 2011 16:26, Ian Crossfield wrote: > What functions or wrappers do people use for minimizing a function in > multidimensional parameter space? ?I have some experience with the > scipy.optimize subfunctions but I've never been entirely happy with > them; there also appear to be options in sage and a Python MPFIT. ?What > I'd like is a method that lets one minimize a function but also allows > things like passed keyword arguments (unsupported in scipy.optimize) and > above all a way to minimize while holding specified parameters fixed -- > so that one can minimize over all parameters, and then repeat with some > parameters held constant without having to write an entirely new > function for each peculiar set of parameters. > > Is anything that offers these features this out there? ?How do people > deal with this? > > Thanks, > > -- > Ian Crossfield > UCLA Astronomy > KH 3-145J > http://www.astro.ucla.edu/~ianc/ > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy > From brefsdal at head.cfa.harvard.edu Tue May 31 10:47:01 2011 From: brefsdal at head.cfa.harvard.edu (Brian Refsdal) Date: Tue, 31 May 2011 10:47:01 -0400 Subject: [AstroPy] Q re. function minimization In-Reply-To: <4DE4FAA4.6020604@ucla.edu> References: <4DE4FAA4.6020604@ucla.edu> Message-ID: <4DE4FF65.1010402@head.cfa.harvard.edu> If you're looking to easily hold parameters fixed, you might look at Sherpa. Sherpa includes minimizers comparable to scipy.optimize and the documentation includes examples of user-defined functions. Sherpa even has methods for computing confidence limits on model parameters like MINUIT. http://pysherpa.blogspot.com/2010/11/user-defined-model-and-fit-statistic.html http://cxc.cfa.harvard.edu/contrib/sherpa/ Brian On 05/31/2011 10:26 AM, Ian Crossfield wrote: > What functions or wrappers do people use for minimizing a function in > multidimensional parameter space? I have some experience with the > scipy.optimize subfunctions but I've never been entirely happy with > them; there also appear to be options in sage and a Python MPFIT. What > I'd like is a method that lets one minimize a function but also allows > things like passed keyword arguments (unsupported in scipy.optimize) and > above all a way to minimize while holding specified parameters fixed -- > so that one can minimize over all parameters, and then repeat with some > parameters held constant without having to write an entirely new > function for each peculiar set of parameters. > > Is anything that offers these features this out there? How do people > deal with this? > > Thanks, > From jeanluc.menut at free.fr Tue May 31 10:53:55 2011 From: jeanluc.menut at free.fr (Jean-Luc Menut) Date: Tue, 31 May 2011 16:53:55 +0200 Subject: [AstroPy] modifying FITS data Message-ID: <4DE50103.8010300@free.fr> Hello All, I'm currently struggling with Pyfits, I hope somebody could help me. My problem is the following : I have a FITS file mostly structured as an OIFITS (but not completly), of which I would like to modify several of the tables' data. The file is structured as the following: In [47]: orig.info() Filename: 2006-03-01-05-03.fits No. Name Type Cards Dimensions Format 0 PRIMARY PrimaryHDU 9 () int16 1 OI_WAVELENGTH BinTableHDU 17 95R x 2C [E, E] 2 OI_VIS BinTableHDU 48 1R x 14C ['I', 'D', 'D', 'D', '113C', '113C', '113D', '113D', '113D', '113D', '1D', '1D', '2I', '113L'] 3 OI_VIS2 BinTableHDU 38 1R x 10C [I, D, D, D, 95D, 95D, 1D, 1D, 2I, 95L] 4 OI_ARRAY BinTableHDU 31 3R x 5C [3A, 2A, 1I, 1E, 3D] 5 OI_TARGET BinTableHDU 49 1R x 17C [1I, 7A, 1D, 1D, 1E, 1D, 1D, 1D, 7A, 7A, 1D, 1D, 1D, 1D, 1E, 1E, 7A] The table OI_VIS has several fields. One of them is 'VISAMP' : In [51]: print orig['OI_VIS'].data['VISAMP'] array([[ 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.]]) What I would like to do is to replace these value by some others (actually interpolated from other FITS data). I probably missed something obvious but I did not manage to find how to do that. By example, if I try to do In [80]: orig['OI_VIS'].data['VISAMP']=tuple(ones(14)) I obtain: ValueError: field named VISAMP not found. So any information is welcome. Thanks by advance, Jean-Luc From erwin at mpe.mpg.de Tue May 31 10:55:48 2011 From: erwin at mpe.mpg.de (Peter Erwin) Date: Tue, 31 May 2011 16:55:48 +0200 Subject: [AstroPy] Q re. function minimization In-Reply-To: <4DE4FAA4.6020604@ucla.edu> References: <4DE4FAA4.6020604@ucla.edu> Message-ID: On May 31, 2011, at 4:26 PM, Ian Crossfield wrote: > What functions or wrappers do people use for minimizing a function in > multidimensional parameter space? I have some experience with the > scipy.optimize subfunctions but I've never been entirely happy with > them; there also appear to be options in sage and a Python MPFIT. What > I'd like is a method that lets one minimize a function but also allows > things like passed keyword arguments (unsupported in scipy.optimize) and > above all a way to minimize while holding specified parameters fixed -- > so that one can minimize over all parameters, and then repeat with some > parameters held constant without having to write an entirely new > function for each peculiar set of parameters. Doesn't the Python version of MPFIT do that? This implementation: http://cars9.uchicago.edu/software/python/mpfit.html has a "parinfo" keyword that allows you to keep specified parameters fixed. (It's basically a translation of Craig Markwardt's IDL MPFIT code into Python; one of Craig's modifications to the original FORTRAN implementations was to add parameter constraints.) cheers, Peter > > Is anything that offers these features this out there? How do people > deal with this? > > Thanks, > > -- > Ian Crossfield > UCLA Astronomy > KH 3-145J > http://www.astro.ucla.edu/~ianc/ > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy ============================================================= Peter Erwin Max-Planck-Insitute for Extraterrestrial erwin at mpe.mpg.de Physics, Giessenbachstrasse tel. +49 (0)89 30000 3695 85748 Garching, Germany fax +49 (0)89 30000 3495 http://www.mpe.mpg.de/~erwin From fred.grollier at gmail.com Tue May 31 12:01:20 2011 From: fred.grollier at gmail.com (=?iso-8859-1?Q?Fr=E9d=E9ric?= Grollier) Date: Tue, 31 May 2011 18:01:20 +0200 Subject: [AstroPy] modifying FITS data In-Reply-To: <4DE50103.8010300@free.fr> References: <4DE50103.8010300@free.fr> Message-ID: <20110531160120.GA12108@free.fr> On Tue, May 31, 2011 at 04:53:55PM +0200, Jean-Luc Menut wrote: > Hello All, [snip] > What I would like to do is to replace these value by some others > (actually interpolated from other FITS data). > > By example, if I try to do > > In [80]: orig['OI_VIS'].data['VISAMP']=tuple(ones(14)) > > I obtain: > ValueError: field named VISAMP not found. Hello, doing this with a loop should work: for i in range(orig['OS_VIS'].data.shape[0]): orig['OI_VIS'].data.field('VISAMP')[i] = the_value Fred. From embray at stsci.edu Tue May 31 12:17:29 2011 From: embray at stsci.edu (Erik Bray) Date: Tue, 31 May 2011 12:17:29 -0400 Subject: [AstroPy] modifying FITS data In-Reply-To: <20110531160120.GA12108@free.fr> References: <4DE50103.8010300@free.fr> <20110531160120.GA12108@free.fr> Message-ID: <4DE51499.9060800@stsci.edu> On 05/31/2011 12:01 PM, Fr?d?ric Grollier wrote: > On Tue, May 31, 2011 at 04:53:55PM +0200, Jean-Luc Menut wrote: >> Hello All, > > [snip] > >> What I would like to do is to replace these value by some others >> (actually interpolated from other FITS data). >> >> By example, if I try to do >> >> In [80]: orig['OI_VIS'].data['VISAMP']=tuple(ones(14)) >> >> I obtain: >> ValueError: field named VISAMP not found. > > Hello, > > doing this with a loop should work: > > for i in range(orig['OS_VIS'].data.shape[0]): > orig['OI_VIS'].data.field('VISAMP')[i] = the_value > > Rather than looping it might be better (certainly faster) to do a slice assignment: orig['OI_VIS'].data['VISAMP'][:] = the_value If the_value is not the same size/dimensions as the array being assigned to, then you can modify the slice accordingly. For example: orig['OI_VIS'].data['VISAMP'][:len(the_value)] = the_value This works because the lookup by field name returns a numpy array that can be manipulated as such. Why direct assignment like you tried originally isn't implemented I'm not entirely sure. Erik