Mailman 3 January 2015 - yt-dev

Request for review Python 3 changes
by Matthew Turk 22 Jan '15

22 Jan '15

Hi all, I'd like to request review on this PR from me, Jill, Kacper and Nathan: https://bitbucket.org/yt_analysis/yt/pull-request/1177/wip-second-set-of-py… I think accepting should wait until we get the bugfix/newdev discussion finished, but I think it's now past WIP. (If we decide to do all bugfixes on stable, I think it could go in relatively soon, pending review.) -Matt _______________________________________________ yt-dev mailing list yt-dev(a)lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

2 7

Proposed change to development practices
by Nathan Goldbaum 22 Jan '15

22 Jan '15

Hi all, Now that yt 3.1 is making its way out the door, I'd like to come back to a discussion we had last year about bugfixes. I've made a pull request to the YTEP repository that summarized the change I'm proposing: https://bitbucket.org/yt_analysis/ytep/pull-request/48/modify-ytep-1776-to-… Basically, I think bugfixes need to go to the stable branch rather than the yt branch. Currently, all new changes go to the yt branch. While this does simplify our development practices, this makes it difficult for us to release new versions that only include fixes for bugs. Instead, even minor version releases that are cut from the yt branch include new features and API breakages. I think this approach violates the principle of least surprise for users who have download a bugfix release. The solution, I think, is to ensure bugfixes are only applied to the stable branch. This will ensure that we can straightforwardly do bugfix releases that inlude only bugfixes and that new features and API changes are isolated to the more "experimental" yt branch. This does come with some possible down sides. In particular, there will likely be some confusion as we switch our development practices. In addition, new contributors may find it difficult to split pull requests into new features that should go to the yt branch and bugfixes that should go to the stable branch. It also adds a new maintenance burden: as soon as a bugfix pull request to stable goes in, there should be an accompanying merge from the stable branch into the yt branch to ensure that both branches get bugfixes. This gets more complicated if the bugfix looks different in the yt branch and the stable branch. All that said, I think these new maintenance burdens can be overcome with a bit of vigilance and maybe some new tooling. I've probably said enough about this. What do you all think? Comments and concerns are very welcome. Best, Nathan Goldbaum _______________________________________________ yt-dev mailing list yt-dev(a)lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

8 20

Issue #977: 3D Rotation Images Come Out Artifacted (yt_analysis/yt)
by Anonymous 22 Jan '15

22 Jan '15

New issue 977: 3D Rotation Images Come Out Artifacted https://bitbucket.org/yt_analysis/yt/issue/977/3d-rotation-images-come-out-… Anonymous: When I try to take images about a rotation the resulting images are very odd (see attachments). It may be that my code is improper, but it seems to be executing correctly (images are being produced about a rotation). I've tried it on the IsolatedGalaxy dataset and the same result occurs. I couldn't find any good info on taking images about a rotation besides the tiny enumerate(rotation()) script on the rotation() method page, so it could be that I'm doing it wrong. I've attached my code and some of the images. _______________________________________________ yt-dev mailing list yt-dev(a)lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

2 1

Near-term maintenance goals
by Matthew Turk 21 Jan '15

21 Jan '15

Hi all, I wanted to lay out a few things that I'm hoping to work on in the near term future. These are all maintenance goals for yt, not new functionality; I'm thinking that for me at least new functionality may need to take a backseat for a little while, even though it's tons of fun. * Performance: there have been performance regressions for patch AMR code, as well as the codebase as a whole, in support of generality. I've started looking at this and hope to spend more time on it. Sam, Kacper and I have all expressed interest in plugging at this. Next week i'm going to try to set up a hangout where we look at issues, set up regression testing for performance, and then even perhaps try to address them in the hangout. * Python3: I have spent a little bit of time working on Python 3 in the past (mostly in collaboration with Jill), and even just updated the PR this morning. There are still test failures and errors which I would like to fix. My plan here is to first get a working 2to3 codebase, and then evaluate single codebase. We've recently gotten requests for this as well. * Infrastructure: shoring up the infrastructure to expand testing and development is a near-term goal, and Kacper and I will probably be partnering on this. -Matt

2 5

pip requirements
by John ZuHone 19 Jan '15

19 Jan '15

Hi dev, Should we have a requirements.txt file for pip? Currently we require users who install via pip to have the required packages set up already (numpy, sympy, h5py, etc.), but if we had a requirements file then it would pick these up automatically if they weren’t already there. https://pip.pypa.io/en/latest/user_guide.html#requirements-files <https://pip.pypa.io/en/latest/user_guide.html#requirements-files> John _______________________________________________ yt-dev mailing list yt-dev(a)lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

5 9

Issue #976: Sphere object creation and RAMSES - fails with a ValueError (yt_analysis/yt)
by Benjamin Thompson 19 Jan '15

19 Jan '15

New issue 976: Sphere object creation and RAMSES - fails with a ValueError https://bitbucket.org/yt_analysis/yt/issue/976/sphere-object-creation-and-r… Benjamin Thompson: Hello. Using the sample dataset hosted on YT http://yt-project.org/data/output_00101.tar.gz, I have encountered a bug with the sphere field. this code ``` #!python import yt from yt import derived_field from yt.units.yt_array import YTArray snap = yt.load("test_gal/output_00101/info_00101.txt") snapa = snap.all_data() b = snapa.ds.arr(2,"kpc") print b.in_units("code_length") sphere = snapa.ds.sphere([ 0.50638366, 0.50013661, 0.49566656], b) print sphere #print sphere["index"], "index" print sphere["particle_mass"], "particle mass" sphere2 = snapa.ds.sphere([ 0.50638366, 0.50013661, 0.49566656], 7.01290365998e-05) print sphere2 #print sphere2["index"], "index" print sphere2["particle_mass"], "particle mass" sphere3 = snapa.ds.sphere([ 0.50638366, 0.50013661, 0.49566656], 0.00869399672932) print sphere3 #print sphere3["index"], "index" print sphere3["particle_mass"], "particle mass" ``` will return this error ``` #!python YTSphere (info_00101): center=[ 0.50638366 0.50013661 0.49566656] code_length, radius=2.3424788685e-05 code_length --------------------------------------------------------------------------- ValueError Traceback (most recent call last) /gpfs/home/bthompson1/ramses_pp/ramses_pp/ben/real_selene_analysis/test_highz.py in <module>() 63 #print sphere["index"], "index" 64 ---> 65 print sphere["particle_mass"], "particle mass" 66 67 /gpfs/home/bthompson1/python/yt/yt-x86_64/src/yt-hg/yt/data_objects/data_containers.pyc in __getitem__(self, key) 246 return self.field_data[f] 247 else: --> 248 self.get_data(f) 249 # fi.units is the unit expression string. We depend on the registry 250 # hanging off the dataset to define this unit object. /gpfs/home/bthompson1/python/yt/yt-x86_64/src/yt-hg/yt/data_objects/data_containers.pyc in get_data(self, fields) 685 686 read_particles, gen_particles = self.index._read_particle_fields( --> 687 particles, self, self._current_chunk) 688 for f, v in read_particles.items(): 689 self.field_data[f] = self.ds.arr(v, input_units = finfos[f].units) /gpfs/home/bthompson1/python/yt/yt-x86_64/src/yt-hg/yt/geometry/geometry_handler.pyc in _read_particle_fields(self, fields, dobj, chunk) 232 self._chunk_io(dobj, cache = False), 233 selector, --> 234 fields_to_read) 235 return fields_to_return, fields_to_generate 236 /gpfs/home/bthompson1/python/yt/yt-x86_64/src/yt-hg/yt/utilities/io_handler.pyc in _read_particle_selection(self, chunks, selector, fields) 181 # mylog.debug("Filling %s from %s to %s with %s", 182 # field_f, my_ind, my_ind+vals.shape[0], field_r) --> 183 rv[field_f][my_ind:my_ind + vals.shape[0],...] = vals 184 ind[field_f] += vals.shape[0] 185 # Now we need to truncate all our fields, since we allow for ``` _______________________________________________ yt-dev mailing list yt-dev(a)lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

1 0

Nightly binary wheels
by John ZuHone 17 Jan '15

17 Jan '15

Hi devs, Just a note--nightly pip wheel binary builds of yt for Linux, Mac OS X, and Windows (all 64-bit) are now available at http://binstar.org/jzuhone/yt/files. These are built every night at 2 am EST of off the tip of the yt branch. Previously, it was just Anaconda Python tarballs. If you have a chance at some point, download one and see how it works. Soon, I will be issuing a docs PR for pip wheels as well as the nightly builds. Best, John John ZuHone Kavli Center for Astrophysics and Space Research Massachusetts Institute of Technology 77 Massachusetts Ave., 37-582G Cambridge, MA 02139 (w) 617-253-2354 (m) 781-708-5004 jzuhone(a)space.mit.edu jzuhone(a)gmail.com http://www.jzuhone.com _______________________________________________ yt-dev mailing list yt-dev(a)lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

3 5

Migrate lists to yt-project.org?
by Matthew Turk 17 Jan '15

17 Jan '15

Hi all, Because of convenience at the time, back in 2007 the mailing lists were set up on spacepope.org. I'd like to propose we migrate them to be on yt-project.org. To do this, I will coordinate with our ISP (dreamhost) to: 1) Move the lists to lists.yt-project.org 2) Set up email aliases for the old addresses 3) Set up web redirects for the old web addresses for archives 4) Encourage people to write to the new list addresses (Anything else need to happen?) In perpetuity we'll keep the old URLs available, which probably means keeping that domain around. -Matt PS You probably won't see this for a while, since our mailing lists are lagging right now. I've filed a ticket about that. _______________________________________________ yt-dev mailing list yt-dev(a)lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

4 6

Issue #975: Quickstart introduction notebook has incorrect dataset URLs (yt_analysis/yt)
by mordecai 14 Jan '15

14 Jan '15

New issue 975: Quickstart introduction notebook has incorrect dataset URLs https://bitbucket.org/yt_analysis/yt/issue/975/quickstart-introduction-note… mordecai: Reporting Version: None In the Quickstart documentation, the notebook _Introduction.ipynb has three dataset download curl commands. All of them need ".gz" appended to the end for consistency with the actual filenames at yt-project.org/data. Otherwise the curl seems to fail silently.

1 0

preliminary release announcement text
by John ZuHone 14 Jan '15

14 Jan '15

Hi all, Here is a first draft of a release announcement. It is a little long and wordy, so please feel free to make changes. I did some minor curation of the “minor or bugfix changes” list, but I’m sure we could do more. Best, John The yt community is proud to announce the release of yt 3.1. yt (http://yt-project.org <http://yt-project.org/>) is an open source, community-developed toolkit for analysis and visualization of volumetric data of all types, with a particular emphasis on astrophysical simulations and nuclear engineering simulations. This is a scheduled feature release. Highlighted changes in yt 3.1: Major changes: ++++++++++++++ * The RADMC-3D export analysis module has been updated. * Performance improvements for grid frontends. * Added a frontend for Dark Matter-only NMSU Art simulations. * The absorption spectrum generator has been updated. * The PerspectiveCamera has been updated and a new SphericalCamera has been added. * The unit system now supports unit equivalencies and has improved support for MKS units. * Data object selection can now be chained, allowing selecting based on multiple constraints. * Added the ability to manually override the simulation unit system. * The documentation has been reorganized and has seen substantial improvements. Minor or bugfix changes: ++++++++++++++++++++++++ * The Gadget InternalEnergy and StarFormationRate fields are now read in with the correct units. * Substantial improvements for the PPVCube analysis module and support for FITS datasets. * The center of a PlotWindow plot can now be set to the maximum or minimum of any field. * Fixes for yt testing infrastructure. * Projections are now performed using an explicit path length field for all coordinate systems. * An example notebook for simulations using the OWLS data format has been added to the documentation. * Fix for the camera.draw_line function. * Minor fixes and improvements for yt plots. * Significant documentation reorganization and improvement. * Fixed a conflict in the CFITSIO library used by the x-ray photon simulator analysis module. * Miscellaneous code cleanup. * yt now hooks up to the python logging infrastructure in a more standard fashion, avoiding issues with yt logging showing up with using other libraries. * A bug in the calculation of the plot bounds for off-axis slice plots has been fixed. * Improvements for the yt-rockstar interface. * Fix issues with plot positioning with saving to postscript or encapsulated postscript. * It is now possible to supply a default value for get_field_parameter. * A bug in the interpretation of the units of RAMSES simulations has been fixed. * Improvements and bugfixes for the halo analysis framework. * Fix issues with the default setting for the "center" field parameter. * yt can now be run in parallel on a subset of available processors using an MPI subcommunicator. * Fix for incorrect units when loading an Athena simulation as a time series. * Improved support for Enzo 3.0 simulations that have not produced any active particles. * Fix for periodic radius vector calculation. * Improvements for the Maestro and Castro frontends. * Clump finding is now supported for more generic types of data. * Fix unit consistency issue when mixing dimensionless unit symbols. * Improved memory footprint in the photon_simulator. * Large grids in Athena datasets produced by the join_vtk script can now be optionally split, improving parallel performance. * Slice plots now accept a “data_source" keyword argument. * Corrected inconsistent octrees in the RAMSES frontend. * Nearest neighbor distance field added. * Improvements for the ORION2 frontend. * Enzo 3.0 frontend can now read active particle attributes that are arrays of any shape. * A “setup_function" has been added to the LightRay initializer. * Fixes for accessing deposit fields for FLASH data. * Added tests for ORION datasets containing sink and star particles. * Added wrapper functions for numpy array manipulation functions. * Added support for packed HDF5 Enzo datasets. A more comprehensive list of the changes in this release can be found at http://yt-project.org/docs/3.1/reference/changelog.html <http://yt-project.org/docs/3.1/reference/changelog.html>. Standard Installation Methods ----------------------------- As with previous releases, you can install yt from source using one of the following methods. 1) From the install script (http://yt-project.org/#getyt): # Installation $ wget http://hg.yt-project.org/yt/raw/stable/doc/install_script.sh $ bash install_script.sh # Update $ yt update 2) From pip (source or binary wheel, see below for more details): # Installation $ pip install yt # Update $ pip install -U yt 3) From the Anaconda Python Distribution (https://store.continuum.io/cshop/anaconda/ <https://store.continuum.io/cshop/anaconda/>): # Installation $ conda install yt # Update $ conda update yt Note that it might take a day or two for the conda package to be updated. Installing Binary Packages via pip ---------------------------------- New to this release is the ability to install binary packages (“wheels”) using pip on Windows and Mac OS X (64-bit only for both). This has the advantage of not needing to install yt from source using a proper compiler setup, which has caused occasional problems on both of these platforms and prevented us from installing yt easily on other Python distributions. We have so far been able to install and run the binary distribution via pip on the following platforms and Python stacks: Windows x86_64: * Enthought Canopy Python (https://www.enthought.com/products/canopy/ <https://www.enthought.com/products/canopy/>) * WinPython (http://winpython.sourceforge.net/ <http://winpython.sourceforge.net/>) Mac OS X x86_64: * Enthought Canopy Python (https://www.enthought.com/products/canopy/ <https://www.enthought.com/products/canopy/>) * Homebrew Python (http://brew.sh/ <http://brew.sh/>) * Python.org <http://python.org/> Python * Mac OS X’s system Python * MacPorts Python (https://www.macports.org/ <https://www.macports.org/>) This is somewhat experimental, so other distributions may work (or not), please submit bug reports or successes to the mailing list or to the Bitbucket issues page (http://bitbucket.org/yt_analysis/yt/issues). All distributions must be Python v. 2.7. The requirements for installing yt via this method are the same as from source: * NumPy * h5py * HDF5 * SymPy * Matplotlib * IPython (not required, but strongly recommended) To install a new version of yt on one of these platforms, simply do $ pip install yt and you should get the binary distribution automatically. Also, if your python installation is system-wide (e.g., the Mac system Python) you might need to run pip with administrator privileges. For more information, including more installation instructions, links to community resources, and information on contributing to yt’s development, please see the yt homepage at http://yt-project.org <http://yt-project.org/> and the documentation for yt-3.1 at http://yt-project.org/docs/3.1 <http://yt-project.org/docs/3.1>. yt is the product of a large community of developers and users and we are extraordinarily grateful for and proud of their contributions. Please forward this announcement on to any interested parties. As always, if you have any questions, concerns, or run into any trouble updating please don't hesitate to send a message to the mailing list or stop by our IRC channel. Thank you, The yt development team

1 0