Proposed change to development practices
by Nathan Goldbaum
Hi all,
Now that yt 3.1 is making its way out the door, I'd like to come back to a
discussion we had last year about bugfixes.
I've made a pull request to the YTEP repository that summarized the change
I'm proposing:
https://bitbucket.org/yt_analysis/ytep/pull-request/48/modify-ytep-1776-t...
Basically, I think bugfixes need to go to the stable branch rather than the
yt branch. Currently, all new changes go to the yt branch. While this
does simplify our development practices, this makes it difficult for us to
release new versions that only include fixes for bugs. Instead, even minor
version releases that are cut from the yt branch include new features and
API breakages.
I think this approach violates the principle of least surprise for users
who have download a bugfix release.
The solution, I think, is to ensure bugfixes are only applied to the stable
branch. This will ensure that we can straightforwardly do bugfix releases
that inlude only bugfixes and that new features and API changes are
isolated to the more "experimental" yt branch.
This does come with some possible down sides. In particular, there will
likely be some confusion as we switch our development practices. In
addition, new contributors may find it difficult to split pull requests
into new features that should go to the yt branch and bugfixes that should
go to the stable branch. It also adds a new maintenance burden: as soon as
a bugfix pull request to stable goes in, there should be an accompanying
merge from the stable branch into the yt branch to ensure that both
branches get bugfixes. This gets more complicated if the bugfix looks
different in the yt branch and the stable branch.
All that said, I think these new maintenance burdens can be overcome with a
bit of vigilance and maybe some new tooling.
I've probably said enough about this. What do you all think? Comments and
concerns are very welcome.
Best,
Nathan Goldbaum
_______________________________________________
yt-dev mailing list
yt-dev(a)lists.spacepope.org
http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
3 years, 10 months
Near-term maintenance goals
by Matthew Turk
Hi all,
I wanted to lay out a few things that I'm hoping to work on in the near
term future. These are all maintenance goals for yt, not new
functionality; I'm thinking that for me at least new functionality may need
to take a backseat for a little while, even though it's tons of fun.
* Performance: there have been performance regressions for patch AMR code,
as well as the codebase as a whole, in support of generality. I've started
looking at this and hope to spend more time on it. Sam, Kacper and I have
all expressed interest in plugging at this. Next week i'm going to try to
set up a hangout where we look at issues, set up regression testing for
performance, and then even perhaps try to address them in the hangout.
* Python3: I have spent a little bit of time working on Python 3 in the
past (mostly in collaboration with Jill), and even just updated the PR this
morning. There are still test failures and errors which I would like to
fix. My plan here is to first get a working 2to3 codebase, and then
evaluate single codebase. We've recently gotten requests for this as well.
* Infrastructure: shoring up the infrastructure to expand testing and
development is a near-term goal, and Kacper and I will probably be
partnering on this.
-Matt
3 years, 10 months
Issue #976: Sphere object creation and RAMSES - fails with
a ValueError (yt_analysis/yt)
by Benjamin Thompson
New issue 976: Sphere object creation and RAMSES - fails with a ValueError
https://bitbucket.org/yt_analysis/yt/issue/976/sphere-object-creation-and...
Benjamin Thompson:
Hello.
Using the sample dataset hosted on YT http://yt-project.org/data/output_00101.tar.gz, I have encountered a bug with the sphere field.
this code
```
#!python
import yt
from yt import derived_field
from yt.units.yt_array import YTArray
snap = yt.load("test_gal/output_00101/info_00101.txt")
snapa = snap.all_data()
b = snapa.ds.arr(2,"kpc")
print b.in_units("code_length")
sphere = snapa.ds.sphere([ 0.50638366, 0.50013661, 0.49566656], b)
print sphere
#print sphere["index"], "index"
print sphere["particle_mass"], "particle mass"
sphere2 = snapa.ds.sphere([ 0.50638366, 0.50013661, 0.49566656], 7.01290365998e-05)
print sphere2
#print sphere2["index"], "index"
print sphere2["particle_mass"], "particle mass"
sphere3 = snapa.ds.sphere([ 0.50638366, 0.50013661, 0.49566656], 0.00869399672932)
print sphere3
#print sphere3["index"], "index"
print sphere3["particle_mass"], "particle mass"
```
will return this error
```
#!python
YTSphere (info_00101): center=[ 0.50638366 0.50013661 0.49566656] code_length, radius=2.3424788685e-05 code_length
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
/gpfs/home/bthompson1/ramses_pp/ramses_pp/ben/real_selene_analysis/test_highz.py in <module>()
63 #print sphere["index"], "index"
64
---> 65 print sphere["particle_mass"], "particle mass"
66
67
/gpfs/home/bthompson1/python/yt/yt-x86_64/src/yt-hg/yt/data_objects/data_containers.pyc in __getitem__(self, key)
246 return self.field_data[f]
247 else:
--> 248 self.get_data(f)
249 # fi.units is the unit expression string. We depend on the registry
250 # hanging off the dataset to define this unit object.
/gpfs/home/bthompson1/python/yt/yt-x86_64/src/yt-hg/yt/data_objects/data_containers.pyc in get_data(self, fields)
685
686 read_particles, gen_particles = self.index._read_particle_fields(
--> 687 particles, self, self._current_chunk)
688 for f, v in read_particles.items():
689 self.field_data[f] = self.ds.arr(v, input_units = finfos[f].units)
/gpfs/home/bthompson1/python/yt/yt-x86_64/src/yt-hg/yt/geometry/geometry_handler.pyc in _read_particle_fields(self, fields, dobj, chunk)
232 self._chunk_io(dobj, cache = False),
233 selector,
--> 234 fields_to_read)
235 return fields_to_return, fields_to_generate
236
/gpfs/home/bthompson1/python/yt/yt-x86_64/src/yt-hg/yt/utilities/io_handler.pyc in _read_particle_selection(self, chunks, selector, fields)
181 # mylog.debug("Filling %s from %s to %s with %s",
182 # field_f, my_ind, my_ind+vals.shape[0], field_r)
--> 183 rv[field_f][my_ind:my_ind + vals.shape[0],...] = vals
184 ind[field_f] += vals.shape[0]
185 # Now we need to truncate all our fields, since we allow for
```
_______________________________________________
yt-dev mailing list
yt-dev(a)lists.spacepope.org
http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
3 years, 10 months
Nightly binary wheels
by John ZuHone
Hi devs,
Just a note--nightly pip wheel binary builds of yt for Linux, Mac OS X, and Windows (all 64-bit) are now available at http://binstar.org/jzuhone/yt/files. These are built every night at 2 am EST of off the tip of the yt branch.
Previously, it was just Anaconda Python tarballs. If you have a chance at some point, download one and see how it works.
Soon, I will be issuing a docs PR for pip wheels as well as the nightly builds.
Best,
John
John ZuHone
Kavli Center for Astrophysics and Space Research
Massachusetts Institute of Technology
77 Massachusetts Ave., 37-582G
Cambridge, MA 02139
(w) 617-253-2354
(m) 781-708-5004
jzuhone(a)space.mit.edu
jzuhone(a)gmail.com
http://www.jzuhone.com
_______________________________________________
yt-dev mailing list
yt-dev(a)lists.spacepope.org
http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
3 years, 10 months
Migrate lists to yt-project.org?
by Matthew Turk
Hi all,
Because of convenience at the time, back in 2007 the mailing lists were set
up on spacepope.org. I'd like to propose we migrate them to be on
yt-project.org.
To do this, I will coordinate with our ISP (dreamhost) to:
1) Move the lists to lists.yt-project.org
2) Set up email aliases for the old addresses
3) Set up web redirects for the old web addresses for archives
4) Encourage people to write to the new list addresses
(Anything else need to happen?)
In perpetuity we'll keep the old URLs available, which probably means
keeping that domain around.
-Matt
PS You probably won't see this for a while, since our mailing lists are
lagging right now. I've filed a ticket about that.
_______________________________________________
yt-dev mailing list
yt-dev(a)lists.spacepope.org
http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
3 years, 10 months
preliminary release announcement text
by John ZuHone
Hi all,
Here is a first draft of a release announcement. It is a little long and wordy, so please feel free to make changes. I did some minor curation of the “minor or bugfix changes” list, but I’m sure we could do more.
Best,
John
The yt community is proud to announce the release of yt 3.1.
yt (http://yt-project.org <http://yt-project.org/>) is an open source, community-developed
toolkit for analysis and visualization of volumetric data of all
types, with a particular emphasis on astrophysical simulations and
nuclear engineering simulations.
This is a scheduled feature release. Highlighted changes in yt 3.1:
Major changes:
++++++++++++++
* The RADMC-3D export analysis module has been updated.
* Performance improvements for grid frontends.
* Added a frontend for Dark Matter-only NMSU Art simulations.
* The absorption spectrum generator has been updated.
* The PerspectiveCamera has been updated and a new SphericalCamera has been added.
* The unit system now supports unit equivalencies and has improved support for MKS units.
* Data object selection can now be chained, allowing selecting based on multiple constraints.
* Added the ability to manually override the simulation unit system.
* The documentation has been reorganized and has seen substantial improvements.
Minor or bugfix changes:
++++++++++++++++++++++++
* The Gadget InternalEnergy and StarFormationRate fields are now read in with the correct units.
* Substantial improvements for the PPVCube analysis module and support for FITS datasets.
* The center of a PlotWindow plot can now be set to the maximum or minimum of any field.
* Fixes for yt testing infrastructure.
* Projections are now performed using an explicit path length field for all coordinate systems.
* An example notebook for simulations using the OWLS data format has been added to the documentation.
* Fix for the camera.draw_line function.
* Minor fixes and improvements for yt plots.
* Significant documentation reorganization and improvement.
* Fixed a conflict in the CFITSIO library used by the x-ray photon simulator analysis module.
* Miscellaneous code cleanup.
* yt now hooks up to the python logging infrastructure in a more standard fashion, avoiding issues
with yt logging showing up with using other libraries.
* A bug in the calculation of the plot bounds for off-axis slice plots has been fixed.
* Improvements for the yt-rockstar interface.
* Fix issues with plot positioning with saving to postscript or encapsulated postscript.
* It is now possible to supply a default value for get_field_parameter.
* A bug in the interpretation of the units of RAMSES simulations has been fixed.
* Improvements and bugfixes for the halo analysis framework.
* Fix issues with the default setting for the "center" field parameter.
* yt can now be run in parallel on a subset of available processors using an MPI subcommunicator.
* Fix for incorrect units when loading an Athena simulation as a time series.
* Improved support for Enzo 3.0 simulations that have not produced any active particles.
* Fix for periodic radius vector calculation.
* Improvements for the Maestro and Castro frontends.
* Clump finding is now supported for more generic types of data.
* Fix unit consistency issue when mixing dimensionless unit symbols.
* Improved memory footprint in the photon_simulator.
* Large grids in Athena datasets produced by the join_vtk script can now be optionally split,
improving parallel performance.
* Slice plots now accept a “data_source" keyword argument.
* Corrected inconsistent octrees in the RAMSES frontend.
* Nearest neighbor distance field added.
* Improvements for the ORION2 frontend.
* Enzo 3.0 frontend can now read active particle attributes that are arrays of any shape.
* A “setup_function" has been added to the LightRay initializer.
* Fixes for accessing deposit fields for FLASH data.
* Added tests for ORION datasets containing sink and star particles.
* Added wrapper functions for numpy array manipulation functions.
* Added support for packed HDF5 Enzo datasets.
A more comprehensive list of the changes in this release can be found at http://yt-project.org/docs/3.1/reference/changelog.html <http://yt-project.org/docs/3.1/reference/changelog.html>.
Standard Installation Methods
-----------------------------
As with previous releases, you can install yt from source using one of
the following methods.
1) From the install script (http://yt-project.org/#getyt):
# Installation
$ wget http://hg.yt-project.org/yt/raw/stable/doc/install_script.sh
$ bash install_script.sh
# Update
$ yt update
2) From pip (source or binary wheel, see below for more details):
# Installation
$ pip install yt
# Update
$ pip install -U yt
3) From the Anaconda Python Distribution (https://store.continuum.io/cshop/anaconda/ <https://store.continuum.io/cshop/anaconda/>):
# Installation
$ conda install yt
# Update
$ conda update yt
Note that it might take a day or two for the conda package to be updated.
Installing Binary Packages via pip
----------------------------------
New to this release is the ability to install binary packages (“wheels”) using
pip on Windows and Mac OS X (64-bit only for both). This has the advantage
of not needing to install yt from source using a proper compiler setup, which
has caused occasional problems on both of these platforms and prevented us
from installing yt easily on other Python distributions.
We have so far been able to install and run the binary distribution via pip on the
following platforms and Python stacks:
Windows x86_64:
* Enthought Canopy Python (https://www.enthought.com/products/canopy/ <https://www.enthought.com/products/canopy/>)
* WinPython (http://winpython.sourceforge.net/ <http://winpython.sourceforge.net/>)
Mac OS X x86_64:
* Enthought Canopy Python (https://www.enthought.com/products/canopy/ <https://www.enthought.com/products/canopy/>)
* Homebrew Python (http://brew.sh/ <http://brew.sh/>)
* Python.org <http://python.org/> Python
* Mac OS X’s system Python
* MacPorts Python (https://www.macports.org/ <https://www.macports.org/>)
This is somewhat experimental, so other distributions may work (or not),
please submit bug reports or successes to the mailing list or to the Bitbucket
issues page (http://bitbucket.org/yt_analysis/yt/issues).
All distributions must be Python v. 2.7. The requirements for installing yt
via this method are the same as from source:
* NumPy
* h5py
* HDF5
* SymPy
* Matplotlib
* IPython (not required, but strongly recommended)
To install a new version of yt on one of these platforms, simply do
$ pip install yt
and you should get the binary distribution automatically. Also, if your
python installation is system-wide (e.g., the Mac system Python)
you might need to run pip with administrator privileges.
For more information, including more installation instructions, links to
community resources, and information on contributing to yt’s
development, please see the yt homepage at http://yt-project.org <http://yt-project.org/> and
the documentation for yt-3.1 at http://yt-project.org/docs/3.1 <http://yt-project.org/docs/3.1>.
yt is the product of a large community of developers and users and we
are extraordinarily grateful for and proud of their contributions. Please
forward this announcement on to any interested parties.
As always, if you have any questions, concerns, or run into any trouble
updating please don't hesitate to send a message to the mailing list or
stop by our IRC channel.
Thank you,
The yt development team
3 years, 11 months