Mailman 3 September 2015 - yt-dev

Anyone know anything about EAGLE data?
by Nathan Goldbaum 19 Sep '15

19 Sep '15

Hi all, Robert Thompson, who is working on a generic SPH FOF/halo analysis framework pointed out to me today that loading EAGLE snipshot files like the example we have is broken. See: https://bitbucket.org/yt_analysis/yt/issues/1085/loading-eagle-snipshot-dat… Does anyone have some experience with EAGLE data to help me debug this? Thanks! -Nathan

2 1

Issue #1085: Loading EAGLE snipshot data is broken (yt_analysis/yt)
by Nathan Goldbaum 17 Sep '15

17 Sep '15

New issue 1085: Loading EAGLE snipshot data is broken https://bitbucket.org/yt_analysis/yt/issues/1085/loading-eagle-snipshot-dat… Nathan Goldbaum: The following test script: ``` import yt ds = yt.load('snipshot_399_z000p000/snip_399_z000p000.0.hdf5') ds.index ``` Triggers an infinite recursion during field detection. Right now the EagleDataset class uses the OWLSFieldInfo class under the hood, which doesn't actually work since the Eagle data doesn't contain any of the chemical species fields needed to generate species fields. One way to "fix" this would be to disable species fields. I have no idea if we *could* actually reconstruct the species fields given the information in the snipshot file.

1 0

yt team meeting 6: Tuesday, September 22 at 2:00pm EDT
by Britton Smith 17 Sep '15

17 Sep '15

Hi everyone, The next yt team meeting will be Tuesday, September 22 at 11am PDT, 1pm CDT, 2pm EDT, 7pm BST. All are welcome to attend. Here's a link to a hangout reminder: https://plus.google.com/events/cggp1jhukf9m5duo4qsnpkc09kc?authkey=CLCAiKyE… Here's a link to the agenda: https://docs.google.com/document/d/1qrbFU8_B0HFxGQFL6hruJ35NHAJTaUgiiIe_AAZ… If you'd like to add something for discussion, please go ahead. See you there! Britton

1 0

Bay Area mini-sprint?
by Matthew Turk 16 Sep '15

16 Sep '15

Hi all, Anybody up for a mini-sprint in the Bay Area in the week of the 16th of October, perhaps focused on VR? Maybe the 14th-16th? Cameron, Suoqing, would you be up for attending? I'll be out there, and the following week is an NDS meeting in San Diego, so there may be some other NCSA folks on that coast then. We might even be able to abscond with Sam for a few hours. -Matt

5 6

BioJulia and community expansion
by Matthew Turk 16 Sep '15

16 Sep '15

Hi all, BioJulia just posted this: https://github.com/BioJulia/Bio.jl/issues/102 I think we should look at this checklist, and see how yt is doing based on these metrics, as well. Ensuring that we are and remain a welcoming, inclusive community is something I think we all truly believe in -- especially as evidenced by our code of conduct discussion some months back. Let's keep going down that road. -Matt

1 0

yt under python 3.5
by Nathan Goldbaum 15 Sep '15

15 Sep '15

Hi all, Just a quick note: yt seems to work correctly under python 3.5 in so far as the unit tests all pass. As always, if you've switched to python 3 as your daily driver and notice any bugs or issues in yt please do let us know. -Nathan

1 0

proposed change to development process
by Britton Smith 11 Sep '15

11 Sep '15

Hi everyone, I had some ideas for improving the yt development process that I wanted to run by everyone. This can be discussed further at our upcoming team meeting and if people are in favor, I will issue a pull request to the relevant YTEP. STATEMENT OF PROBLEM Currently, development proceeds roughly as follows. The two main active branches within the central yt repository are *yt* and *stable*. The tip of *stable* is the latest release and the *yt* branch is the de facto "development version" of the code. Until recently, we have not been very good at regularly scheduled minor releases and so the *stable* branch sits for quite some time with many bugs that are fixed within the development branch. This effectively makes *stable* unusable and pushes most users to the *yt* branch. When new features are developed, pull requests are issued to the single head of the *yt* branch. Because this is the version most people are actually using, the current policy is to not allow PR with new functionality to be accepted until they are 100% ready (full functionality, tests, docs, etc). As we have already seen, this makes collaborative development very cumbersome, as it requires people to create forks of the fork from which the PR originates. They then must issue PRs to that fork after which time the original PR is updated. The current volume render refactor is the perfect example of this. PROPOSED SOLUTION Before I lay out the proposed solution, I want to list a number of recent developments that I think will make this possible: 1. Nathan's new script for backporting changes now keeps *stable* and *yt* synced on bugfixes. 2. We have returned to doing minor releases containing only bugfixes, thanks again to Nathan's hard work. This and point 1 means that users are once again safe to be on *stable*, and now *should* be there most of the time. 3. Bitbucket now supports bookmarks, meaning that PRs can be issued to specific bookmarks instead of to branches or heads named only by the changeset hash. 4. The weekly PR triage hangouts are making it easier to process PRs and also providing a place to strategize getting larger PRs accepted. Thanks to Hilary for keeping this going. With the above in mind, I propose the following: 1. Create a "development" bookmark to sit at the tip of the *yt* branch. All PRs containing relatively small new features are issued to this. The requirements for acceptance remain the same: PRs accepted to "development" must contain all intended functionality and be fully documented. This allows the "development" bookmark to be defined explicitly as everything that will be included in the next major release. 2. Large new features should have a corresponding YTEP that has been accepted. After the YTEP has been accepted, a PR should be issued to the *yt* branch. After some initial discussion, this PR is pulled into the main yt repo with a bookmark named after the feature. Once this has happened, developers can now issue new PRs specifically to this bookmark. This is effectively what we have now with the volume render work in the "experimental" bookmark, only we would rename the bookmark to something like "vr-refactor". As with PRs issued directly to "development", only after the new feature is 100% ready shall it be merged into the "development" bookmark. 3. We continue to make use of the PR triage hangouts to establish when large features are ready to be merged. I believe this will have the following benefits: 1. Large, new features can be developed collaboratively without the need for forks of forks of forks. 2. New, underdevelopment features are more accessible to the larger community by simply updating to named bookmarks from the main repo (no need for "just pull these changes from my fork"). 3. The "development" branch is preserved as a place only for ready-to-be-released features (i.e., polished and documented). All told, this is really just a small tweak on our current process. Please comment with any thoughts, or even a +/-1 if your feelings can be summed up thusly. Britton

7 23

Close the yt-3.0 branch?
by Nathan Goldbaum 10 Sep '15

10 Sep '15

Hi all, I just noticed today that the yt-3.0 branch still shows up in "hg branches" for fresh clones of the main yt repository. I think that was done on purpose so that changes on the yt-3.0 branch could be included into the yt branch after the point when the merged yt-3.0 into the yt branch heading up to the release of yt-3.0. At this point, I think we've long passed the time when people might want to submit pull requests from the yt-3.0 branch. I can make it so that the yt-3.0 branch will no longer show up in "hg branches" or more importantly in the branch dropdown menu for commits or pull requests on bitbucket by creating a --close-branch commit on the yt-3.0 branch. The yt-3.0 branch will still live on in history and will also show up in "hg branches --closed". Unfortunately, it seems I can't make a pull request that includes only a branch closing commit. Would anyone object to me pushing such a commit directly to yt_analysis/yt? -Nathan

3 3

ANN: yt-3.2.1 release
by Nathan Goldbaum 10 Sep '15

10 Sep '15

We are proud to announce the release of yt 3.2.1! yt (http://yt-project.org) is an open-source, community-developed toolkit for analysis and visualization of volumetric data of all types, with a particular emphasis on astrophysical and nuclear engineering simulations. Version 3.2.1 is a regularly scheduled bugfix release including fixes for a number of issues reported since the release of yt 3.2. These changes are summarized below and include a number of bugfixes, performance improvements, and documentation improvements. We urge all users of version 3.2 to upgrade to yt 3.2.1 immediately. Summary of changes ------------------ * Fixed an issue with reading ORION2 star particle data * The "particle_velocity_cylindrical_radius" field now uses the correct field definition. * Prevent unit test failure from occuring if scipy or astropy is not installed * Present better error messages when trying to load empty files or directories * Add support for the fabs ufunc to YTArray * Remove enzo installation option from the install script * Fixed issue with the EPS writer when it is supplied a ProfilePlot * Fixed reading of the NMSU-ART star particle creation time field * Use a stronger hashing algorithm for calculating selection hashes to fix issues with data object selection in Python 3 * Fix an error in the docstrings of Cosmology.t_from_z() * Fix python3 answer test failure for Enzo species field names * Fix incorrect use of "normal" instead of "center" in spherical coordinate fields. * Avoid NameError when comparing temperature units with offsets * Avoid creating YTQuantity instances with more than one value * Include requirements files and cuda sources in source distribution. Avoid including a full docs build. * Add search bar to main docs page * Add canonical link for documentation to improve google search results for yt documentation * Use nice latex representation for unit labels returned by DerivedField.get_label() * Add assert_allclose_units to avoid test failures in Numpy 1.10 and newer * Avoid unsafe casting error in Numpy 1.10 or newer * Fix incorrect plots created by scripts that call PlotContainer._setup_plots() directly. * Allow the creation of gradient fields based on fields with dimensionless units * Correct two incorrect paths in the documentation on loading data * The MaxLocation and MinLocation derived quantities now work correctly with non-cartesian data and particle fields. * Use OrderedDict in PlotWindow answer tests to avoid unstable test names in Python 3 * Speed improvements for the absorption_spectrum analysis module * Speed improvements for smoothed covering grids * Fixed a small CSS issue in the main docs page * Fix incorrect velocity and time units for Gadget cosmological datasets * Use truediv operator in unit object tests for Python 3 compatibility * Fix several bugs in the photon simulator analysis module. * Ensure camera width and center are in code units before rendering, avoiding a possible segmentation fault. * Ensure fields passed to process_octree during smoothing are contiguous. Fixes a regression in the values of smoothed SPH fields. Coming Soon! ------------ The next major release of yt will be version 3.3, which is slated to include an overhaul of the volume rendering system and support for analyzing and visualizing unstructured mesh data. Standard Installation Methods ----------------------------- As with previous releases, you can install yt from source using one of the following methods. 1) From the install script (http://yt-project.org/#getyt ): If you previously installed yt from the install script, you can update your installation in-place using the following command: # Update $ yt update If you are on the “stable” branch, updating will bring you from yt 3.2 to 3.2.1, incorporating all changes since 3.2, whereas if you are on the development “yt” branch, only the changes since your last update should be incorporated. To install from scratch, do the following: # Installation $ wget http://bitbucket.org/yt_analysis/yt/raw/stable/doc/install_script.sh $ bash install_script.sh 2) From pip (source or binary wheel, see below for more details): # Installation $ pip install yt # Update $ pip install -U yt 3) From the Anaconda Python Distribution (https://store.continuum.io/cshop/anaconda/): # Installation $ conda install yt # Update $ conda update yt Note that it might take up to a week for the conda package built by Continuum Analytics to be updated for yt 3.2.1. Installing Binary Packages via pip If you do not have compilers available, it is also possible to install binary packages (“wheels”) using pip on Windows and Mac OS X (64-bit only for both). This has the advantage of not needing to install yt from source using a proper compiler setup, which has caused occasional problems on both of these platforms and prevented us from installing yt easily on other Python distributions. Note that it may take several days for binary wheels for yt 3.2.1 to be uploaded. We have so far been able to install and run the binary distribution via pip on the following platforms and Python stacks: Note that it might take a day or two for the pip wheels to be updated. Windows x86_64: Enthought Canopy Python (https://www.enthought.com/products/canopy/) WinPython (http://winpython.sourceforge.net/) Mac OS X x86_64: Enthought Canopy Python (https://www.enthought.com/products/canopy/) Homebrew Python (http://brew.sh/) Python.org Python Mac OS X’s system Python MacPorts Python (https://www.macports.org/) This is somewhat experimental, so other distributions may work (or not), please submit bug reports or successes to the mailing list or to the Bitbucket issues page (http://bitbucket.org/yt_analysis/yt/issues). Both Python 2.7 and Python 3.4 are supported. The dependencies for installing yt via this method are the same as from source: NumPy h5py HDF5 SymPy Matplotlib IPython (not required, but strongly recommended) To install a new version of yt on one of these platforms, simply do $ pip install yt and you should get the binary distribution automatically. Also, if your python installation is system-wide (e.g., the Mac system Python) you might need to run pip with administrator privileges. For more information, including more installation instructions, links to community resources, and information on contributing to yt’s development, please see the yt homepage at http://yt-project.org and the documentation for yt-3.2 at http://yt-project.org/docs/3.2. yt is the product of a large community of developers and users and we are extraordinarily grateful for and proud of their contributions. Please forward this announcement on to any interested parties. As always, if you have any questions, concerns, or run into any trouble updating please don't hesitate to send a message to the mailing list or stop by our IRC channel. All the best, Nathan Goldbaum, on behalf of the yt development team

2 1

pycharm
by Matthew Turk 10 Sep '15

10 Sep '15

Hi all, I'm thinking about getting open source project licenses for PyCharm for yt. If you'd like one, ping me off list. -Matt

3 2