Hello,
First an introduction, I am Stuart Mumford I am doing a solar physics PhD
at Sheffield University in the UK. I work primarily on MHD wave simulations
of the solar atmosphere studying energy propagation and wave excitation.
The code SAC (http://www.aanda.org/10.1051/0004-6361:200809800) solves for
perturbations ontop of a fixed background to deal with the very wide range
of the parameters e.g. density and temperature in the solar atmosphere. I
also attended SciPy this year, where I met some of you and discovered yt,
mainly as a consequence of my contributions to SunPy (www.sunpy.org) which
is a solar physics data analysis library.
So that's that over with, now having been playing with yt on and off since
SciPy I started using it properly this week, as the need arose. I am very
impressed with yt's capabilities and I want to be able to make better use
of it. To that end I am trying to write a frontend for SAC, as I have been
speaking to some of you about in IRC (I am Cadair in IRC and on GH).
I have some questions regarding this, as far as I can ascertain a frontend
provides a mapper between the stored data and a abstracted format that yt
understands? I have started looking in the code for FLASH and Stream to try
and get an idea where to start and not got very far. So some questions to
start:
Where should the file handling be done?
Where is the file opened?
What things should the StaticOutput subclass define?
Thanks once again for your help on IRC over the last few days and I hope
you can point me in the right direction.
Stuart
New issue 638: Document IPython integration
https://bitbucket.org/yt_analysis/yt/issue/638/document-ipython-integration
Nathan Goldbaum:
There should be a section in the narrative docs that goes over yt's IPython integration. Specifically, this should cover `imods`, `yt notebook`, and IPython pretty display for plots and volume renderings.
New issue 637: Matplotlib 1.3.0 doesn't seem compatible with yt.pmods
https://bitbucket.org/yt_analysis/yt/issue/637/matplotlib-130-doesnt-seem-c…
Matthew Turk:
Running "from yt.pmods import *" even from within a non-MPI python results in:
```
#!python
File "pm.py", line 1, in <module>
from yt.pmods import *
File "/home/mturk/yt/yt/yt/pmods.py", line 364, in <module>
from yt.mods import *
File "/home/mturk/yt/yt/yt/pmods.py", line 235, in __import_hook__
m = __load_tail__(q, tail)
File "/home/mturk/yt/yt/yt/pmods.py", line 339, in __load_tail__
m = __import_module__(head, mname, m)
File "/home/mturk/yt/yt/yt/pmods.py", line 279, in __import_module__
m = imp.load_module(fqname, fp, pathname, stuff)
File "/home/mturk/yt/yt/yt/mods.py", line 129, in <module>
from yt.visualization.api import \
File "/home/mturk/yt/yt/yt/pmods.py", line 235, in __import_hook__
m = __load_tail__(q, tail)
File "/home/mturk/yt/yt/yt/pmods.py", line 339, in __load_tail__
m = __import_module__(head, mname, m)
File "/home/mturk/yt/yt/yt/pmods.py", line 279, in __import_module__
m = imp.load_module(fqname, fp, pathname, stuff)
File "/home/mturk/yt/yt/yt/visualization/api.py", line 31, in <module>
from color_maps import \
File "/home/mturk/yt/yt/yt/pmods.py", line 234, in __import_hook__
q, tail = __find_head_package__(parent, name)
File "/home/mturk/yt/yt/yt/pmods.py", line 323, in __find_head_package__
q = __import_module__(head, qname, parent)
File "/home/mturk/yt/yt/yt/pmods.py", line 279, in __import_module__
m = imp.load_module(fqname, fp, pathname, stuff)
File "/home/mturk/yt/yt/yt/visualization/color_maps.py", line 26, in <module>
import matplotlib
File "/home/mturk/yt/yt/yt/pmods.py", line 234, in __import_hook__
q, tail = __find_head_package__(parent, name)
File "/home/mturk/yt/yt/yt/pmods.py", line 328, in __find_head_package__
q = __import_module__(head, qname, parent)
File "/home/mturk/yt/yt/yt/pmods.py", line 279, in __import_module__
m = imp.load_module(fqname, fp, pathname, stuff)
File "/home/mturk/yt-x86_64/lib/python2.7/site-packages/matplotlib-1.3.0-py2.7-linux-x86_64.egg/matplotlib/__init__.py", line 130, in <module>
from matplotlib.compat import subprocess
File "/home/mturk/yt/yt/yt/pmods.py", line 239, in __import_hook__
__ensure_fromlist__(m, fromlist)
File "/home/mturk/yt/yt/yt/pmods.py", line 357, in __ensure_fromlist__
submod = __import_module__(sub, subname, m)
File "/home/mturk/yt/yt/yt/pmods.py", line 279, in __import_module__
m = imp.load_module(fqname, fp, pathname, stuff)
File "/home/mturk/yt-x86_64/lib/python2.7/site-packages/matplotlib-1.3.0-py2.7-linux-x86_64.egg/matplotlib/compat/subprocess.py", line 79, in <module>
CalledProcessError = subprocess.CalledProcessError
AttributeError: 'module' object has no attribute 'CalledProcessError'
```
Hi all,
We're proud to release yt version 2.5.5. This is an unscheduled point
release that includes all bug fixes identified and fixed since the
release of 2.5.4 on July 2nd. We missed our scheduled release date a
bit, and have continued to push off a 2.6 as development on the 2.x
branch winds down.
Additions, changes and bug fixes include:
* New absorption spectrum analysis module with documentation
* Fix for volume rendering on the command line
* map_to_colormap will no longer return out-of-bounds errors
* Fixes for dds in covering grid calculations
* Library searching for build process is now more reliable
* Unit fix for "VorticityGrowthTimescale" field
* Pyflakes stylistic fixes
* Adding ability to draw lines with Grey Opacity in volume rendering
* Updated physical constants to reflect 2010 CODATA data
* Number density added to FLASH
* Dependency updates (including IPython 1.0)
* Many fixes for Athena frontend
* Radius and ParticleRadius now work for reduced-dimensionality datasets
* Source distributions now work again!
* Better notebook support for yt plots
* Athena data now 64 bits everywhere
* Grids displays on plots are now shaded to reflect the level of refinement
* show_colormaps() is a new function for displaying all known colormaps
* PhasePlotter by default now adds a colormap.
If you are using the stable branch of yt from an installation script,
you can upgrade using "yt update" or "yt update --all" to upgrade your
full dependency stack. If you are using the development branch, you
may already have these fixes. A tarball of this release has been
uploaded to the Python Package Index (PyPI).
yt releases often feature contributions from many individuals; this
release includes first time contributions from John Forbes, Noel
Scudder and Hilary Egan.
Documentation for this release can be found at:
http://yt-project.org/docs/dev/
Thanks very much,
Matt, on behalf of the yt development team
Hi all,
I wanted to call attention to an oddity I discovered yesterday with
regard to some Athena data I'm working with.
For background, Athena data consists of "domains" which may or may not
exist on different levels of refinement. They adhere to some fairly
strict rules, which may be found here:
https://trac.princeton.edu/Athena/wiki/AthenaDocsSMRGeom
Domains are further subdivided into "grids". If the run was done on a
single processor, then each domain just has one grid, and the two are
identical. If it was a parallel job, then a domain will usually have
multiple grids. The important point here is that there is no
requirement that a grid on a given refinement level needs to be fully
self-contained within a parent grid on the level just below it (though
Athena does adhere to refine_by = 2).
To further complicate things, for parallel runs the grids are split
into different vtk files in different directories for each processor
and also different directories for each refinement level. We've
constructed the Athena frontend so that this data may be read as well
as data that was produced by a serial run. There is a tool in the
Athena source, join_vtk, which allows one to take the data from a
parallel run and merge it into single vtk files, which has the
consequence of reducing the number of grids to one per domain.
This brings me to the dataset I mentioned in the beginning. It
consists of a simple set of 5 3D cubical nested domains, all centered
on the domain center, which all of them except the highest-level
domain exactly half the size on a side of the previous one. The last
one is 3/4 the size on a side of the previous one. The run was done on
512 processors, and each domain is split up into 512 grids.
The result looks like this:
https://hub.yt-project.org/nb/bfd6uv
As you can see, the issue is that the grids on the highest refinement
level really don't have a well-defined notion of parent grids, because
these grids are overlapping multiple grids on the parent level.
However, at the cell level there are no overlaps of this type--every
cell on a given level is fully contained within a cell on the level
below it.
Here is the same data, except that it has been merged with join_vtk
and now there are only five grids:
https://hub.yt-project.org/nb/7qxvas
In this case, every grid has a well-defined parent, so everything looks correct.
In fact, I realized today that there is no provision in the Athena
frontend for establishing parent-child relationships amongst grids--so
apparently this is being determined somewhere in the code as a
fallback when the frontend doesn't provide this information?
I'd just like to open this up for discussion as to how we might make
it possible to at least work with this data in a limited way, since it
seems like it's a situation that might come up often enough. I've
created an issue:
https://bitbucket.org/yt_analysis/yt/issue/636/overlapping-grids-in-athena-…
Best,
John Z
--
John ZuHone
Postdoctoral Researcher
NASA/Goddard Space Flight Center
jzuhone(a)gmail.com
john.zuhone(a)nasa.g
Hi everyone,
We're proud to announce the third ALPHA release of yt 3.0. yt has
recently transitioned to a time-based release plan (
https://ytep.readthedocs.org/en/latest/YTEPs/YTEP-0008.html ) and this
is the third scheduled alpha of 3.0, although we've grossly missed the
estimated date of July 15. No date for a "final" release has yet been
set, but there will likely be several more alphas before that time.
The yt 2.5 codebase, and further updates in the 2.x series, will be
supported for a considerable amount of time and you do not need to
upgrade.
= yt 3.0?! =
yt 3.0 represents a new direction forward for yt: getting rid of all
the underlying assumptions that data needs to be sectioned off into
nice little grid patches. This includes supporting Octree codes
natively (NMSU-ART and RAMSES), eventual support for SPH codes, and
even opaque data structures where the data is extremely large (ARTIO).
We're even planning support for natively handling cylindrical and
spherical coordinates.
More: http://blog.yt-project.org/post/WhatsUpWith30.html
However, this *is* an alpha release. Not all of the existing codes
have been ported to 3.0.
Additionally, this release benefits from the technical and
non-technical contributions from many new people. yt is developed in
the context of a community of contributors, and with the push toward a
new architecture, we aim to expand that community considerably. In
particular, this release has considerably benefited from contributions
from many new individuals.
= Getting It! =
To try out yt 3.0, you can now pull from the main yt repository,
update to the yt-3.0 branch, and rebuild your extensions. Or, if you
would like to create a new, safely sectioned off environment, simply
re-run the normal "development" install script after changing the
variable BRANCH to "yt-3.0".
If you would like to try out yt 3.0 and are having trouble, please
write to the yt-users mailing list for assistance.
The yt 3.0 install script may also work, which can be obtained by
executing these commands:
wget http://hg.yt-project.org/yt/raw/yt-3.0/doc/install_script.sh
bash install_script.sh
= What's New? =
A demo notebook demonstrating much new functionality, and including
the full release notes, can be found here:
http://hub.yt-project.org/nb/88u14f
The bullet-pointed release notes can be found at the end of this
email, as well. The main improvements include *considerable* memory
usage reduction, ARTIO spatial data indexing support, better particle
support, an overhaul of the selection methods for data objects, and a
mechanism for on-the-fly definitions of particle types based on
boolean filters.
Additionally, this release was used in the production of the AGORA
project flagship paper. The analysis script used there can be seen at
this short URL:
http://goo.gl/SwY9Uv
and the paper can be found here: http://arxiv.org/abs/1308.2669
= Reporting Problems =
If you test out yt 3.0 we want to hear if it DID or DID NOT work!
Feedback is crucial at this time. yt-users and yt-dev are both good
forums for discussion, asking questions, and reporting problems. Lots
of things have changed on the backend, but we have attempted to
minimize the user-facing changes.
To report a bug please go here:
https://bitbucket.org/yt_analysis/yt/issues/new
Note that you will not receive updates if you are not logged in when
you create the bug.
= What's Next? =
The next alpha release (3.0a4) will be released sometime this fall,
but development can be monitored either at
http://bitbucket.org/yt_analysis/yt-3.0 or in the main yt repository
under the named branch "yt-3.0". We hope to focus on generalizing
Octree support further, adding better non-Cartesian support, and
supporting generic smoothing kernel definitions.
If you'd like to participate in yt development, please stop by #yt on
irc.freenode.net ( http://yt-project.org/irc.html ) or yt-dev (
http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org ), or
submit a pull request on BitBucket.
Thanks very much,
Matt, on behalf of the yt development team
Changelog:
(Including 434 changesets from 7 contributors since 3.0a2)
* Fixes for volume rendering and porting of the Cython kD-tree. VR
now works for octree codes, albeit slowly.
* Major reorganization and simplification of selector code, including
generic periodicity
* ARTIO now has spatial data support
* Major bug fix for FLASH IO. FLASH works again, following a
regression in 3.0a2.
* Merged with mainline yt development, bringing developments from yt
2.5.3 and 2.5.4 into 3.0.
* OWLS (Gadget-HDF5) data now updated to match functionality of other
Gadget datasets.
* Creationg of "arbitrary_grid" data selector for flexible particle
deposition operations. This enables creation of a grid of arbitrary
dimensions that can have particles smoothed or deposited within them.
* Conversion of all CIC operations to particle deposition operations.
This results in much more flexible particle type selection and speed
improvements.
* Refactoring of particle fields to enable simpler creation and
collection of particle fields. Particle fields defined for Eulerian
codes and Lagrangian codes are now interchangeable.
* New, currently unused parallel ring iteration. To be explored in
the future for smoothing kernels.
* Many improvements for NMSU-ART
* Fixed a crazy QuadTree deposition bug for projections that showed up
when projecting octree particle deposition results through a
restricted data object.
* Fixed a crazy off-by-one bug for binary Gadget IO.
* Enormous Octree diet.
* Oct leaf nodes now cost 256 bits each, which may eventually be
reduced to 192 bits. Previous versions were considerably larger.
* Octree traversal is strictly recursive, enabling
distributed-memory octrees to be implemented.
* RAMSES octrees are created by-domain instead of globally. This
will enable better parallelism in the future and faster ghost zone
generation when that is implemented.
* Particle octree are now constructed via Z-curve generation.
This is considerably faster (10-20x) as well as much more memory
conservative. This was designed to utilize parallel octree
construction in the future.
* Particle octrees are now indexed by coarse base-level indexing
of domain regions. This enables unsorted particle files to be indexed
for IO as well as spatial selection without mandating that leaf nodes
are fully-contained within a single file. Reduction in total oct leaf
count of ~8x for multi-file particle datasets.
* YTDataContainer no longer requires .shape and .size
* Only requested fields are plotted, not translated fields
* Reduce memory usage of spatially-chunked fields for patch datasets
* Added arbitrary particle loader, load_particles.
* Units fix for RAMSES boxlen!=1.0 datasets
* Particle filtering for dynamically-created particle types, similar
to derived fields
* Improve type consistency in TotalQuantity and other derived quantities.
* Fixed major error with ghost zone generation and uninitialized values
* Some resiliency for particle datasets that do not specify a domain
* Bugfix for 1- and 2-D Enzo datasets
* Improvements to the Tipsy frontend, including auto-detecting parameter files
* Enable specification of n_ref when creating particle dataset
* Enable particle deposition to back-act on particle fields
* Major fix for octree particle counting, resulting in smaller meshes
* Enable "source" specification for volume rendering
* Enable initial volume rendering of octree datasets
* Fixed error with RAMSES C/F ordering of data
* Many fixes related to field dependency calculation
* FLASH cylindrical & polar pixelization fix
New issue 636: Overlapping grids in Athena SMR data
https://bitbucket.org/yt_analysis/yt/issue/636/overlapping-grids-in-athena-…
John ZuHone:
Athena grids on a given refinement level are not required to be fully self-contained within grids on the level just below them, so unique parent-child relationships are not established in some cases.
A notebook showing this issue:
https://hub.yt-project.org/nb/bfd6uv
If the data from the different grids is merged together (outside of yt) so that there is one grid per domain, the data looks correct:
https://hub.yt-project.org/nb/7qxvas
Responsible: jzuhone
New issue 635: find_max() is broken for particle codes:
https://bitbucket.org/yt_analysis/yt/issue/635/find_max-is-broken-for-parti…
Nathan Goldbaum:
```
pf.h.find_max(('deposit', 'Gas_density'))
---------------------------------------------------------------------------
AttributeError Traceback (most recent call last)
<ipython-input-5-d81b7d6ae9ea> in <module>()
----> 1 pf.h.find_max(('deposit', 'Gas_density'))
AttributeError: 'ParticleGeometryHandler' object has no attribute 'find_max'
```
I think this operation will fail for RAMSES as well.
New issue 634: Set up derived quantities for individual particle types
https://bitbucket.org/yt_analysis/yt/issue/634/set-up-derived-quantities-fo…
Matthew Turk:
Much like how we have the `particle_fields.py` file for creating specific types of particle fields, we should have this for derived quantities as well.
Responsible: MatthewTurk
Hello everyone (and Britton and Kacper),
I had a PR accepted last week to create an additional cookbook entry on
overplotting grid edges on a PlotWindow instance. I've looked, and the
change is in the repository: yt-doc/source/cookbook/complex_plots.rst ( the
header: "Plotting Grid Edges Over Fluids" right after "Plotting Particles
Over Fluids"). Unfortunately, it does not appear on either the
yt.readthedocs.org page, nor does it appear yt-project.org docs page. I
thought that the readthedocs page got rebuilt immediately after every PR,
or is this not the case? Does anyone know what might be going on here?
I wonder how long this may have been happening and if our docs are lagged
behind the changes that have occurred to them recently?
Britton, Matt suggested I talk to you since you own the docs repo on BB.
Kacper, I figured I'd include you here since you're in charge of the
yt-project.org/docs page.
Cameron
Cameron Hummels
Postdoctoral Researcher
Steward Observatory
University of Arizona
http://chummels.org