I have a work-in-progress PR in the hopper regarding GDF reading and writing:
What it does is implement direct writing of covering grids to GDF
files (something I've been using in my research, to work with the
resulting files directly in yt) and clobbering existing files with the
same path if the user allows for it.
However, when fiddling around with this I determined that it was
necessary to make GDF more unit-aware. Ultimately, I determined that
changes were needed that would break the standard that we had
determined for the files (http://yt-project.org/gdf.txt).
The two main changes are:
1) Remove "field_to_cgs", and allow the fields to be in whatever units
we wish them to be in the file (which are specified as HDF5
2) Add a new top-level group containing the information for
"length","mass", and "time" units. These will be used when the file is
opened up in yt to determine the units.
Since (for now) we do automatic conversion to cgs, that is something
that is still left unimplemented, but otherwise I think this is all to
do for now.
I'm writing the email in case anyone has any suggestions or objections
to the format changes--particularly Jeff or Kacper. We'll obviously
need to document them if they are accepted.
NASA/Goddard Space Flight Center
Greeting yt developers,
First, I want to congratulate everyone here on the successful release
of yt-3.0. This was a massive effort on the part of so many and a
true testament to the strength of this team.
At the time of writing this, there are 78 members of the yt-dev
mailing list. As someone who does most of their work in very small
collaborations, this amazes me and make me very proud. In case you're
wondering, the yt-users list has 268 members.
As a project, yt has a significant amount of infrastructure: code
review with pull requests, issue tracking, automated testing, emails
lists, an IRC channel, enhancement proposals, workshops. All of this
is evidence of our legitimacy as a Real Thing. However, one big
missing piece is a system of governance. I don't know exactly what
this means, but I have some ideas, which I will share below. What I
want to do right now is to start a discussion that will, hopefully,
include as many people as possible on this list.
For me, governance means (roughly) the following:
- a set of procedures in writing for how various things are to be
done, such as acceptance of pull requests, releases, designating
developers as core contributors, etc.
- a governing body to make decisions and help guide the project.
This accomplishes a number of things, which as a project I think we
need, such as:
- overall stability of the project.
- providing a system for conflict resolution.
- maintaining the spirit of yt as a team effort.
- providing a way for active contributors to get credit for their
contribution in the form of official recognition.
So, these are my initial thoughts, but I really think this deserves a
thorough discussion with as many people participating as possible.
Please, think about what governance means to you, whether we need it,
what it should be, and what we might get out of it, and share your
thoughts over the next few days. I look forward to this discussion.
I have a set of fixes for Issue #887 (field detection). Most apply without breaking any tests, but to address all of the problems in that issue I need to add a check to YTDataContainer._determine_fields that doesn't allow a field to be added if it doesn't exist in field_list or derived_field_list.
This fixes my bug, but breaks a test based on code in yt/visualization/volume_rendering/camera.py which adds a field "temp_weightfield" (not a tuple!).
When this field is passed to _determine_fields, _get_field_info first guesses at ("gas","temp_weightfield"), but since that doesn't exist in field_info, it replaces it with ("index","temp_weightfield"), which I don't see as being any better.
There are several places where the syntax _get_field_info(*field) is used, which will break when field is a string, not a tuple. Should I fix camera.py to use an explicit fluid type? If so, what? I've tried to use the fluid type of the input field, but that apparently is also not always a tuple.
fields = [field]
if self.weight is not None:
self.weightfield = (field, "temp_weightfield")
# This is a temporary field, which we will remove at the end.
def _make_wf(f, w):
def temp_weightfield(a, b):
tr = b[f].astype("float64") * b[w]
return b.apply_units(tr, a.units)
# Now we have to tell the dataset to add it and to calculate
# its dependencies..
deps, _ = ds.field_info.check_derived_fields([self.weightfield])
fields = [self.weightfield, self.weight]
Scientific Computing Consultant
Research Computing Center
New issue 889: yt update --all fails when updating from numpy 1.7 to 1.8 on some systems
Since the addition of Pull Request #1168, which updates the version of numpy from 1.7 to 1.8, some attempts to update yt's dependencies in place fail with this error:
ImportError: cannot import name multiarray
See here for full traceback: http://paste.yt-project.org/show/5077/
This appears to be a problem on mac systems and TACC Stampede, and perhaps others.
One solution to this is to wipe yt and reinstall from the install script, but a different solution is desired.
A while back, Jeremy Ritter asked the yt-users list if yt could produce a
particle plot like that obtained through annotate_particles(), but without
loading any mesh data. The answer was "no", but that it isn't too hard to
use matplotlib directly. Personally, this is a visualization task that I do
all the time, and I think it would be useful to make this type of plot a
one-liner in yt - provided it handles units and labels and such
automatically like any other yt plot.
I've added this feature in my fork of yt. The interface is the same as for
any other yt plots, and the standard plot modification mechanisms should
work. Examples are here
<http://nbviewer.ipython.org/gist/atmyers/f8616c9ed5a9d2b027e8> and here
If there is interest in having this in mainline yt I'll write some docs and
issue a pull request.
New issue 887: Asking for naked ARTIO particle field type corrupts field detection
Field detection shows strange behavior when asking for a field without a particle type when that field doesn't exist for all types.
- Example script: http://paste.yt-project.org/show/5071/
- Example output: http://paste.yt-project.org/show/5072/
Explanation: I load a dataset with STAR type particles which has the unique field "BIRTH_TIME"
- ("STAR","BIRTH_TIME") in ds.field_list == True
- ("all","BIRTH_TIME") in ds.field_list == False
If I request ["all","BIRTH_TIME"], I correctly receive the exception
Could not find field '('all', 'BIRTH_TIME')' in AGORA_LOW_1.art.
If I request ["BIRTH_TIME"], it promotes this to ("all","BIRTH_TIME"), but then proceeds to chunk and attempt to load this non-existent field, leading to the exception (after I/O!):
Something has gone terribly wrong, _function is NullFunc for ('STAR', 'BIRTH_TIME')
Subsequent requests for ("all","BIRTH_TIME") then produce this latest exception, showing that field detection is caching some bad information somewhere.
I've tracked through the entire line of execution (I originally included it but it's so crazy and convoluted I think it's only useful for me). I've found several things that look like bugs, and I'll submit a PR with some "fixes." Since I don't know this part of the code it will need some review.
I'd like to setup deposition fields for particle velocities. This means
doing a mass-weighted nearest neighbor or cloud-in-cell deposition of the
particle velocities in a grid dataset. I'm using this to analyze the
stellar velocity dispersions in a simulation, so it's much more convenient
to have the particle velocities interpolated onto a grid to calculate the
local velocity dispersion using a 3D convolution.
Does anyone have an example of setting up a particle deposition field? If
not, where should I look in the source?
Thanks for your help!
I wanted to write to the list to let you all know that I'm looking at
having a dev workshop at NCSA on October 16 and 17, with overflow onto
the 18th for whoever wants to have a weekend here. This will follow
on the heels of a Blue Waters workshop on the 13-15. I'll be
following up with some more information when I get it, including a
"proper" announcement, but I thought I'd send out something of a
If there's some reason that this definitely works, or definitely
*doesn't* work for you and you really want to go, please let me know
off-list and we'll see what we can work out.
Also, there will be a barbecue at my place. So please, someone remind
me to buy a barbecue. And to learn how to use it. (Maybe I should
file a ticket on bitbucket.)