Revising the skeleton frontend
by Hsi-Yu Schive
Hi all,
I just finish the GAMER frontend for YT, and below I want to summarize some
issues and things that might be confusing in the current skeleton frontend.
I appreciate if you could help validate these issues and/or clarify some
points in the section "Things to be clarified". Hopefully it will make
frontend implementation easier in the future. I'll collect all comments and
issue a PR for the revised skeleton. It will be complementary to the recent
PR #2130 by Jonah Miller, which provides an extremely helpful example
frontend.
*Things to be clarified:*============================
1. Difference between *FieldInfoContainer.known_other_fields* and
*GridIndex._detect_output_field*:
=> known_other_fields contains fields that *might* be in an output, and
lists their units and aliases. _detect_output_fields finds the fields that
are defined in the currently loaded dataset.
2. *Dataset.unique_identifier*:
=> ??
3. Set *AMRGridPatch.Parent = None* for grids without parents, and
*AMRGridPatch.Children
= []* for grids without children
4. Field array returned by *BaseIOHandler._read_fluid_selection* should be
contiguous along the z instead of x direction. Therefore, for C-like array
with the dimension [x][y][z] and for Fortran-like array with the the
dimension (z,y,x), a matrix transpose is required (e.g., using
np_array.transpose() or np_array.swapaxes(0,2))
5. *start_index, stop_index, *and *ActiveDimension*s in the AMRGridPatch
objects
=> ??
(1) It looks like stop_index is not used anymore
(2) ActiveDimensions will be set by AMRGridPatch._prepare_grid. So
perhaps frontend does not need to set it explicitly anymore
(3) It seems that start_index is also calculated automatically
6. *chunk, selector, *and *_read_chunk_data* in io.py.
=> ??
Also the following comment about caching is confusing:
def _read_chunk_data(self, chunk, fields):
# This reads the data from a single chunk, and is only used for caching.
7. float type:
(1) *GridIndex.float_type* is the float type for left and right
simulation edges and must be float64 now
(2) *BaseIOHandler._read_fluid_selection* should always return float64
now even if the on-disk data are stored in single precision
8. Difference between *add_output_field *and *add_field *used in
FieldInfoContainer.setup_fluid_fields:
9. Dataset._parse_parameter_file: what does the following comment mean
"Note that these are all assumed to be in code units; domain_left_edge and
domain_right_edge will be updated to be in code units at a later time"
=> Perhaps it means that "domain_left_edge and domain_right_edge will be
converted to YTArray automatically at a later time"?
*Missing procedure*============================
1. Add the new frontend in *frontends/api.py*
2. Set *GridIndex.max_level*
3. Set *Dataset.refine_by*
*Bugin the current skeleton frontend*
============================
1. data_structures.py: *self.dataset* is not set in GridIndex.__init__ when
calling self.index_filename = self.dataset.parameter_filename
2. data_structures.py: replace self.Parent=[] by *self.Parent = None*
3. fields.py: the *field_list *argument is missing in
FieldInfoContainer.__init__. This issue has been fixed in Jonah's PR.
def __init__(self, ds, field_list):
super(SkeletonFieldInfo, self).__init__(ds, field_list)
Sincerely,
Hsi-Yu (Justin)
2 years, 8 months
Issue #1214: vector annotation of off-axis plot generates inconsistent result (yt_analysis/yt)
by Suoqing Ji
New issue 1214: vector annotation of off-axis plot generates inconsistent result
https://bitbucket.org/yt_analysis/yt/issues/1214/vector-annotation-of-off...
Suoqing Ji:
Test dataset is [here](https://dl.dropboxusercontent.com/u/83265784/data.npy)
Test script:
```
#!python
import yt
import numpy as np
dims = (200,200,200)
x = np.load('data.npy')
field1 = x[:,2].reshape(dims, order="F")
field2 = x[:,1].reshape(dims, order="F")
field3 = x[:,0].reshape(dims, order="F")
data = dict(velocity_x = field1,
velocity_y = field2,
velocity_z = field3)
bbox = np.array([[-100, 99], [-100, 99], [-100, 99]])
ds = yt.load_uniform_grid(data, dims, 'km', bbox=bbox, nprocs=1)
L = [0, -1, 0]
north_vector = [0, 0, 1]
slc1 = yt.OffAxisSlicePlot(ds, L, 'velocity_x', center=[0,0,0], width=(180,'km'), north_vector=north_vector)
slc1.annotate_velocity(factor=40, normalize=True)
slc1.save()
L = [0, 1, 0]
north_vector = [0, 0, 1]
slc2 = yt.SlicePlot(ds, L, 'velocity_x', center=[0,0,0], width=(180,'km'), north_vector=north_vector)
slc2.annotate_velocity(factor=40, normalize=True)
slc2.save()
```
On-axis plot:
Off-axis plot:
2 years, 8 months
Revising the skeleton frontend
by Hsi-Yu Schive
Hi all,
I just finish the GAMER frontend for YT, and below I want to summarize some
issues and things that might be confusing in the current skeleton frontend.
I appreciate if you could help validate these issues and/or clarify some
points in the section "Things to be clarified". Hopefully it will make
frontend implementation easier in the future. I'll collect all comments and
issue a PR for the revised skeleton. It will be complementary to the recent
PR #2130 by Jonah Miller, which provides an extremely helpful example
frontend.
*Things to be clarified:*============================
1. Difference between *FieldInfoContainer.known_other_fields* and
*GridIndex._detect_output_field*:
=> known_other_fields contains fields that *might* be in an output, and
lists their units and aliases. _detect_output_fields finds the fields that
are defined in the currently loaded dataset.
2. *Dataset.unique_identifier*:
=> ??
3. Set *AMRGridPatch.Parent = None* for grids without parents, and
*AMRGridPatch.Children
= []* for grids without children
4. Field array returned by *BaseIOHandler._read_fluid_selection* should be
contiguous along the z instead of x direction. Therefore, for C-like array
with the dimension [x][y][z] and for Fortran-like array with the the
dimension (z,y,x), a matrix transpose is required (e.g., using
np_array.transpose() or np_array.swapaxes(0,2))
5. *start_index, stop_index, *and *ActiveDimension*s in the AMRGridPatch
objects
=> ??
(1) It looks like stop_index is not used anymore
(2) ActiveDimensions will be set by AMRGridPatch._prepare_grid. So
perhaps frontend does not need to set it explicitly anymore
(3) It seems that start_index is also calculated automatically
6. *chunk, selector, *and *_read_chunk_data* in io.py.
=> ??
Also the following comment about caching is confusing:
def _read_chunk_data(self, chunk, fields):
# This reads the data from a single chunk, and is only used for caching.
7. float type:
(1) *GridIndex.float_type* is the float type for left and right
simulation edges and must be float64 now
(2) *BaseIOHandler._read_fluid_selection* should always return float64
now even if the on-disk data are stored in single precision
8. Difference between *add_output_field *and *add_field *used in
FieldInfoContainer.setup_fluid_fields:
9. Dataset._parse_parameter_file: what does the following comment mean:
"Note that these are all assumed to be in code units; domain_left_edge and
domain_right_edge will be updated to be in code units at a later time"?
=> Perhaps it means that "domain_left_edge and domain_right_edge will be
converted to YTArray automatically at a later time"?
*Missing procedure*============================
1. Add the new frontend in *frontends/api.py*
2. Set *GridIndex.max_level*
3. Set *Dataset.refine_by*
*Bugin the current skeleton frontend*
============================
1. data_structures.py: *self.dataset* is not set in GridIndex.__init__ when
calling self.index_filename = self.dataset.parameter_filename
2. data_structures.py: replace self.Parent=[] by *self.Parent = None*
3. fields.py: the *field_list *argument is missing in
FieldInfoContainer.__init__. This issue has been fixed in Jonah's PR.
def __init__(self, ds, field_list):
super(SkeletonFieldInfo, self).__init__(ds, field_list)
Sincerely,
Hsi-Yu (Justin)
2 years, 9 months
Issue #1213: annotate_magnetic_field fails with off-axis slice plot (yt_analysis/yt)
by Suoqing Ji
New issue 1213: annotate_magnetic_field fails with off-axis slice plot
https://bitbucket.org/yt_analysis/yt/issues/1213/annotate_magnetic_field-...
Suoqing Ji:
Test script:
```
#!python
import yt
import numpy as np
nn = 100
data = dict(magnetic_field_x = np.random.random((nn,nn,nn)),
magnetic_field_y = np.random.random((nn,nn,nn)),
magnetic_field_z = np.random.random((nn,nn,nn)))
bbox = np.array([[-1, 1], [-1, 1], [-1, 1]])
ds = yt.load_uniform_grid(data, (nn,nn,nn), length_unit="cm", bbox=bbox, nprocs=1)
L = [1, 1, 0]
north_vector = [0, 0, 1]
slc = yt.SlicePlot(ds, L, 'magnetic_field_x', width=(1 ,'cm'), north_vector=north_vector)
slc.annotate_magnetic_field(factor=40, normalize=True)
slc.save()
```
Error:
```
#!python
Traceback (most recent call last):
File "yt_magnetic_fields.py", line 16, in <module>
slc.save()
File "/Users/jisuoqing/Workspace/code/yt-x86_64/src/yt-hg/yt/visualization/plot_container.py", line 78, in newfunc
args[0]._setup_plots()
File "/Users/jisuoqing/Workspace/code/yt-x86_64/src/yt-hg/yt/visualization/plot_window.py", line 936, in _setup_plots
self.run_callbacks()
File "/Users/jisuoqing/Workspace/code/yt-x86_64/src/yt-hg/yt/visualization/plot_window.py", line 989, in run_callbacks
sys.exc_info()[2])
File "/Users/jisuoqing/Workspace/code/yt-x86_64/src/yt-hg/yt/visualization/plot_window.py", line 985, in run_callbacks
callback(cbw)
File "/Users/jisuoqing/Workspace/code/yt-x86_64/src/yt-hg/yt/visualization/plot_modifications.py", line 316, in __call__
return qcb(plot)
File "/Users/jisuoqing/Workspace/code/yt-x86_64/src/yt-hg/yt/visualization/plot_modifications.py", line 838, in __call__
plot.data[self.field_x],
File "/Users/jisuoqing/Workspace/code/yt-x86_64/src/yt-hg/yt/data_objects/data_containers.py", line 263, in __getitem__
f = self._determine_fields([key])[0]
File "/Users/jisuoqing/Workspace/code/yt-x86_64/src/yt-hg/yt/data_objects/data_containers.py", line 987, in _determine_fields
finfo = self.ds._get_field_info("unknown", fname)
File "/Users/jisuoqing/Workspace/code/yt-x86_64/src/yt-hg/yt/data_objects/static_output.py", line 626, in _get_field_info
raise YTFieldNotFound((ftype, fname), self)
yt.utilities.exceptions.YTPlotCallbackError: annotate_magnetic_field callback failed with the following error: Could not find field '('io', 'cutting_plane_bx')' in UniformGridData.
```
2 years, 9 months
Issue #1212: SPH and Octree AMR Datasets Failing in volume rendering, OAProj, Interactive Data Viz (yt_analysis/yt)
by Cameron Hummels
New issue 1212: SPH and Octree AMR Datasets Failing in volume rendering, OAProj, Interactive Data Viz
https://bitbucket.org/yt_analysis/yt/issues/1212/sph-and-octree-amr-datas...
Cameron Hummels:
This similar to Issues #788, #986, #1008, #1183, but I think this description generalizes and distills all of these problems.
It appears that all particle-based (or Octree-based) datasets are segfaulting when one attempts to do anything involving the KDTree with them. The following three scripts all fail with the same result:
```
#!python
import yt
ds = yt.load("gizmo_cosmology_plus/snap_N128L16_151.hdf5")
L = ds.arr([1,0,0], 'unitary')
yt.OffAxisProjectionPlot(ds, L, 'density').save()
```
```
#!python
import yt
ds = yt.load("gizmo_cosmology_plus/snap_N128L16_151.hdf5")
im, sc = yt.volume_render(ds, 'density', fname='rendering.png')
```
```
#!python
import yt
ds = yt.load("gizmo_cosmology_plus/snap_N128L16_151.hdf5")
yt.interactive_render(ds)
```
These all fail with some variation on this output:
```
#!python
Failed to split grids.
Failed to split grids.
Segmentation fault
```
The above scripts each use the publicly available dataset `gizmo_cosmology_plus`, but one gets similar results by using the same simple commands on any of the SPH datasets on http://yt-project.org/data/ e.g. `TipsyGalaxy`, `snapshot_033`, `GadgetDiskGalaxy`.
Notably, however, when you run the same script with Octree-based AMR outputs, you get similar results. Example datasets that do this are Ramses (e.g. `output_00080`), and ART (e.g. `D9p_500`).
These scripts were run on an OS X installation with the tip of yt-dev (843a342ee510).
2 years, 9 months
vertex-centered data
by Jonah Miller
Hi yt-dev,
Nathan Goldbaum and I were discussing the possibility of handling
vertex-centered data natively in yt. We think that roughly this is quite
do-able and that most of the pieces are already in place. Nathan
suggested I summarize our discussion and email it to all of you to ask
for your opinions.
Internally, all that's required is using the generated ghost cells to
convert between cell- and vertex-centered data. (For non-periodic
boundaries, linear extrapolation, which would need to be implemented,
should be good enough.)
In the userspace side, we need a concept for this type of data so that
it can be extracted and manipulated. One option would be to make
centering an attribute for datasets. It could be immutable or mutable,
depending on design choices. In both cases, we would create a method to
go between centerings:
ds.to_centering(centering_type)
in the former case, this method would return a new dataset. In the
latter case, it would reset the dataset.
In principle, centering is a metadata property. The input data can be
interpolated when it's explored and not before. So for this reason, it
makes some sense to leave the centering as a dataset property rather
than as, say, a field property.
I think that's about it. Nathan can correct me if I forgot something.
Thanks!
Best,
Jonah Miller
2 years, 9 months
Issue #1211: HDF5FileHandler is missing close() (yt_analysis/yt)
by Kacper Kowalik
New issue 1211: HDF5FileHandler is missing close()
https://bitbucket.org/yt_analysis/yt/issues/1211/hdf5filehandler-is-missi...
Kacper Kowalik:
```
/home/yt/WindTunnel $ yt search
( 0 candidates) Examining .
( 0/ 1) Evaluating ./windtunnel_4lev_hdf5_plt_cnt_0030
yt : [INFO ] 2016-04-18 13:54:14,762 Parameters: current_time = 3.00091455609
yt : [INFO ] 2016-04-18 13:54:14,762 Parameters: domain_dimensions = [120 40 1]
yt : [INFO ] 2016-04-18 13:54:14,763 Parameters: domain_left_edge = [ 0. 0. 0.]
yt : [INFO ] 2016-04-18 13:54:14,763 Parameters: domain_right_edge = [ 3. 1. 1.]
yt : [INFO ] 2016-04-18 13:54:14,764 Parameters: cosmological_simulation = 0.0
Traceback (most recent call last):
File "/home/xarth/local/bin/yt", line 9, in <module>
load_entry_point('yt', 'console_scripts', 'yt')()
File "/home/xarth/codes/yt-my/yt/utilities/command_line.py", line 1193, in run_main
args.func(args)
File "/home/xarth/codes/yt-my/yt/utilities/command_line.py", line 115, in run
self(args)
File "/home/xarth/codes/yt-my/yt/utilities/command_line.py", line 1176, in __call__
ds.close()
File "/home/xarth/codes/yt-my/yt/frontends/flash/data_structures.py", line 425, in close
self._handle.close()
AttributeError: 'HDF5FileHandler' object has no attribute 'close'
```
Version:
```
Version = 3.3-dev
Changeset = b10ac4289001 (yt) tip issue_1206
```
Responsible: xarthisius
2 years, 9 months
