Mailman 3 May 2016 - yt-dev

Issue #1219: Shader files are not installed correctly (yt_analysis/yt)
by Kacper Kowalik 18 May '16

18 May '16

New issue 1219: Shader files are not installed correctly https://bitbucket.org/yt_analysis/yt/issues/1219/shader-files-are-not-insta… Kacper Kowalik: As reported by @a_gilbert ``` copying yt/visualization/volume_rendering/shaders/default.vertexshader -> build/bdist.linux-x86_64/egg warning: install_data: setup script did not provide a directory for 'yt/visualization/volume_rendering/shaders/passthrough.vertexshader' -- installing right in 'build/bdist.linux-x86_64/egg' ``` shaders are installed into the main egg directory instead of `yt/visualization/volume_rendering/shaders` Responsible: xarthisius

1 0

Issue #1218: How to Mock X-ray Observations with Gadget gas particle outputs? (yt_analysis/yt)
by Weiguang Cui 13 May '16

13 May '16

New issue 1218: How to Mock X-ray Observations with Gadget gas particle outputs? https://bitbucket.org/yt_analysis/yt/issues/1218/how-to-mock-x-ray-observat… Weiguang Cui: Dear, I have checked the available examples online, all of them used the mesh outputs. Can I directly use the gas particle (instead of smoothing them into cells) in the photon simulator? Since Veronica's PHOX is originally used for gadget output, it would be nice to apply this X-ray simulator to gadget output directly. Thanks a lot for your help. Kind regards, Weiguang

1 0

New project members: Bili Dong and Brian Crosby
by Nathan Goldbaum 12 May '16

12 May '16

Hi all, I'd like to nominate two new project members: Bili Dong and Brian Crosby. Both of them have made substantial contributions to the yt codebase: https://bitbucket.org/yt_analysis/yt/commits/all?search=author(qobilidop) https://bitbucket.org/yt_analysis/yt/commits/all?search=author(bcrosby) If you don't know or remember, yt project members are given official recognition on the website and are also given commit bits for all yt repositories on bitbucket. For more details, see YTEP-1776: http://ytep.readthedocs.io/en/latest/YTEPs/YTEP-1776.html We need three additional members to approve the nominations before the new members get added. -Nathan

5 4

PR review hangout: Wednesday May 11, 1:00 PM CDT
by Nathan Goldbaum 11 May '16

11 May '16

Hi all, Here's the link for this week's PR review hangout: https://plus.google.com/b/107728486871834552760/events/cc5o4fqfnp3cg96qb8vh… -Nathan

1 1

Issue #1217: Slicing semi structured grid fails (yt_analysis/yt)
by Kacper Kowalik 11 May '16

11 May '16

New issue 1217: Slicing semi structured grid fails https://bitbucket.org/yt_analysis/yt/issues/1217/slicing-semi-structured-gr… Kacper Kowalik: Following script: ``` #!python import yt import numpy as np xgrid = np.array([-1, -0.65, 0, 0.65, 1]) ygrid = np.array([-1, 0, 1]) zgrid = np.array([-1, -0.447, 0.447, 1]) coords, conn = yt.hexahedral_connectivity(xgrid, ygrid, zgrid) arr = np.ones((len(xgrid)-1, len(ygrid)-1, len(zgrid)-1)) bbox = np.array([[np.min(xgrid),np.max(xgrid)], [np.min(ygrid),np.max(ygrid)], [np.min(zgrid),np.max(zgrid)]]) data = {"density" : arr} ds = yt.load_hexahedral_mesh(data, conn, coords, 1.0, bbox=bbox) yt.SlicePlot(ds, "y", "density") ``` yields ``` Traceback (most recent call last): File "ala.py", line 17, in <module> yt.SlicePlot(ds, "y", "density") File "/home/xarth/codes/yt-my/yt/visualization/plot_window.py", line 1867, in SlicePlot return AxisAlignedSlicePlot(ds, normal, fields, *args, **kwargs) File "/home/xarth/codes/yt-my/yt/visualization/plot_window.py", line 1240, in __init__ slc.get_data(fields) File "/home/xarth/codes/yt-my/yt/data_objects/data_containers.py", line 1172, in get_data fluids, self, self._current_chunk) File "/home/xarth/codes/yt-my/yt/geometry/geometry_handler.py", line 245, in _read_fluid_fields chunk_size) File "/home/xarth/codes/yt-my/yt/frontends/stream/io.py", line 229, in _read_fluid_selection ind += g.select(selector, ds, rv[field], ind) # caches File "/home/xarth/codes/yt-my/yt/data_objects/unstructured_mesh.py", line 142, in select dest[offset:offset+count] = source[mask, ...] IndexError: index 4 is out of bounds for axis 0 with size 4 ``` Regression was introduced by c697901a , specifically: ``` --- a/yt/data_objects/unstructured_mesh.py +++ b/yt/data_objects/unstructured_mesh.py @@ -127,7 +138,7 @@ mask = self._get_selector_mask(selector) count = self.count(selector) if count == 0: return 0 - dest[offset:offset+count] = source.flat[mask] + dest[offset:offset+count] = source[mask,...] return count def count(self, selector): ```

1 0

(no subject)
by mika harisetry Rafieferantsoa 11 May '16

11 May '16

Hi there, I've been recently using yt. The following code spit an error. Can anyone please help? In [*2*]: import yt In [*3*]: ds = yt.load('/data/snap_008.hdf5') ... In [*5*]: ds.camera() *---------------------------------------------------------------------------* *AttributeError* Traceback (most recent call last) *<ipython-input-5-14e99c8871cc>* in <module>*()* *----> 1* ds*.*camera*()* *AttributeError*: 'GadgetHDF5Dataset' object has no attribute 'camera'

1 0

Particle phase plotting
by John ZuHone 07 May '16

07 May '16

Hi all, I had a question about how we do particle phase plots / 2d profiles. I’ve noticed that we way we do this involves some kind of deposition of the particle data into a grid. This necessitates never allowing one to make log-spaced bins for a particle 2d profile, though you can of course plot them in logspace. This has rather nasty consequences, however, for data which has a huge dynamic range over orders of magnitude and for which log-spaced bins would make a lot of sense. See, for example, this plot: http://i.imgur.com/RvCp3uD.png <http://i.imgur.com/RvCp3uD.png> where a lot of the interesting data is at low density but has simply been swallowed up into the first bin in density space because of the linear spacing, making the plot essentially useless. Note this plot was made with a ridiculous amount of bins, 16000x16000, and it still washes things out at low density too much. So I guess my questions are: 1. Why do we need to use deposition instead of simply binning up the particle data? Isn’t this what we do for grid-spaced data? 2. Is there any reason why we cannot use log-spaced bins with deposition (even if we have to hack it somehow)? Best, John

2 1

[ANN] Introducing the yt Hub
by Kacper Kowalik 06 May '16

06 May '16

Hi Everyone! The yt community is proud to announce the release of yt Hub 1.0. The yt Hub (https://hub.yt) is an environment where data can be published, analyzed and shared with other people. The yt Hub relies on multiple open source microservices that are loosely coupled together. At its core we utilize: * Girder - as a data management platform (https://girder.hub.yt) * Jupyter Notebook - for interactive data exploration * Curldrop - for a quick and easy data sharing * MediaGoblin - for a quick and easy image sharing We encourage you to view this short video: https://www.youtube.com/watch?v=Uq_3uT7JHGA to see how we integrated Girder with Jupyter Notebook or read https://docs.hub.yt/girder.html Each authenticated user (Google, Bitbucket, or GitHub) is provided with 10GB of free space that can be used for uploading data sets and sharing them with other people. We plan to gradually increase the storage capacity of the Hub. Additionally, we're developing a process for sharing of large, community-provided data sets. If your data requirements exceed the current limit, please don't hesitate to contact us directly so that we can develop a plan to integrate your data in an efficient way. Over time we plan to expand the capabilities of the existing services and add new ones. A short set of documentation describing the current services and what you can do with them can be found at https://docs.hub.yt/ As always, if you have any questions, concerns, or run into any trouble please don't hesitate to send a message to the mailing list or stop by our IRC channel. We thank the National Science Foundation (ACI-1535651) and the Gordon and Betty Moore Foundation (GBMF4561) for their support in developing and deploying this service. Thank you, Kacper Kowalik on behalf of the yt development team

2 4

A set of benchmarks
by Jonah Miller 06 May '16

06 May '16

Hi yt-dev, I am developing a frontend for Einstein Toolkit for yt and in the process I generated some crude, preliminary benchmarks which I thought I would share in case anybody is interested. I performed three tests: 1. I just load the dataset and calculate (say) the maximum of some quantity on each grid. 2. I load in the dataset and calculate the maximum of the magnitude of a gradient on each grid. This requires the generation of ghost zones at grid boundaries. 3. I load in the dataset and perform a volume rendering. I make a "movie" with 4 frames where I rotate around the volume. Salient details: * I performed these tests with the attached scripts. I turned openmp and MPI off (set OMP_NUM_THREADS=1). * The domain sizes were: 64^3, 128^3, 256^3, and 512^3. * All four datasets have only one refinement level. However the 512^3 dataset has multiple grids on that refinement level. * The size of a dataset ranges from about 10MB for the 64^3 set to 10GB for the 512^3 * All data was performed on a single modern workstation. Details: 2.6GHz clock with AVX2 instructions and a 60MB L3 cache. Results: * Reading in the data is (for the datasets I tested) extremely fast. The 512^3 dataset takes only about 20 seconds to read the whole thing in. * Generating the first frame in a volume render is extremely slow. On the order of 4 minutes for the 512^3 data set. After the first frame is produced, new frames are fast, even with openmp off. With 1 openmp thread, it takes on the order of 10s of seconds for a new frame. * Generating ghost zones is very fast for datasets with only one grid. It is incredibly slow for datasets with multiple grids, dominating the run time. * I attach a plot comparing the three tests. Naively, it seems to me that there must be several sources of overhead when I perform volume rendering that are significantly more costly than simply reading in the data (which seems to be fast). Clearly one of these is the generation of the ghost zones, and another is the actual ray tracing (although the ray tracing itself seems to be quite fast.). However I'm not sure that these two operations alone explain the cost of the volume rendering. I'd love to know other people's experience with benchmarking. Do these costs seem normal, up to an order of magnitude or so? Would you have any insight into what contributes to the cost of generating the first frame when volume rendering? Thanks very much! Best, Jonah Miller

2 2

Particle phase plotting
by John Zuhone 06 May '16

06 May '16

Hi all, I had a question about how we do particle phase plots / 2d profiles. I’ve noticed that we way we do this involves some kind of deposition of the particle data into a grid. This necessitates never allowing one to make log-spaced bins for a particle 2d profile, though you can of course plot them in logspace. This has rather nasty consequences, however, for data which has a huge dynamic range over orders of magnitude and for which log-spaced bins would make a lot of sense. See, for example, this plot: http://i.imgur.com/RvCp3uD.png <http://i.imgur.com/RvCp3uD.png> where a lot of the interesting data is at low density but has simply been swallowed up into the first bin in density space because of the linear spacing, making the plot essentially useless. Note this plot was made with a ridiculous amount of bins, 16000x16000, and it still washes things out at low density too much. So I guess my questions are: 1. Why do we need to use deposition instead of simply binning up the particle data? Isn’t this what we do for grid-spaced data? 2. Is there any reason why we cannot use log-spaced bins with deposition (even if we have to hack it somehow)? Best, John

2 1