Mailman 3 September 2016 - yt-dev

Issue #1287: New camera position cannot be aligned with north_vector that was used in previous invocation of set_position (yt_analysis/yt)
by Alexander Lindsay 30 Sep '16

30 Sep '16

New issue 1287: New camera position cannot be aligned with north_vector that was used in previous invocation of set_position https://bitbucket.org/yt_analysis/yt/issues/1287/new-camera-position-cannot… Alexander Lindsay: Load any data set. The following script will produce an error: sc = yt.create_scene(ds) cam = sc.camera cam.focus = [0, 0, 0] cam_pos = [1, 0, 0] north_vector = [0, 1, 0] cam.set_position(cam_pos, north_vector) cam_pos = [0, 1, 0] north_vector = [0, 0, 1] cam.set_position(cam_pos, north_vector) Clearly, the user is specifying cam_pos and north_vector that are **not** aligned. However, in the current code set-up, position's fget() passes north_vector=None to `switch_orientation`. Because `north_vector` is None in `switch_orientation`, the old value of north_vector is passed to `_setup_normalized_vectors` which then errors out because the new camera position-focus vector is aligned with the old north_vector. This issue will only arise in rare cases; however, it took me a very perplexed hour (and learning about python properties :-)) to figure out what was going on.

1 0

Request review for two pull requests
by Nathan Goldbaum 30 Sep '16

30 Sep '16

Hi all, Matt has had two of his pull requests in the queue for a while. I think both are very worthwhile changes that will unblock other work, so I'd like to request some more pairs of eyes on them: https://bitbucket.org/yt_analysis/yt/pull-requests/2257 https://bitbucket.org/yt_analysis/yt/pull-requests/2245 The first implements boolean objects, an oft-requested feature from yt 2.x. The second makes it so our pixelization routines produce images via compositing onto an existing image rather than returning an image which does not exist yet. It also fixes some long-standing issues with the orientation of images returned by the pixelizer. The latter in particular will unblock work on supporting datasets that contain multiple disjoint but continuous meshes - something that's common in numerical relativity data. -Nathan

1 0

Issue #1286: Issues with BibTeX citation (yt_analysis/yt)
by Aaron Meurer 28 Sep '16

28 Sep '16

New issue 1286: Issues with BibTeX citation https://bitbucket.org/yt_analysis/yt/issues/1286/issues-with-bibtex-citation Aaron Meurer: At http://yt-project.org/doc/about/index.html#how-do-i-cite-yt, there are two issues: - It uses \apjs, but this is not defined unless you use certain cls files. - The pages are listed as `9-+`. Google Scholar gives just `9`. Is the `-+` a typo?

1 0

Issue #1285: Yt loglevel: Many methods, why only one working? (yt_analysis/yt)
by scott_feister 28 Sep '16

28 Sep '16

New issue 1285: Yt loglevel: Many methods, why only one working? https://bitbucket.org/yt_analysis/yt/issues/1285/yt-loglevel-many-methods-w… scott_feister: There are several methods noted in the yt documentation that one can change the log level. The duplicated documentation is a problem for another day (that I may attempt to address, even), but not the subject of this bug report. Here is the relevant documentation: http://yt-project.org/doc/reference/configuration.html http://yt-project.org/doc/faq/index.html#how-can-i-change-yt-s-log-level And here are the methods: Method 1. From text editor: edit $HOME/.yt file to read: ``` [yt] loglevel = 50 ``` Method 2. From command line, call your script "my_script.py" that includes yt analysis: ``` $ python2.7 my_script.py --config loglevel=50 ``` Method 3. Within python: ``` >> yt.funcs.mylog.setLevel(50) ``` Method 4. Within python: ``` >> yt.config.ytcfg[“yt”, “loglevel”] = “50” ``` Method 5. Within python: ``` >> yt.config.ytcfg.set(“yt”, “loglevel”, “50”) ``` Method 6. Within python: ``` >> from yt.config import ytcfg >> ytcfg[ …. # (variant on methods 4 and 5) ``` While trying to change the log level, I learned about and tried them all, but only method 4 worked. The others leave the "[INFO]" messages on. Scott

1 0

Issue #1284: YTInvalidShaderType: Can't identify shader_type for file 'default.vertexshader.' (yt_analysis/yt)
by honghe 28 Sep '16

28 Sep '16

New issue 1284: YTInvalidShaderType: Can't identify shader_type for file 'default.vertexshader.' https://bitbucket.org/yt_analysis/yt/issues/1284/ytinvalidshadertype-cant-i… honghe: Follow the following guide to Interactive Data Visualization http://yt-project.org/doc/visualizing/interactive_data_visualization.html#i… The code is: ``` import yt ds = yt.load("IsolatedGalaxy/galaxy0030/galaxy0030") sp = ds.sphere("max", (0.1, "Mpc")) cam_pos = ds.arr([0.1, 0.1, 0.1], "Mpc").in_units("code_length") yt.interactive_render(sp, field="pressure", cam_position=cam_pos, window_size=(512, 512)) ``` But get error: ``` yt : [INFO ] 2016-09-28 15:00:15,483 Parameters: current_time = 0.00600002000283 yt : [INFO ] 2016-09-28 15:00:15,484 Parameters: domain_dimensions = [32 32 32] yt : [INFO ] 2016-09-28 15:00:15,485 Parameters: domain_left_edge = [ 0. 0. 0.] yt : [INFO ] 2016-09-28 15:00:15,485 Parameters: domain_right_edge = [ 1. 1. 1.] yt : [INFO ] 2016-09-28 15:00:15,486 Parameters: cosmological_simulation = 0.0 Parsing Hierarchy : 100%|██████████| 173/173 [00:00<00:00, 56131.71it/s] yt : [INFO ] 2016-09-28 15:00:15,505 Gathering a field list (this may take a moment.) yt : [INFO ] 2016-09-28 15:00:16,718 Max Value is 7.73427e-24 at 0.5040893554687500 0.4998168945312500 0.5001831054687500 Loading textures. Starting rendering... --------------------------------------------------------------------------- YTInvalidShaderType Traceback (most recent call last) <ipython-input-1-935c3c7fb8ea> in <module>() 6 cam_pos = ds.arr([0.1, 0.1, 0.1], "Mpc").in_units("code_length") 7 yt.interactive_render(sp, field="pressure", cam_position=cam_pos, ----> 8 window_size=(512, 512)) /home/honhe/Envs/yt/local/lib/python2.7/site-packages/yt/visualization/volume_rendering/interactive_vr_helpers.pyc in _render_opengl(data_source, field, window_size, cam_position, cam_focus) 100 c = TrackballCamera(position=cam_position, focus=cam_focus, near_plane=near_plane, 101 far_plane=far_plane, aspect_ratio=aspect_ratio) --> 102 rc.start_loop(scene, c) 103 104 /home/honhe/Envs/yt/local/lib/python2.7/site-packages/yt/visualization/volume_rendering/interactive_loop.pyc in start_loop(self, scene, camera) 182 def start_loop(self, scene, camera): 183 callbacks = self.setup_loop(scene, camera) --> 184 for i in self(scene, camera, callbacks): 185 pass 186 /home/honhe/Envs/yt/local/lib/python2.7/site-packages/yt/visualization/volume_rendering/interactive_loop.pyc in __call__(self, scene, camera, callbacks) 191 camera.compute_matrices() 192 scene.set_camera(camera) --> 193 scene.render() 194 glfw.SwapBuffers(self.window) 195 callbacks.draw = False /home/honhe/Envs/yt/local/lib/python2.7/site-packages/yt/visualization/volume_rendering/interactive_vr.pyc in run_program(self) 878 # render collections to fb 879 for collection in self.collections: --> 880 collection.run_program() 881 # unbind FB 882 GL.glBindFramebuffer(GL.GL_FRAMEBUFFER, 0) /home/honhe/Envs/yt/local/lib/python2.7/site-packages/yt/visualization/volume_rendering/interactive_vr.pyc in run_program(self) 291 292 def run_program(self): --> 293 with self.program.enable(): 294 if len(self.vert_arrays) != 1: 295 raise NotImplementedError /home/honhe/Envs/yt/local/lib/python2.7/site-packages/yt/visualization/volume_rendering/interactive_vr.pyc in program(self) 281 self._program.delete_program() 282 self._program = ShaderProgram(self.vertex_shader, --> 283 self.fragment_shader) 284 self._program_invalid = False 285 return self._program /home/honhe/Envs/yt/local/lib/python2.7/site-packages/yt/visualization/volume_rendering/shader_objects.pyc in __init__(self, vertex_shader, fragment_shader) 47 pass 48 elif None not in (vertex_shader, fragment_shader): ---> 49 self.link(vertex_shader, fragment_shader) 50 else: 51 raise RuntimeError /home/honhe/Envs/yt/local/lib/python2.7/site-packages/yt/visualization/volume_rendering/shader_objects.pyc in link(self, vertex_shader, fragment_shader) 61 self.vertex_shader = vertex_shader 62 self.fragment_shader = fragment_shader ---> 63 GL.glAttachShader(self.program, vertex_shader.shader) 64 GL.glAttachShader(self.program, fragment_shader.shader) 65 GL.glLinkProgram(self.program) /home/honhe/Envs/yt/local/lib/python2.7/site-packages/yt/visualization/volume_rendering/shader_objects.pyc in shader(self) 202 def shader(self): 203 if self._shader is None: --> 204 self.compile() 205 return self._shader 206 /home/honhe/Envs/yt/local/lib/python2.7/site-packages/yt/visualization/volume_rendering/shader_objects.pyc in compile(self, source, parameters) 186 if parameters is not None: 187 raise NotImplementedError --> 188 source = self._get_source(source) 189 shader_type_enum = getattr(GL, 190 'GL_%s_SHADER' % self.shader_type.upper()) /home/honhe/Envs/yt/local/lib/python2.7/site-packages/yt/visualization/volume_rendering/shader_objects.pyc in _get_source(self, source) 177 fn = os.path.join(sh_directory, source) 178 if not os.path.isfile(fn): --> 179 raise YTInvalidShaderType(source) 180 return open(fn, 'r').read() 181 YTInvalidShaderType: Can't identify shader_type for file 'default.vertexshader.' ``` ## Environment: ➜ ~ uname -a Linux honhe 4.4.0-38-generic #57~14.04.1-Ubuntu SMP Tue Sep 6 17:20:43 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux GPU: Nvidia Geforce 950 GLFW: 3.1.2 pip list: ``` backports-abc (0.4) backports.shutil-get-terminal-size (1.0.0) backports.ssl-match-hostname (3.5.0.1) certifi (2016.9.26) configparser (3.5.0) cycler (0.10.0) cyglfw3 (3.1.0.2) Cython (0.24.1) decorator (4.0.10) entrypoints (0.2.2) enum34 (1.1.6) functools32 (3.2.3.post2) h5py (2.6.0) ipykernel (4.5.0) ipython (5.1.0) ipython-genutils (0.1.0) ipywidgets (5.2.2) Jinja2 (2.8) jsonschema (2.5.1) jupyter (1.0.0) jupyter-client (4.4.0) jupyter-console (5.0.0) jupyter-core (4.2.0) MarkupSafe (0.23) matplotlib (1.5.3) mistune (0.7.3) mpmath (0.19) nbconvert (4.2.0) nbformat (4.1.0) notebook (4.2.3) numpy (1.11.1) pathlib2 (2.1.0) pexpect (4.2.1) pickleshare (0.7.4) pip (8.1.2) prompt-toolkit (1.0.7) ptyprocess (0.5.1) Pygments (2.1.3) PyOpenGL (3.1.0) pyparsing (2.1.9) python-dateutil (2.5.3) pytz (2016.6.1) pyzmq (15.4.0) qtconsole (4.2.1) setuptools (27.3.0) simplegeneric (0.8.1) singledispatch (3.4.0.3) six (1.10.0) sympy (1.0) terminado (0.6) tornado (4.4.1) traitlets (4.3.0) wcwidth (0.1.7) wheel (0.30.0a0) widgetsnbextension (1.2.6) yt (3.3.1) ``` Is any thing I did incorrectly?

1 0

Proposed change to how to run the unit tests
by Nathan Goldbaum 27 Sep '16

27 Sep '16

Hi all, I just opened a pull request that makes a minor change to how you run the yt unit tests in some cases: https://bitbucket.org/yt_analysis/yt/pull-requests/2395 See the pull request description for more details. I'm raising this on the mailing list because it will impact how you run the unit tests if you specify a path to a test file. I would very much appreciate hearing from anyone that has concerns about this. Just raising it here out of an abundance of caution, I expect everyone to be pretty happy with this change as it's a common source of user error when running tests locally. -Nathan

2 1

demeshening sprint summary, plans going forward
by Nathan Goldbaum 26 Sep '16

26 Sep '16

Hi all, tl;dr: We are working on improving support and performance for particle data in yt. We want feedback on our ideas and ask for people uninvolved in this effort to do code reviews and participate in design discussions and workflow discussions going forward Last week we had a sprint here at NCSA around the "demeshening" idea a few of us have been throwing around. The basic idea here is to incorporate the code Meagan Lang has been working on over the past year into yt, allowing improved performance for particle codes. One consequence of this work is that there will no longer be a space-filling octree mesh for SPH data, instead the data will be indexed by local mesh patches built on compressed EWAH bitmap indices As part of this sprint, we had lots of discussions about how this work might proceed, and people also did some work on moving several aspects of this effort forward: I started work on an SPH pixelizer we can use for SlicePlot and ProjectionPlot: https://bitbucket.org/yt_analysis/yt/pull-requests/2382 Bili Dong started working on a ray data object that can be used to generate absorption spectra from SPH data without depositing data onto an octree mesh: https://bitbucket.org/yt_analysis/yt/pull-requests/2386 And Matt Turk and Alex Lindsay worked on improvements for the field system: https://bitbucket.org/yt_analysis/yt/pull-requests/2383 https://bitbucket.org/yt_analysis/yt/pull-requests/2385 Alex's pull request also received some love today and I'd like to request review on it, as it will unblock further work on the pixelizers. In particular, I'd like to build on top of Alex's pull request to add a new kind of sampling_type, "local" which correspond to fully local fields. This generalizes the current default situation (i.e. the field does not have a ValidateSpatial validator). My ultimate goal here is to make it easier to support SPH particle fields. For fields that are fully local, we don't need to create different implementations for the 'cell' and 'particle' samping_type, instead we can just use the same implementation for all field types. Once that is in, I think I'll be able to make it so projections and slices work with the majority of yt derived fields using the new SPH pixelizers I've added in PR 2382: https://bitbucket.org/yt_analysis/yt/pull-requests/2382 Unfortunately PR 2382 won't be mergeable at that point because we also need to come up with a solution for fields that require finite differences (this is discussed a bit in one of Daniel Price's papers, starting around section 3.3.2: http://arxiv.org/pdf/1012.1885v1.pdf) I think once we've agreed on the way SPH particle fields should work going forward (I'm currently drafting a YTEP for this) then the demeshening work can build on top of that in a piecemeal fashion. If we can I'd like it if we try to upstream stuff early and often to make this work easier to review and to improve testing by users and other developers. I don't want to move development into another fork of yt or a new named branch, since we had issues with that in the past. This approach might add some work we wouldn't have otherwise done (i.e. we might need to add compatibility layers that will eventually be torn down), but it will help keep us honest and ensure we're getting reviews from people uninvolved in this effort. That said, if you'd like to try out the new code or even chip in, I'm happy to pull changes from people's forks or to review pull requests against my fork of yt. I think that's it, sorry for the novel! I'd very much appreciate comments or questions about this. -Nathan

1 0

Issue #1283: 1D Exodus II support (yt_analysis/yt)
by Andrew Myers 22 Sep '16

22 Sep '16

New issue 1283: 1D Exodus II support https://bitbucket.org/yt_analysis/yt/issues/1283/1d-exodus-ii-support Andrew Myers: Right now, the Exodus II frontend can read 2D and 3D datasets, but not 1D. Adding support for 1D should not be too hard. Responsible: atmyers

1 0

Issue #1282: Docs on loading generic unstructured mesh data need uploading (yt_analysis/yt)
by Nathan Goldbaum 22 Sep '16

22 Sep '16

New issue 1282: Docs on loading generic unstructured mesh data need uploading https://bitbucket.org/yt_analysis/yt/issues/1282/docs-on-loading-generic-un… Nathan Goldbaum: Here's the page: http://yt-project.org/docs/dev/examining/loading_data.html#unstructured-gri… Right now this loads an Exodus-II dataset using a custom parser. This example predates the Exodus-II frontend, and given the existence of the Exodus-II frontend is somewhat confusing for new users with Exodus-II data. We should probably deprecate `yt.utilities.exodusII_reader` in favor of the Exodus II frontend and update the docs section on loading generic unstructured mesh data to use data defined inline in the script. Responsible: ngoldbaum

1 0

Issue #1281: YTQuadTreeProj contains many unusable methods (yt_analysis/yt)
by scott_feister 21 Sep '16

21 Sep '16

New issue 1281: YTQuadTreeProj contains many unusable methods https://bitbucket.org/yt_analysis/yt/issues/1281/ytquadtreeproj-contains-ma… scott_feister: Unless I am understanding something wrong, there are several non-functioning methods listed in YTQuadTreeProj. The [auto-generated documentation](yt-project.org/doc/reference/api/generated/yt.data_objects.c… lists methods like "min()", "max()", "mean()", "integrate()", etc. These methods are also available to the python interface, but almost all of them just throw errors at me that say "Data selector... not implemented". This is some combination of: 1. A bug 2. An oversight in documentation 3. An issue with method inheritance for YTQuadTreeProj OR I just messed something up. ``` #!python import yt exampledir = "/home/sfeister/test/yttest/ytExampleData/" ds = yt.load(exampledir + "Enzo_64/DD0043/data0043") reg = ds.all_data() print reg.min('Density') # Works proj = reg.integrate('Density', 2) # Do a projection of the region print(proj.__doc__) # Methods of YTQuadTree include "integrate", "min", etc. print("/n/n") print proj.min('Density') # Fails, with error below print proj.integrate('Density', 1) # Fails, similar error ``` Error is as follows: ``` Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/home/sfeister/anaconda2/lib/python2.7/site-packages/yt/data_objects/data_containers.py", line 778, in min rv += (self._compute_extrema(f)[0],) File "/home/sfeister/anaconda2/lib/python2.7/site-packages/yt/data_objects/data_containers.py", line 706, in _compute_extrema mi, ma = self.quantities.extrema(field) File "/home/sfeister/anaconda2/lib/python2.7/site-packages/yt/data_objects/derived_quantities.py", line 510, in __call__ rv = super(Extrema, self).__call__(fields, non_zero) File "/home/sfeister/anaconda2/lib/python2.7/site-packages/yt/data_objects/derived_quantities.py", line 66, in __call__ for sto, ds in parallel_objects(chunks, -1, storage = storage): File "/home/sfeister/anaconda2/lib/python2.7/site-packages/yt/utilities/parallel_tools/parallel_analysis_interface.py", line 508, in parallel_objects for result_id, obj in oiter: File "/home/sfeister/anaconda2/lib/python2.7/site-packages/yt/data_objects/data_containers.py", line 1087, in chunks for chunk in self.index._chunk(self, chunking_style, **kwargs): File "/home/sfeister/anaconda2/lib/python2.7/site-packages/yt/geometry/geometry_handler.py", line 251, in _chunk self._identify_base_chunk(dobj) File "/home/sfeister/anaconda2/lib/python2.7/site-packages/yt/geometry/grid_geometry_handler.py", line 293, in _identify_base_chunk gi = dobj.selector.select_grids(self.grid_left_edge, File "/home/sfeister/anaconda2/lib/python2.7/site-packages/yt/data_objects/data_containers.py", line 1075, in selector raise YTDataSelectorNotImplemented(self._type_name) yt.utilities.exceptions.YTDataSelectorNotImplemented: Data selector 'proj' not implemented. ``` Best, Scott

1 0