Mailman 3 May 2016 - yt-dev

Revising the skeleton frontend
by Hsi-Yu Schive 06 May '16

06 May '16

Hi all, I just finish the GAMER frontend for YT, and below I want to summarize some issues and things that might be confusing in the current skeleton frontend. I appreciate if you could help validate these issues and/or clarify some points in the section "Things to be clarified". Hopefully it will make frontend implementation easier in the future. I'll collect all comments and issue a PR for the revised skeleton. It will be complementary to the recent PR #2130 by Jonah Miller, which provides an extremely helpful example frontend. *Things to be clarified:*============================ 1. Difference between *FieldInfoContainer.known_other_fields* and *GridIndex._detect_output_field*: => known_other_fields contains fields that *might* be in an output, and lists their units and aliases. _detect_output_fields finds the fields that are defined in the currently loaded dataset. 2. *Dataset.unique_identifier*: => ?? 3. Set *AMRGridPatch.Parent = None* for grids without parents, and *AMRGridPatch.Children = []* for grids without children 4. Field array returned by *BaseIOHandler._read_fluid_selection* should be contiguous along the z instead of x direction. Therefore, for C-like array with the dimension [x][y][z] and for Fortran-like array with the the dimension (z,y,x), a matrix transpose is required (e.g., using np_array.transpose() or np_array.swapaxes(0,2)) 5. *start_index, stop_index, *and *ActiveDimension*s in the AMRGridPatch objects => ?? (1) It looks like stop_index is not used anymore (2) ActiveDimensions will be set by AMRGridPatch._prepare_grid. So perhaps frontend does not need to set it explicitly anymore (3) It seems that start_index is also calculated automatically 6. *chunk, selector, *and *_read_chunk_data* in io.py. => ?? Also the following comment about caching is confusing: def _read_chunk_data(self, chunk, fields): # This reads the data from a single chunk, and is only used for caching. 7. float type: (1) *GridIndex.float_type* is the float type for left and right simulation edges and must be float64 now (2) *BaseIOHandler._read_fluid_selection* should always return float64 now even if the on-disk data are stored in single precision 8. Difference between *add_output_field *and *add_field *used in FieldInfoContainer.setup_fluid_fields: 9. Dataset._parse_parameter_file: what does the following comment mean "Note that these are all assumed to be in code units; domain_left_edge and domain_right_edge will be updated to be in code units at a later time" => Perhaps it means that "domain_left_edge and domain_right_edge will be converted to YTArray automatically at a later time"? *Missing procedure*============================ 1. Add the new frontend in *frontends/api.py* 2. Set *GridIndex.max_level* 3. Set *Dataset.refine_by* *Bugin the current skeleton frontend* ============================ 1. data_structures.py: *self.dataset* is not set in GridIndex.__init__ when calling self.index_filename = self.dataset.parameter_filename 2. data_structures.py: replace self.Parent=[] by *self.Parent = None* 3. fields.py: the *field_list *argument is missing in FieldInfoContainer.__init__. This issue has been fixed in Jonah's PR. def __init__(self, ds, field_list): super(SkeletonFieldInfo, self).__init__(ds, field_list) Sincerely, Hsi-Yu (Justin)

2 2

Small change for default plot style
by Nathan Goldbaum 04 May '16

04 May '16

Hi all, On a whim I just tried out matplotlib's master branch, which includes the changes to the default plot style that will be in matplotlib 2.0. One of the things they changed is the way tickmarks are displayed. Currently, they go *inwards* in a plot, in matplotlib 2.0 they will go *outwards*. Changes to matplotlib shouldn't trigger changes to yt plots just because matplotlib changed, but perhaps we should purposefully adopt this change, since it can make plots easier to read. To make that concrete, here's a plot I just made using yt and matplotlib's master branch: http://imgur.com/BtEWO3U I don't understand why the fonts aren't correct - that will be fixed - but as far as the tickmarks go, I actually kind of like this. I think it's easier to read, especially for dark colormaps. What do you all think? What do you think about adopting outward pointing tickmarks for yt plots? -Nathan

4 4

PR review hangout: Wednesday May 4, 1:00 PM CDT
by Nathan Goldbaum 04 May '16

04 May '16

Hi all, Here's the link for this week's PR review hangout: https://plus.google.com/b/107728486871834552760/events/c7nu7kn6uqo4h4d04bdc… -Nathan

1 1

Issue #1216: Compiling error. Revision 19859 (yt_analysis/yt)
by Igor Krasnogorov 04 May '16

04 May '16

New issue 1216: Compiling error. Revision 19859 https://bitbucket.org/yt_analysis/yt/issues/1216/compiling-error-revision-1… Igor Krasnogorov: I can't compile Yt since 19859 revision. I use Windows 7-64 bit and run command ``` python setup.py build --compiler=mingw32 install ``` The problem is in the different defines of 'fmax', 'fmin' and 'log2' in 'platform_dep.h' and 'math.h' (from the mingw include). the error log: ``` skipping 'yt/geometry\grid_container.c' Cython extension (up-to-date) building 'yt.geometry.grid_container' extension C:\Anaconda2\Scripts\gcc.bat -DMS_WIN64 -mdll -O -Wall -Iyt/utilities/lib/ -IC:\Anaconda2\include -IC:\Anaconda2\PC -IC: \Anaconda2\lib\site-packages\numpy\core\include -c yt/geometry\grid_container.c -o build\temp.win-amd64-2.7\Release\yt\g eometry\grid_container.o In file included from C:\Anaconda2\lib\site-packages\numpy\core\include/numpy/ndarraytypes.h:1781:0, from C:\Anaconda2\lib\site-packages\numpy\core\include/numpy/ndarrayobject.h:18, from C:\Anaconda2\lib\site-packages\numpy\core\include/numpy/arrayobject.h:4, from yt/geometry\grid_container.c:242: C:\Anaconda2\lib\site-packages\numpy\core\include/numpy/npy_1_7_deprecated_api.h:12:9: note: #pragma message: C:\Anacond a2\lib\site-packages\numpy\core\include/numpy/npy_1_7_deprecated_api.h(12) : Warning Msg: Using deprecated NumPy API, di sable it by #defining NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION In file included from yt/geometry\grid_container.c:244:0: yt/utilities/lib/platform_dep.h:7:0: warning: "isnormal" redefined [enabled by default] In file included from C:\Anaconda2\include/pyport.h:325:0, from C:\Anaconda2\include/Python.h:58, from yt/geometry\grid_container.c:4: c:\anaconda2\mingw\bin\../lib/gcc/x86_64-w64-mingw32/4.7.0/../../../../x86_64-w64-mingw32/include/math.h:374:0: note: th is is the location of the previous definition In file included from yt/geometry\grid_container.c:244:0: yt/utilities/lib/platform_dep.h:20:24: error: static declaration of 'fmax' follows non-static declaration yt/utilities/lib/platform_dep.h:23:24: error: static declaration of 'fmin' follows non-static declaration yt/utilities/lib/platform_dep.h:26:24: error: static declaration of 'log2' follows non-static declaration yt/geometry\grid_container.c: In function '__Pyx_BufFmt_ProcessTypeChunk': yt/geometry\grid_container.c:26921:26: warning: unknown conversion type character 'z' in format [-Wformat] yt/geometry\grid_container.c:26921:26: warning: unknown conversion type character 'z' in format [-Wformat] yt/geometry\grid_container.c:26921:26: warning: too many arguments for format [-Wformat-extra-args] yt/geometry\grid_container.c:26973:20: warning: unknown conversion type character 'z' in format [-Wformat] yt/geometry\grid_container.c:26973:20: warning: unknown conversion type character 'z' in format [-Wformat] yt/geometry\grid_container.c:26973:20: warning: too many arguments for format [-Wformat-extra-args] yt/geometry\grid_container.c: In function '__pyx_buffmt_parse_array': yt/geometry\grid_container.c:27035:25: warning: unknown conversion type character 'z' in format [-Wformat] yt/geometry\grid_container.c:27035:25: warning: format '%d' expects argument of type 'int', but argument 3 has type 'siz e_t' [-Wformat] yt/geometry\grid_container.c:27035:25: warning: too many arguments for format [-Wformat-extra-args] yt/geometry\grid_container.c: In function '__Pyx_GetBufferAndValidate': yt/geometry\grid_container.c:27220:7: warning: unknown conversion type character 'z' in format [-Wformat] yt/geometry\grid_container.c:27220:7: warning: format '%s' expects argument of type 'char *', but argument 3 has type 'P y_ssize_t' [-Wformat] yt/geometry\grid_container.c:27220:7: warning: unknown conversion type character 'z' in format [-Wformat] yt/geometry\grid_container.c:27220:7: warning: too many arguments for format [-Wformat-extra-args] yt/geometry\grid_container.c: In function '__Pyx_RaiseArgtupleInvalid': yt/geometry\grid_container.c:27324:18: warning: unknown conversion type character 'z' in format [-Wformat] yt/geometry\grid_container.c:27324:18: warning: format '%s' expects argument of type 'char *', but argument 5 has type ' Py_ssize_t' [-Wformat] yt/geometry\grid_container.c:27324:18: warning: unknown conversion type character 'z' in format [-Wformat] yt/geometry\grid_container.c:27324:18: warning: too many arguments for format [-Wformat-extra-args] yt/geometry\grid_container.c: In function '__Pyx_RaiseTooManyValuesError': yt/geometry\grid_container.c:27774:18: warning: unknown conversion type character 'z' in format [-Wformat] yt/geometry\grid_container.c:27774:18: warning: too many arguments for format [-Wformat-extra-args] yt/geometry\grid_container.c: In function '__Pyx_RaiseNeedMoreValuesError': yt/geometry\grid_container.c:27780:18: warning: unknown conversion type character 'z' in format [-Wformat] yt/geometry\grid_container.c:27780:18: warning: format '%s' expects argument of type 'char *', but argument 3 has type ' Py_ssize_t' [-Wformat] yt/geometry\grid_container.c:27780:18: warning: too many arguments for format [-Wformat-extra-args] yt/geometry\grid_container.c: In function '__Pyx_ValidateAndInit_memviewslice': yt/geometry\grid_container.c:31184:22: warning: unknown conversion type character 'z' in format [-Wformat] yt/geometry\grid_container.c:31184:22: warning: format '%s' expects argument of type 'char *', but argument 3 has type ' Py_ssize_t' [-Wformat] yt/geometry\grid_container.c:31184:22: warning: unknown conversion type character 'z' in format [-Wformat] yt/geometry\grid_container.c:31184:22: warning: too many arguments for format [-Wformat-extra-args] In file included from C:\Anaconda2\lib\site-packages\numpy\core\include/numpy/ndarrayobject.h:27:0, from C:\Anaconda2\lib\site-packages\numpy\core\include/numpy/arrayobject.h:4, from yt/geometry\grid_container.c:242: yt/geometry\grid_container.c: At top level: C:\Anaconda2\lib\site-packages\numpy\core\include/numpy/__multiarray_api.h:1634:1: warning: '_import_array' defined but not used [-Wunused-function] error: command 'C:\\Anaconda2\\Scripts\\gcc.bat' failed with exit status 1 ```

1 0

Should support for octree data in the AMRKDTree be a blocker for 3.3?
by Nathan Goldbaum 03 May '16

03 May '16

Hi all, I'm moving some discussion from the comments in issue #1212 over to the mailing list: https://bitbucket.org/yt_analysis/yt/issues/1212 I wanted to check with the list since Cameron objected a bit in the issue discussion. The issue Cameron identifies is pretty serious, but fully fixing it will require pretty substantial work - adding support for octree data to the AMRKDTree. I've gone ahead and marked it for version 3.4 - i.e. that it shouldn't be a blocker for the 3.3 release. I've also created issue 1215: https://bitbucket.org/yt_analysis/yt/issues/1215 which will be fixed when yt produces a useful error message rather than seg faulting when trying to construct an AMRKDTree for unsupported data. I think that requiring someone to add support for octree data to the AMRKDTree before we release 3.3 is effectively delaying the 3.3 release arbitrarily far into the future, since no one is currently willing to work on that project. That would be unfortunate, because there's lots of cool stuff people have worked on that deserves a stable release. -Nathan

4 8

Issue #1215: Constructing AMRKDTree should fail early for oct and particle codes (yt_analysis/yt)
by Nathan Goldbaum 02 May '16

02 May '16

New issue 1215: Constructing AMRKDTree should fail early for oct and particle codes https://bitbucket.org/yt_analysis/yt/issues/1215/constructing-amrkdtree-sho… Nathan Goldbaum: Currently the AMRKDTree will seg on construction for oct and particle codes (see #1212). We should fail early instead of seg faulting. We should also mention in the documentation that this is an issue. In the long term we will avoid constructing a KDTree at all for these codes.

1 0