Mailman 3 February 2013 - yt-dev

NumberDensity in clumps and on FLASH data
by Matthew Turk July 9, 2013

July 9, 2013

Hi all, Mark Richardson wrote to the yt_analysis project with a problem with clumps on FLASH datasets. Here's his email: "Below includes my script and the output (it wouldn't let me attach two files). I get some output in the clump file up until I'd expect to see the Jeans Mass dumped. The output makes me think that it can't calculate this point because Gamma is unknown, or some other essential variable needed to get the number density. I've attempted to change the data_structure.py frontend for Flash to alias Gamma to gamc and game but I still get the 'cannot find Gamma' when loading my dataset." The issue from the traceback is that JeansMassMsun requires MeanMolecularWeight which needs NumberDensity. This can't be generated for his data. I am not an expert on FLASH, but it seems that this should be straightforward to fix, I just don't know how. Anybody from the FLASH or Clump side have any ideas? -Matt

5 5

memory usage with --parallel and periodic radius
by Michael Kuhlen Feb. 28, 2013

Feb. 28, 2013

Hi all and Nathan specifically I've found that the changes having to do with periodic fields are causing excessive memory usage for me when doing parallel halo profiling, to the point where I can no longer do analysis that used to work fine on my 24GB memory workstation. A simple HaloProfiler script (http://paste.yt-project.org/show/3211/), run in parallel on 8 processors on the halo catalog obtained from one of my cosmology simulations, currently almost immediate runs out of memory, while it used to complete without problems. I tried to use hg bisect to find the problematic changeset but had to skip the testing of several revisions because of runtime yt errors, so in the end bisect only gave me a range of potentially bad revisions: http://paste.yt-project.org/show/3210/. As you can see, they're all related to the periodic radius mods from about a month ago. I tried replicating this with the standard Enzo_64 example dataset from http://yt-project.org/data/, but I guess that one is too small to produce this problem on my machine. If you want to see the problem for yourself, then you can download this tarball ( http://astro.berkeley.edu/~mqk/transfer/RD0003.tar 4.5GB) and run this script (http://paste.yt-project.org/show/3211/) on it, like so: $ mpirun -np 8 python ./profile_halos.py --parallel When I run this with the current tip (ccfe34e70803) the total memory usage grows to >24GB. With the last good changeset the total memory usage never gets above ~8.5GB (estimated from top). It'd be great if we could find out what the problem is and fix it, because this is a major performance regression for me, that, as I already said, is seriously impacting my ability to do analysis with the current yt tip. Let me know if I should file a BB issue about this, and/or if there's some way I can assist with fixing this. Cheers, Mike -- ********************************************************************* * * * Dr. Michael Kuhlen Theoretical Astrophysics Center * * email: mqk(a)astro.berkeley.edu UC Berkeley * * cell phone: (831) 588-1468 B-116 Hearst Field Annex # 3411 * * skype username: mikekuhlen Berkeley, CA 94720 * * * *********************************************************************

2 4

[yt_analysis/yt] Memory usage and radius fields (issue #516)
by Michael Kuhlen Feb. 28, 2013

Feb. 28, 2013

--- you can reply above this line --- New issue 516: Memory usage and radius fields https://bitbucket.org/yt_analysis/yt/issue/516/memory-usage-and-radius-fiel… Michael Kuhlen: As discussed on the yt-dev mailing list, the new radius definitions, which more correctly handle non-periodic datasets, result in a very substantial (~3 times) increase in memory usage. This makes analysis (especially using sphere containers) of large datasets difficult. We should figure out how to revert back to the older, less memory intensive way of doing things, while preserving the improvements that the new definitions brought. -- This is an issue notification from bitbucket.org. You are receiving this either because you are the owner of the issue, or you are following the issue.

1 0

[yt_analysis/yt] Build fails because fKDpy.so has already been moved. (issue #515)
by Michael Kuhlen Feb. 27, 2013

Feb. 27, 2013

--- you can reply above this line --- New issue 515: Build fails because fKDpy.so has already been moved. https://bitbucket.org/yt_analysis/yt/issue/515/build-fails-because-fkdpyso-… Michael Kuhlen: See my comment here: https://bitbucket.org/yt_analysis/yt/commits/8ed119eea298b5561af7cecc44dc51… [I'm filing this issue, because I'm not sure whether the list gets notified about comments added to a changeset after it has been pulled in.] Responsible: xarthisius -- This is an issue notification from bitbucket.org. You are receiving this either because you are the owner of the issue, or you are following the issue.

1 0

[yt_analysis/yt] Install_script fails to build fKDpy.so using freshly installed Forthon (issue #514)
by Kacper Kowalik Feb. 27, 2013

Feb. 27, 2013

--- you can reply above this line --- New issue 514: Install_script fails to build fKDpy.so using freshly installed Forthon https://bitbucket.org/yt_analysis/yt/issue/514/install_script-fails-to-buil… Kacper Kowalik: As reported on [yt-users ml](http://lists.spacepope.org/pipermail/yt-users-spacepope.org/2013-Februa… by Mike Butler installation process was broken by pull request #441. -- This is an issue notification from bitbucket.org. You are receiving this either because you are the owner of the issue, or you are following the issue.

1 0

[yt_analysis/yt] Athena unitary units broken for domain_left_edge = 0.0 (issue #513)
by Sam Skillman Feb. 27, 2013

Feb. 27, 2013

--- you can reply above this line --- New issue 513: Athena unitary units broken for domain_left_edge = 0.0 https://bitbucket.org/yt_analysis/yt/issue/513/athena-unitary-units-broken-… Sam Skillman: Reported by Nick Moekel: an example, with domain_left_edge = [0, 0, 0], and right_edge = [1, 1, 1] ``` #!python In [18]: center=[0.5,0.5,0.5] In [19]: lc = [0,0, 0.0, ,0.4] In [20]: rc=[1.0,1.0,0.6] In [21]: region = pf.h.region(center, lc, rc) In [22]: region.volume() Out[22]: -inf ``` This is caused by pf['unitary'] = -np.inf, because during the parsing of the parameter file, we guess that the domain_right_edge = - domain_left_edge. This gets corrected during the hierarchy build, but the change doesn't propogate to fix pf['unitary']. Responsible: samskillman -- This is an issue notification from bitbucket.org. You are receiving this either because you are the owner of the issue, or you are following the issue.

1 0

PR for YTEP-0001 modification
by Matthew Turk Feb. 27, 2013

Feb. 27, 2013

Hi all, Just a quick note, since some of you may be interested. Because yt 3.0 does away with the _grids attribute on data containers, I have implemented something of a workaround for those routines (clump finding, volume rendering) that need 3D data arrays, masks (i.e., _get_cut_mask or _get_point_indices), and possibly even spatial locations from data sources. https://bitbucket.org/yt_analysis/ytep/pull-request/10/adding-a-blocks-attr… -Matt

1 0

[yt_analysis/yt] GDF sidecar file needs to be ported to 3.0 (issue #512)
by Matthew Turk Feb. 26, 2013

Feb. 26, 2013

--- you can reply above this line --- New issue 512: GDF sidecar file needs to be ported to 3.0 https://bitbucket.org/yt_analysis/yt/issue/512/gdf-sidecar-file-needs-to-be… Matthew Turk: The sidecar file should be inserted at the level of the geometry handler, where if it exists it can be called with the selector. Responsible: MatthewTurk -- This is an issue notification from bitbucket.org. You are receiving this either because you are the owner of the issue, or you are following the issue.

1 0

amrspace + load_amr_grids
by Sam Skillman Feb. 26, 2013

Feb. 26, 2013

Hi all, I'm trying to figure out how to use amrspace combined with load_amr_grids to create a decently complex amr structure for testing. However, I can't find any example of doing such a thing in any documentation or source. My attempt below fails in that it creates no level-0 grids, and also has the ordering of grids messed up grid 0 is level 4. Any advice on combining these two functions? http://paste.yt-project.org/show/3206/ Thanks, Sam

2 4

YTEP-0008: Release procedure
by Matthew Turk Feb. 25, 2013

Feb. 25, 2013

Hi all, Because of the somewhat dysfunctional release process we've had recently, there was some discussion of how to improve this in IRC. I've tried to synthesize many of those comments (thanks, Sam, Nathan, Cameron, Jeff, Britton, Kacper, etc...) and come up with a plan for releases. I'd appreciate feedback on this in the PR. I anticipate that this YTEP will be one of the most updated, as it will need updates as we fine tune things, and as we move forward with new versions. Please leave comments on the PR, or if you feel they warrant much further discussion, in reply to this email. https://bitbucket.org/yt_analysis/ytep/pull-request/9/adding-draft-of-ytep-… -Matt

4 6