Mailman 3 April 2016 - yt-dev

parallelHOP in yt-3
by Josh Moloney 05 Apr '16

05 Apr '16

I've recently been trying to get parallelHOP integrated into the current version of yt-3. (I need halo finding with star particles which I think rules out Rockstar, and I eventually need to work with grids up to 1536^3 which would be problematic with regular HOP.) I currently have it working properly for single core runs, and it produces output that matches the yt-2.x version. It also runs to completion with mpi, and produces mostly sensible output. However, when run on multiple cores it has problems with the halos near the region boundaries. In particular it produces a large number (~10% of total) of typically small "bad halos" which have 0 values for the halo maximum density/location (i.e. [0, 0, 0, 0]), and at least one negative value for the center of mass location (due to incorrect handling of periodicity). In addition, some of the "real" halos near the region boundaries have fewer particles in parallel runs. Comparing halo particle lists between serial and mpi runs shows that some of these missing particles are located in the "bad halos". Running with premerge=False improves the situation slightly (slightly fewer "bad halos", better agreement in some of the real ones). I think this means that the halo finder isn't correctly joining up some chains across processors, but I'm not familiar enough with the parallelHOP code to know where to look first. The only changes I've made to the code are in halo_objects.py, updating the old self.hierarchy.region_strict and similar calls as well as fixing units in a few places. I haven't made any changes in parallel_hop_interface.py. Has anyone run into a similar problem before? If integrating parallelHOP is known to be unworkable (or extremely difficult) I'll start looking at other options. Otherwise, if anyone has some insight into where the problem might lie or what to look at first, I'd appreciate any advice I can get. Thanks. - Josh

2 1

Issue #1205: default disparity in StereoPerspective lens doesn't show much difference between left and right (yt_analysis/yt)
by Michael Zingale 02 Apr '16

02 Apr '16

New issue 1205: default disparity in StereoPerspective lens doesn't show much difference between left and right https://bitbucket.org/yt_analysis/yt/issues/1205/default-disparity-in-stere… Michael Zingale: Depending on what you are rendering, when you do a volume rendering with the lens="stereo-perspective", the left and right images do not look at all different. To fix this, you need to change the disparity parameter, like: ``` sc.camera.lens.disparity = cam.width[0]/10.0 ``` this setting is 50x greater than the default, and gives a noticable left-right difference. We need to figure out what a reasonable default is for the `disparity`, and also we should document this better in the volume rendering docs.

1 0

Issue #1204: test_composite_mesh_render randomly fails (yt_analysis/yt)
by Kacper Kowalik 02 Apr '16

02 Apr '16

New issue 1204: test_composite_mesh_render randomly fails https://bitbucket.org/yt_analysis/yt/issues/1204/test_composite_mesh_render… Kacper Kowalik: Example bad [results](https://tests.yt-project.org/job/yt_py27/lastCompletedBuild/testRe… Test is currently disabled on jenkins. Responsible: atmyers

1 0

New install script
by Nathan Goldbaum 01 Apr '16

01 Apr '16

Hi all, I'm almost finished updating the install script so that by default it creates a conda-based installation, but can also optionally do a from-source bootstrapped installation. https://bitbucket.org/yt_analysis/yt/pull-requests/2009 Before the pull request can be merged, it needs testing on a variety of platforms as well as some sort of test script we can run as part of the test suite. If any of you have some time and interest in helping me out, I'd appreciate it if you could try. You should read the description at the top of the pull request and modify any variables you would like to change to customize your installation. You should be able to get the latest version of the script by doing: $ curl -O https://bitbucket.org/ngoldbaum/yt/raw/install-script-updates/doc/install_s… If you run into any problems, either reply here or on the pull request with a description of the issue or a solution and I will try to fix them. Thanks! -Nathan

1 0

Sequence of buff_size
by Yi-Hao Chen 01 Apr '16

01 Apr '16

Hi all, Running this test script http://paste.yt-project.org/show/6379/ will result in this image https://slack-files.com/T042F73QM-F0X0YJE3V-4a579d9431 Apparently, the sequence of buff_size is switched so that set_buff_size((8,4)) gives a 4 by 8 image. I have traced down the codes and found the following lines are the cause of the problem. https://bitbucket.org/yt_analysis/yt/src/d18f33211199f71e2fac4d927307b015d5… https://bitbucket.org/yt_analysis/yt/src/d18f33211199f71e2fac4d927307b015d5… If rows=nx and cols=ny, the two functions (pixelize_cartesian and pixelize_off_axis_cartesian) should take rows first and then cols. The same API has been around in _MPL.c since 2010 and thus changing it will be backward incompatible. However, to be consistent with other parts of the code (e.g. here <https://bitbucket.org/yt_analysis/yt/src/d18f33211199f71e2fac4d927307b015d5…>, here <https://bitbucket.org/yt_analysis/yt/src/d18f33211199f71e2fac4d927307b015d5…>, here <https://bitbucket.org/yt_analysis/yt/src/d18f33211199f71e2fac4d927307b015d5…>, and here <https://bitbucket.org/yt_analysis/yt/src/d18f33211199f71e2fac4d927307b015d5…>), in which buff_size = (nx, ny) is assumed and used, I propose to change the behavior of the above two functions. What do you think? Anyone has other suggestions? Thanks! Yi-Hao

3 5

Contribution graph
by Matthew Turk 01 Apr '16

01 Apr '16

Hi folks, I know we don't have a contribution graph for yt on bitbucket the way github does, but this piece still really hit home for me. Thanks to Nathan for posting it in Slack; I thought it deserved amplification here. http://erik.io/blog/2016/04/01/how-github-contribution-graph-is-harmful/ -Matt

1 0

[ANN] xs 1.0: The next generation of yt
by Matthew Turk 31 Mar '16

31 Mar '16

Dear everyone, I am proud to announce the release of xs 1.0: the next generation of yt. xs is a new package, designed from the ground up to take advantage of emerging architectures and to address challenges presented by the next generation of computing hardware. In keeping with this vision, xs facilitates the deployment of complex analysis pipelines as defined at compile time, rather than run time as is common in most yt usage. For more information, including compilation instructions, contribution guidelines, and download information, please see the homepage at: http://dxl.ncsa.illinois.edu/~mturk/xs/ Best wishes, Matt

1 0