Mailman 3 February 2013 - yt-dev

NumberDensity in clumps and on FLASH data
by Matthew Turk July 9, 2013

July 9, 2013

Hi all, Mark Richardson wrote to the yt_analysis project with a problem with clumps on FLASH datasets. Here's his email: "Below includes my script and the output (it wouldn't let me attach two files). I get some output in the clump file up until I'd expect to see the Jeans Mass dumped. The output makes me think that it can't calculate this point because Gamma is unknown, or some other essential variable needed to get the number density. I've attempted to change the data_structure.py frontend for Flash to alias Gamma to gamc and game but I still get the 'cannot find Gamma' when loading my dataset." The issue from the traceback is that JeansMassMsun requires MeanMolecularWeight which needs NumberDensity. This can't be generated for his data. I am not an expert on FLASH, but it seems that this should be straightforward to fix, I just don't know how. Anybody from the FLASH or Clump side have any ideas? -Matt

5 5

memory usage with --parallel and periodic radius
by Michael Kuhlen Feb. 28, 2013

Feb. 28, 2013

Hi all and Nathan specifically I've found that the changes having to do with periodic fields are causing excessive memory usage for me when doing parallel halo profiling, to the point where I can no longer do analysis that used to work fine on my 24GB memory workstation. A simple HaloProfiler script (http://paste.yt-project.org/show/3211/), run in parallel on 8 processors on the halo catalog obtained from one of my cosmology simulations, currently almost immediate runs out of memory, while it used to complete without problems. I tried to use hg bisect to find the problematic changeset but had to skip the testing of several revisions because of runtime yt errors, so in the end bisect only gave me a range of potentially bad revisions: http://paste.yt-project.org/show/3210/. As you can see, they're all related to the periodic radius mods from about a month ago. I tried replicating this with the standard Enzo_64 example dataset from http://yt-project.org/data/, but I guess that one is too small to produce this problem on my machine. If you want to see the problem for yourself, then you can download this tarball ( http://astro.berkeley.edu/~mqk/transfer/RD0003.tar 4.5GB) and run this script (http://paste.yt-project.org/show/3211/) on it, like so: $ mpirun -np 8 python ./profile_halos.py --parallel When I run this with the current tip (ccfe34e70803) the total memory usage grows to >24GB. With the last good changeset the total memory usage never gets above ~8.5GB (estimated from top). It'd be great if we could find out what the problem is and fix it, because this is a major performance regression for me, that, as I already said, is seriously impacting my ability to do analysis with the current yt tip. Let me know if I should file a BB issue about this, and/or if there's some way I can assist with fixing this. Cheers, Mike -- ********************************************************************* * * * Dr. Michael Kuhlen Theoretical Astrophysics Center * * email: mqk(a)astro.berkeley.edu UC Berkeley * * cell phone: (831) 588-1468 B-116 Hearst Field Annex # 3411 * * skype username: mikekuhlen Berkeley, CA 94720 * * * *********************************************************************

2 4

[yt_analysis/yt] Memory usage and radius fields (issue #516)
by Michael Kuhlen Feb. 28, 2013

Feb. 28, 2013

--- you can reply above this line --- New issue 516: Memory usage and radius fields https://bitbucket.org/yt_analysis/yt/issue/516/memory-usage-and-radius-fiel… Michael Kuhlen: As discussed on the yt-dev mailing list, the new radius definitions, which more correctly handle non-periodic datasets, result in a very substantial (~3 times) increase in memory usage. This makes analysis (especially using sphere containers) of large datasets difficult. We should figure out how to revert back to the older, less memory intensive way of doing things, while preserving the improvements that the new definitions brought. -- This is an issue notification from bitbucket.org. You are receiving this either because you are the owner of the issue, or you are following the issue.

1 0

[yt_analysis/yt] Build fails because fKDpy.so has already been moved. (issue #515)
by Michael Kuhlen Feb. 27, 2013

Feb. 27, 2013

--- you can reply above this line --- New issue 515: Build fails because fKDpy.so has already been moved. https://bitbucket.org/yt_analysis/yt/issue/515/build-fails-because-fkdpyso-… Michael Kuhlen: See my comment here: https://bitbucket.org/yt_analysis/yt/commits/8ed119eea298b5561af7cecc44dc51… [I'm filing this issue, because I'm not sure whether the list gets notified about comments added to a changeset after it has been pulled in.] Responsible: xarthisius -- This is an issue notification from bitbucket.org. You are receiving this either because you are the owner of the issue, or you are following the issue.

1 0

[yt_analysis/yt] Install_script fails to build fKDpy.so using freshly installed Forthon (issue #514)
by Kacper Kowalik Feb. 27, 2013

Feb. 27, 2013

--- you can reply above this line --- New issue 514: Install_script fails to build fKDpy.so using freshly installed Forthon https://bitbucket.org/yt_analysis/yt/issue/514/install_script-fails-to-buil… Kacper Kowalik: As reported on [yt-users ml](http://lists.spacepope.org/pipermail/yt-users-spacepope.org/2013-Februa… by Mike Butler installation process was broken by pull request #441. -- This is an issue notification from bitbucket.org. You are receiving this either because you are the owner of the issue, or you are following the issue.

1 0

[yt_analysis/yt] Athena unitary units broken for domain_left_edge = 0.0 (issue #513)
by Sam Skillman Feb. 27, 2013

Feb. 27, 2013

--- you can reply above this line --- New issue 513: Athena unitary units broken for domain_left_edge = 0.0 https://bitbucket.org/yt_analysis/yt/issue/513/athena-unitary-units-broken-… Sam Skillman: Reported by Nick Moekel: an example, with domain_left_edge = [0, 0, 0], and right_edge = [1, 1, 1] ``` #!python In [18]: center=[0.5,0.5,0.5] In [19]: lc = [0,0, 0.0, ,0.4] In [20]: rc=[1.0,1.0,0.6] In [21]: region = pf.h.region(center, lc, rc) In [22]: region.volume() Out[22]: -inf ``` This is caused by pf['unitary'] = -np.inf, because during the parsing of the parameter file, we guess that the domain_right_edge = - domain_left_edge. This gets corrected during the hierarchy build, but the change doesn't propogate to fix pf['unitary']. Responsible: samskillman -- This is an issue notification from bitbucket.org. You are receiving this either because you are the owner of the issue, or you are following the issue.

1 0

PR for YTEP-0001 modification
by Matthew Turk Feb. 27, 2013

Feb. 27, 2013

Hi all, Just a quick note, since some of you may be interested. Because yt 3.0 does away with the _grids attribute on data containers, I have implemented something of a workaround for those routines (clump finding, volume rendering) that need 3D data arrays, masks (i.e., _get_cut_mask or _get_point_indices), and possibly even spatial locations from data sources. https://bitbucket.org/yt_analysis/ytep/pull-request/10/adding-a-blocks-attr… -Matt

1 0

[yt_analysis/yt] GDF sidecar file needs to be ported to 3.0 (issue #512)
by Matthew Turk Feb. 26, 2013

Feb. 26, 2013

--- you can reply above this line --- New issue 512: GDF sidecar file needs to be ported to 3.0 https://bitbucket.org/yt_analysis/yt/issue/512/gdf-sidecar-file-needs-to-be… Matthew Turk: The sidecar file should be inserted at the level of the geometry handler, where if it exists it can be called with the selector. Responsible: MatthewTurk -- This is an issue notification from bitbucket.org. You are receiving this either because you are the owner of the issue, or you are following the issue.

1 0

amrspace + load_amr_grids
by Sam Skillman Feb. 26, 2013

Feb. 26, 2013

Hi all, I'm trying to figure out how to use amrspace combined with load_amr_grids to create a decently complex amr structure for testing. However, I can't find any example of doing such a thing in any documentation or source. My attempt below fails in that it creates no level-0 grids, and also has the ordering of grids messed up grid 0 is level 4. Any advice on combining these two functions? http://paste.yt-project.org/show/3206/ Thanks, Sam

2 4

YTEP-0008: Release procedure
by Matthew Turk Feb. 25, 2013

Feb. 25, 2013

Hi all, Because of the somewhat dysfunctional release process we've had recently, there was some discussion of how to improve this in IRC. I've tried to synthesize many of those comments (thanks, Sam, Nathan, Cameron, Jeff, Britton, Kacper, etc...) and come up with a plan for releases. I'd appreciate feedback on this in the PR. I anticipate that this YTEP will be one of the most updated, as it will need updates as we fine tune things, and as we move forward with new versions. Please leave comments on the PR, or if you feel they warrant much further discussion, in reply to this email. https://bitbucket.org/yt_analysis/ytep/pull-request/9/adding-draft-of-ytep-… -Matt

4 6

Emails from BitBucket about Pull Requests
by Matthew Turk Feb. 25, 2013

Feb. 25, 2013

Hi everyone in yt and Enzo lands, BitBucket just changed how notifications are handled: http://blog.bitbucket.org/2013/02/25/pull-requests-now-with-reviewers-and-s… This seems like a good step in the right direction. So, if you've been getting *too many* emails about pull requests, or if you've been getting *not enough* emails, this change will help. -Matt

1 0

[yt_analysis/yt] SlicePlot fails in parallel with multiple fields when one of which is a derived fields (issue #511)
by Sam Skillman Feb. 23, 2013

Feb. 23, 2013

--- you can reply above this line --- New issue 511: SlicePlot fails in parallel with multiple fields when one of which is a derived fields https://bitbucket.org/yt_analysis/yt/issue/511/sliceplot-fails-in-parallel-… Sam Skillman: If I only do ['Density'], or ['Pressure'], or even ['Density', 'Temperature'], this is fine. Something seems messed up with the derived field. FWIW, ProjectionPlot works fine. mpirun -np 2 python script.py --parallel ``` #!python from yt.mods import * pf = load('IsolatedGalaxy/galaxy0030/galaxy0030') field = ['Density','Pressure'] slc = SlicePlot(pf, 'z', field) slc.save() ``` gives: ``` #!text P001 yt : [DEBUG ] 2013-02-23 15:55:18,394 Appending object to galaxy0030 (type: <class 'yt.data_objects.hierarchy.AMRSlice'>) Traceback (most recent call last): File "dummy_script.py", line 6, in <module> slc = SlicePlot(pf, 'z', field) File "/home/skillman/local/src/yt-hg/yt/visualization/plot_window.py", line 1134, in __init__ Traceback (most recent call last): File "dummy_script.py", line 6, in <module> slc = SlicePlot(pf, 'z', field) File "/home/skillman/local/src/yt-hg/yt/visualization/plot_window.py", line 1134, in __init__ slc = pf.h.slice(axis, center[axis], center=center, fields=fields, **field_parameters) File "/home/skillman/local/src/yt-hg/yt/data_objects/data_containers.py", line 1049, in __init__ self._refresh_data() File "/home/skillman/local/src/yt-hg/yt/data_objects/data_containers.py", line 318, in _refresh_data self.get_data() File "/home/skillman/local/src/yt-hg/yt/data_objects/data_containers.py", line 845, in get_data slc = pf.h.slice(axis, center[axis], center=center, fields=fields, **field_parameters) File "/home/skillman/local/src/yt-hg/yt/data_objects/data_containers.py", line 360, in _generate_field self._refresh_data() File "/home/skillman/local/src/yt-hg/yt/data_objects/data_containers.py", line 318, in _refresh_data self[field] = self.pf.field_info[field](self) File "/home/skillman/local/src/yt-hg/yt/data_objects/field_info_container.py", line 393, in __call__ self.get_data() File "/home/skillman/local/src/yt-hg/yt/data_objects/universal_fields.py", line 216, in _Pressure data["Density"] * data["ThermalEnergy"] ValueError: operands could not be broadcast together with shapes (19686) (40177) ``` -- This is an issue notification from bitbucket.org. You are receiving this either because you are the owner of the issue, or you are following the issue.

1 0

[yt_analysis/yt] Switch docs theme (issue #510)
by Matthew Turk Feb. 23, 2013

Feb. 23, 2013

--- you can reply above this line --- New issue 510: Switch docs theme https://bitbucket.org/yt_analysis/yt/issue/510/switch-docs-theme Matthew Turk: The docs theme currently has poor information density. Something like the bootstrap theme should help with this. Responsible: MatthewTurk -- This is an issue notification from bitbucket.org. You are receiving this either because you are the owner of the issue, or you are following the issue.

1 0

Thoughts on docs theme?
by Matthew Turk Feb. 23, 2013

Feb. 23, 2013

Hi all, I've been working on the docs a bit, and I was wondering if I could take a pulse of the group on the theme. I think the fonts are a bit too big, and that poses a challenge to reading the docs. http://yt-project.org/docs/2.5/ I have been experimenting with a couple other themes. I uploaded the bootstrap theme here: http://yt-project.org/docs/dev/ The logo and paragraph symbols aside, I think the fonts look a lot nicer here, easier to read, easier to skim, etc. Some particular pages: http://yt-project.org/docs/dev/cookbook/simple_plots.html http://yt-project.org/docs/2.5/cookbook/simple_plots.html [+-]1 on this theme? I figure we could maybe make the navbar green to match the color. There are a bunch of other themes on pypi.python.org , if you search "sphinx." I think I'm just unsatisfied with the information density and style of the current documentation, so anything could potentially be an improvement. -Matt

9 10

[yt_analysis/yt] Unit and answer tests should be parallel safe (issue #509)
by Kacper Kowalik Feb. 22, 2013

Feb. 22, 2013

--- you can reply above this line --- New issue 509: Unit and answer tests should be parallel safe https://bitbucket.org/yt_analysis/yt/issue/509/unit-and-answer-tests-should… Kacper Kowalik: Goal is to make the testsuite much faster than it's now. The most demanding tests should be identified and rewritten if necessary. Tentative plan is to achieve less than 5 minutes for total time. Responsible: xarthisius -- This is an issue notification from bitbucket.org. You are receiving this either because you are the owner of the issue, or you are following the issue.

1 0

pmods
by Stephen Skory Feb. 22, 2013

Feb. 22, 2013

Hi all, I think I've found a hacky way to get around the pmods import problem we've been seeing. Since it's so hacky, I'm not going to issue a PR until somone else lays eyes on it and tries it and lets me know if it's A) too hacky, B) just hacky enough, or C) what? here's the diff http://paste.yt-project.org/show/3201/ Thanks! -- Stephen Skory s(a)skory.us http://stephenskory.com/ 510.621.3687 (google voice)

2 3

[yt_analysis/yt] bug with volume rendering function draw_grids (issue #508)
by Cameron Hummels Feb. 21, 2013

Feb. 21, 2013

--- you can reply above this line --- New issue 508: bug with volume rendering function draw_grids https://bitbucket.org/yt_analysis/yt/issue/508/bug-with-volume-rendering-fu… Cameron Hummels: the function currently draws grids from the entire simulation volume, even if the user specifies a subregion to render. need to limit to just overplotting grids in the subregion if specified. Responsible: chummels -- This is an issue notification from bitbucket.org. You are receiving this either because you are the owner of the issue, or you are following the issue.

1 0

[yt_analysis/yt] Plot window sizes should be adjustable (issue #507)
by John Wise Feb. 21, 2013

Feb. 21, 2013

--- you can reply above this line --- New issue 507: Plot window sizes should be adjustable https://bitbucket.org/yt_analysis/yt/issue/507/plot-window-sizes-should-be-… John Wise: Plots made with the PlotWindow infrastructure are not easily adjustable and are fixed at 10 inches and 800 pixels. Also the fontsize arguments are not functional. Responsible: jwise77 -- This is an issue notification from bitbucket.org. You are receiving this either because you are the owner of the issue, or you are following the issue.

1 0

Spatial fields for Octrees in 3.0
by Matthew Turk Feb. 21, 2013

Feb. 21, 2013

Hi all, Octree support in 3.0 is coming along quite well. The main stumbling block is in support seems to be related to spatial fields. In particular, fields that rely on being passed 3D arrays of data. Many of these fields (such as DivV) rely on ghost zone information. Following on this, we need to be able to place particles inside individual Octs. In the past, I've assumed that the overhead of a function call being called on many, many small arrays (2x2x2) would outweigh the gain from vectorized numpy arithmetic. This morning, after testing, it seems the situation is a bit more complex. Here's a sample script: http://paste.yt-project.org/show/3184/ The results: http://paste.yt-project.org/show/3183/ Now, what this is doing is simulating the process of going from one big grid down to many small grids, such that at the end we're doing 2x2x2 grid objects, for a relatively complex field (RadiusCode). These results are not at *all* what I expected -- doing this grid-by-grid for octs outperforms doing it for the big array. My first guess at why this is is that the RadiusCode field likely performs a number of copies. Now, if we do this instead with DensitySquared (which is a flop-light routine) we see: http://paste.yt-project.org/show/3185/ (Here I've also upped to a 512^3 domain dimensions) Again, the Octs perform pretty well, out-performing everything else. I assume, perhaps incorrectly, this is related to memory allocation patterns or something. Regardless, it seems that what I had previously discounted as too costly (iterating over octs) is not so costly after all. I'd like to propose -- pending input from at least Doug, Sam and Chris -- that the spatial chunking for Octrees be constructed like so: * When a chunk_spatial call is made, iterate over io chunks. This will enable management based on IO, delegated to the individual OctGeometryHandler. (If number of requested ghost zones > 0, we can also build that information in here, as well.) * Inside the iteration of io chunks, read as much as is necessary from each (without child masking, as is usually the case for spatial data). yield either an Oct singleton or a dict pre-filled with the field data needed. (At this time we will know which fields must be read from disk and which must be generated.) * Back at top level, where the spatial chunking was called, fill back into the original data selector. This will likely require a new per-geometry handler function that accepts a spatial iterator and then fills a flat, non-spatial selector. I think this is reasonable, and I can have a pass at implementing it. I'll update the YTEP once it's done. However, it's also brought up the point that during the workshop we absolutely need to figure out a distributed memory octree approach. Before we implement such a thing, we also need rigorous testing of the three Octree codes that are included in 3.0. I believe this strategy for spatial fields in Octrees will also work for particle fields; you should be able to read a particle field and use it to generate a fluid field. For instance, the CICDeposit functions. Any feedback would be appreciated, but I think modulo strong objections I want to get started on this early next week, once my schedule lightens up a bit. Also, this should be fun for the workshop! :) -Matt

2 2

YTEP: Automatic Pull Requests’ validation
by Kacper Kowalik Feb. 21, 2013

Feb. 21, 2013

Hi all, My YTEP-0007 on Automatic Pull Requests’ validation was just accepted (Thanks Matt!) https://yt.readthedocs.org/projects/ytep/en/latest/YTEPs/YTEP-0007.html It's now target only at 2.x branch, but I hope we will extend it to 3.0 soon. If you have any further suggestions or enhancements please reply to this mail. TIA! Best, Kacper

1 0

[yt_analysis/yt] fKDpy.so should be built automatically (issue #506)
by Kacper Kowalik Feb. 21, 2013

Feb. 21, 2013

--- you can reply above this line --- New issue 506: fKDpy.so should be built automatically https://bitbucket.org/yt_analysis/yt/issue/506/fkdpyso-should-be-built-auto… Kacper Kowalik: If there are no apparent reasons against that idea, I think it would be convenient if `fKDpy.so` was built while running `python setup.py build` or `install` -- This is an issue notification from bitbucket.org. You are receiving this either because you are the owner of the issue, or you are following the issue.

1 0

[yt_analysis/yt] Plot window answer tests (issue #505)
by Nathan Goldbaum Feb. 20, 2013

Feb. 20, 2013

--- you can reply above this line --- New issue 505: Plot window answer tests https://bitbucket.org/yt_analysis/yt/issue/505/plot-window-answer-tests Nathan Goldbaum: The correctness of many of the plot callbacks and modifications can be easily evaluated by looking at the plot but are difficult to evaluate in an automated, non-visual way. We should add some tests that create plot window plots, modify them, run callbacks on them, and then save them to disk. We can then store the old images, which can be verified visually by a human, and then every time the answer tests get run, compare the new plots to the known good plots stored in the answers by doing a bitwise diff or some other automated comparison procedure. Responsible: ngoldbaum -- This is an issue notification from bitbucket.org. You are receiving this either because you are the owner of the issue, or you are following the issue.

1 0

[yt_analysis/yt] Plot window axes labels and tick labels should have the same font by default. (issue #504)
by Nathan Goldbaum Feb. 20, 2013

Feb. 20, 2013

--- you can reply above this line --- New issue 504: Plot window axes labels and tick labels should have the same font by default. https://bitbucket.org/yt_analysis/yt/issue/504/plot-window-axes-labels-and-… Nathan Goldbaum: Right now the axes labels and tick label fonts are chosen by matplotlib. Since we use mathtext to generate the axes labels, they get a serif font while the tick labels are sans serif by default. We should figure out a way to force the two fonts to be the same. I'd prefer to use the font the axes labels currently use, as it looks similar to publication fonts. Responsible: ngoldbaum -- This is an issue notification from bitbucket.org. You are receiving this either because you are the owner of the issue, or you are following the issue.

1 0

Avoiding roundoff errors in comparison of floats
by Kacper Kowalik Feb. 19, 2013

Feb. 19, 2013

Hi All! I've been recently struck by roundoff errors in yt. AFAIR some variants of such bugs have been already reported from time to time on #yt or mailing lists and I'd like to propose a general solution that should solve this. The problem can be seen here: http://imgur.com/oIaarPR It's a dense blob (\rho = 2) moving diagonally across uniform medium (\rho = 1). As you can see there are blueish stripes on the few block boundaries. They're caused by yt thinking that blocks on n+1 refinement level overlap with those regions in blocks on "n" level. It all happens here: (yt/data_objects/object_finding_mixin.py) ObjectFindingMixin::get_box_grids ... grid_i = np.where( (np.all(self.grid_right_edge > left_edge, axis=1) & np.all(self.grid_left_edge < right_edge, axis=1)) == True ) The problem is that comparing floats is *not* accurate. What should be done: instead of "a > b", we should do e.g. "(a - b) > eps", where eps is small number grater than machine precision. I've fixed this particular bit as follows: ... dx = self.get_smallest_dx() * 0.1 grid_i = np.where( (np.all(self.grid_right_edge > (left_edge + dx), axis=1) & np.all(self.grid_left_edge < (right_edge - dx), axis=1)) == True ) 10% of the smallest cell size seems to be good approximation of eps at this point. As a result I've got: http://imgur.com/vCGQUKv Still one artifact remained. It was caused by exact the same issue in other place in the code (utilities/lib/geometry_utils.pyx:get_box_grids_level) Using similar trick there yielded: http://imgur.com/wpOCGaf \o/ Sorry for the long mail, but I wanted to share this so we could inspect other place that could suffer from similar issues too. Cheers, Kacper P.s. Nice animations are available here: http://tinyurl.com/aw2tbof http://tinyurl.com/amp7noc http://tinyurl.com/arnqmfw

2 3

[yt_analysis/yt] pmods.py broken (issue #503)
by Cameron Hummels Feb. 15, 2013

Feb. 15, 2013

--- you can reply above this line --- New issue 503: pmods.py broken https://bitbucket.org/yt_analysis/yt/issue/503/pmodspy-broken Cameron Hummels: I get a failure, when I try to run a yt script in parallel using the header line: from yt.pmods import * while executing as: mpirun -np 2 python script.py --parallel The traceback is here: http://paste.yt-project.org/show/3171/ I do this to follow the prescription in http://yt.readthedocs.org/en/latest/advanced/parallel_computation.html -- This is an issue notification from bitbucket.org. You are receiving this either because you are the owner of the issue, or you are following the issue.

1 0

2.5 release next Thursday
by Matthew Turk Feb. 15, 2013

Feb. 15, 2013

Hi all, Okay, after a discussion in IRC about releasing 2.5, we've decided to just start pushing ahead. I'd like to propose ([+-][01]) that we set a hard cutoff of a week from today, Valentine's day. If anything doesn't make it in, that's fine. We just put out a release without it and wrap it up in 2.6. (Yeah, we probably need a 2.6.) The remaining blockers: * I want Andrew's pull request to make it in. This is a huge win for everyone. Once it's in, we can add documentation about it, and then it'll be a big feature. * John ZuHone has an outstanding piece of code that builds Enzo data from in-memory hierarchies. I would like to see this make it in, but it is worth a release all by itself. * The remaining tickets are mostly all documentation based. We need: bootcamp in the docs, rotation information in the docs, offaxisprojection in the docs. * We have a need for answer testing for Orion, FLASH, NYX and optionally GDF datasets. I have been the bottleneck here. If you have pinged me in the past, ping me again and I will do this with you. If we don't get to this by 14th, this will be bumped. However: after 2.5 goes out, I would like for us to stop doing *major* features on the 2.x branch and try to focus on 3.0. This may not be possible for all things, but I would like to try to push for that. At a bare minimum, every new feature in 2.6 should be well-tested enough that we can run those tests on 3.0 to determine if it is directly portable. And future-compatible requirements will be in place for new features in 2.6. Also, I'd like to nominate John ZuHone to send out the release announcement, particularly since he has worked so hard to put together initial conditions generation in this release. -Matt

2 4

[yt_analysis/yt] Slices in GDF no longer respect data ordering (issue #502)
by Kacper Kowalik Feb. 15, 2013

Feb. 15, 2013

--- you can reply above this line --- New issue 502: Slices in GDF no longer respect data ordering https://bitbucket.org/yt_analysis/yt/issue/502/slices-in-gdf-no-longer-resp… Kacper Kowalik: Recent changes in [gdf/io.py](https://bitbucket.org/xarthisius/yt/commits/e1b7792e680790618385… result in plots being transposed when column-ordered data is being used. -- This is an issue notification from bitbucket.org. You are receiving this either because you are the owner of the issue, or you are following the issue.

1 0

[yt_analysis/yt] Install failure with >=cython-0.17 (issue #501)
by Kacper Kowalik Feb. 15, 2013

Feb. 15, 2013

--- you can reply above this line --- New issue 501: Install failure with >=cython-0.17 https://bitbucket.org/yt_analysis/yt/issue/501/install-failure-with-cython-… Kacper Kowalik: I have weird issue, probably related to >=cython-0.17: ``` #!bash cythonc:> yt/utilities/lib/grid_traversal.c Error compiling Cython file: ------------------------------------------------------------ ... np.float64_t right_edge[3] np.float64_t dds[3] np.float64_t idds[3] int dims[3] ctypedef void sample_function( ^ ------------------------------------------------------------ yt/utilities/lib/grid_traversal.pyx:70:0: 'sample_function' redeclared error: 1 errors while compiling 'yt/utilities/lib/grid_traversal.pyx' with Cython ``` Doing s/sample_function/sampler_function/ fixes it. PR will follow shortly. -- This is an issue notification from bitbucket.org. You are receiving this either because you are the owner of the issue, or you are following the issue.

1 0

[yt_analysis/yt] HydroMethod etc (issue #500)
by Matthew Turk Feb. 15, 2013

Feb. 15, 2013

--- you can reply above this line --- New issue 500: HydroMethod etc https://bitbucket.org/yt_analysis/yt/issue/500/hydromethod-etc Matthew Turk: All of the parameters `HydroMethod` `EOSType` `DualEnergyFormalism` are proxies for actual information about the dataset. We should make those actual pieces of information available, instead. And then, get rid of these. Responsible: MatthewTurk -- This is an issue notification from bitbucket.org. You are receiving this either because you are the owner of the issue, or you are following the issue.

1 0

Answer testing
by Matthew Turk Feb. 15, 2013

Feb. 15, 2013

Hi all, Currently we run all unit and answer tests on every push. Running answer testing, particularly with FLASH data, takes a long time. (Sidenote: FLASH in 2.x is particularly bad for large data objects, which answer testing does.) This will get worse once Orion and Chombo have been added. I think we're looking at up to 15 minutes per run. This has two effects: 1) Changes which break unit tests take longer to get reported. 2) We'll go through our ShiningPanda monthly allocation quite quickly. There are two solutions that I see: 1) Buy more time on ShiningPanda 2) Only run answer testing at most once a day, but still run unit tests on every push. My proposal is to go with #2 with the backup of #1 as needed. Can I get [+-]1 on this? -Matt

4 4

RFC: yt 2.5 changelog
by Matthew Turk Feb. 14, 2013

Feb. 14, 2013

Hi all, I've opened a pull request here: https://bitbucket.org/yt_analysis/yt-doc/pull-request/74/changelog-update/d… This includes everyone I was able to identify that was a new contributor as well as a changelog of improvements to the code. I did this by going through the pull request log, but I might have missed something. Please make suggestions for new additions or changes via comments in the pull request, either inline or at the top. Thanks, Matt

1 0

Fwd: Announcement of SciPy 2013 Themes
by Matthew Turk Feb. 13, 2013

Feb. 13, 2013

SciPy is always a really fun conference, and not just for the python aspects. Last year was really great and I came away feeling like I learned a lot -- not just about Python, but about a wide variety of scientific topics. And, I think if a couple of us go, we can probably have a fun Austin-based sprint. ---------- Forwarded message ---------- From: SciPy 2013 <scipy-organizers(a)scipy.org> Date: Wed, Feb 13, 2013 at 6:19 PM Subject: Announcement of SciPy 2013 Themes To: Matthew <matthewturk(a)gmail.com> Email not displaying correctly? View it in your browser.<http://us1.campaign-archive1.com/?u=e91b4574d5d1709a9dc4f7ab7&id=0230006689…> Scientific Computing with Python-Austin, Texas-June 24-29, 2013 *Greetings and News from the SciPy 2013<http://scipy.us1.list-manage.com/track/click?u=e91b4574d5d1709a9dc4f7ab7&id…>Planning Committee: * SciPy 2013 is about four months away, and we’ve been working hard to make this the best one yet. We are very excited to announce the themes of this year’s conference. The main conference themes which will be Machine Learning & Tools for Reproducible Science. Like last year, there will also be 4 mini-symposia, and thanks to your contributions to the poll, the following themes have been selected: - Meteorology, climatology, and atmospheric and oceanic science - Astronomy & astrophysics - Bio-informatics - Medical imaging Thanks to everyone who answered the survey on these themes. The participation was tremendous. Hope to see you in June! Andy Terrel, Co-chair Jonathan Rocher, Co-chair Katy Huff, Program Committee co-chair Matt McCormick, Program Committee co-chair Dharhas Pothina, Tutorial co-chair Francesc Alted, Tutorial co-chair Corran Webster, Sprint co-chair Peter Wang, Sprint co-chair Matthew Turk, BoF co-chair Jarrod Millman, Proceeding co-chair Stéfan van der Walt, Proceeding co-chair Anthony Scopatz, Community contact co-chair Majken Tranby, Community contact co-chair Jeff Daily, Financial Aid co-chair John Wiggins, Financial Aid co-chair Leah Jones, Operations chair Brett Murphy, Sponsor chair Bill Cowan, Financial chair You are receiving this email because you subscribed to the mailing list or registered for the SciPy 2010 or SciPy 2011 conference in Austin, TX. Unsubscribe<http://scipy.us1.list-manage1.com/unsubscribe?u=e91b4574d5d1709a9dc4f7ab7&i…> matthewturk(a)gmail.com from this list | Forward to a friend<http://us1.forward-to-friend.com/forward?u=e91b4574d5d1709a9dc4f7ab7&id=023…>| Update your profile<http://scipy.us1.list-manage.com/profile?u=e91b4574d5d1709a9dc4f7ab7&id=069…> *Our mailing address is:* Enthought, Inc. 515 Congress Ave. Austin, TX 78701 Add us to your address book<http://scipy.us1.list-manage.com/vcard?u=e91b4574d5d1709a9dc4f7ab7&id=069dc…> *Copyright (C) 2013 Enthought, Inc. All rights reserved.*

1 0

[yt_analysis/yt] Missing hierarchy functions (issue #499)
by Sam Skillman Feb. 13, 2013

Feb. 13, 2013

--- you can reply above this line --- New issue 499: Missing hierarchy functions https://bitbucket.org/yt_analysis/yt/issue/499/missing-hierarchy-functions Sam Skillman: Currently looks like some of the functions hanging off the hierarchy are missing. In 2.x: ``` #!text pf.h.find_field_value_at_point pf.h.find_max_cell_location pf.h.find_particles_by_type pf.h.find_points pf.h.find_slice_grids pf.h.find_max pf.h.find_min pf.h.find_point pf.h.find_ray_grids pf.h.find_sphere_grids ``` In 3.0: ``` #!text pf.h.find_max pf.h.find_max_cell_location pf.h.find_particles_by_type pf.h.find_points ``` -- This is an issue notification from bitbucket.org. You are receiving this either because you are the owner of the issue, or you are following the issue.

1 0

[yt_analysis/yt] Imports in blenders.py (issue #498)
by Nathan Goldbaum Feb. 13, 2013

Feb. 13, 2013

--- you can reply above this line --- New issue 498: Imports in blenders.py https://bitbucket.org/yt_analysis/yt/issue/498/imports-in-blenderspy Nathan Goldbaum: The enhance function, defined inside `yt.visualization.volume_rendering.blenders`, has a `from yt.mods import *` in it. This causes imports elsewhere in yt to break. Here's an example import that doesn't work anymore: ``` #!python >>> from yt.analysis_modules.cosmological_observation.light_cone.light_cone import LightCone ``` Responsible: samskillman -- This is an issue notification from bitbucket.org. You are receiving this either because you are the owner of the issue, or you are following the issue.

1 0

[yt_analysis/yt] Consistent docstrings (issue #497)
by Nathan Goldbaum Feb. 12, 2013

Feb. 12, 2013

--- you can reply above this line --- New issue 497: Consistent docstrings https://bitbucket.org/yt_analysis/yt/issue/497/consistent-docstrings Nathan Goldbaum: We need to rethink how we handle docstrings. Right now some classes have docstrings that live at the class level, some that live inside __init__, others don't have docstrings at all. This needs to be rationalized and needs to be done in a way that makes the api docs clearer and with more useful information. I've put myself down but if anyone else wants to contribute, please feel free. Responsible: ngoldbaum -- This is an issue notification from bitbucket.org. You are receiving this either because you are the owner of the issue, or you are following the issue.

1 0

FLASH data in 2.x versus 3.0
by Matthew Turk Feb. 12, 2013

Feb. 12, 2013

Hi all, Because of some testing/modification for the pull request that Andrew submitted, I tested some IO speeds for FLASH, which I decided to share here. What it came down to was testing a full dataset read, with slicing, versus constructing the H5S and only reading the 8x8 slab in question. I ran this with 30 iterations on the cluster merger FLASH dataset. Constructing a slice and only reading that in 2.x: 1.212s per slice Reading the full block dataset and discarding all but the slice of interest: 1.283s per slice In 3.0: 0.4856s per slice Mind you, seems like we have a big performance regression for finding the location of the most dense point, but that should be easily fixable. The vast majority of the savings come from pushing the selection of points *inside* the IO function, which lets us retain the H5D handles in between IO calls. -Matt

1 0

[yt_analysis/yt] Update to Mercurial 2.5.1 (issue #496)
by Matthew Turk Feb. 11, 2013

Feb. 11, 2013

--- you can reply above this line --- New issue 496: Update to Mercurial 2.5.1 https://bitbucket.org/yt_analysis/yt/issue/496/update-to-mercurial-251 Matthew Turk: We should update the install script to include Mercurial 2.5.1, and possibly put the evolve extension in yt-supplemental. Responsible: MatthewTurk -- This is an issue notification from bitbucket.org. You are receiving this either because you are the owner of the issue, or you are following the issue.

1 0

Testing new AMR kD-tree
by Matthew Turk Feb. 7, 2013

Feb. 7, 2013

Hi all, Sam's got a PR out for a new AMR kD-tree: https://bitbucket.org/yt_analysis/yt/pull-request/383/amrkdtree-rewrite-vol… The idea is to make things more flexible, easier to improve, and eventually maybe even move it into Cython. It also includes some really cool improvements to alpha channel blending and the like. (He had a demo in IRC the other day of a plaid background over which he's volume rendered.) This needs user-testing. He has added tests for nose. I think if we can get three more "approve" clicks, which I hope will be from actually having run the code, we can give it an accept. -Matt

1 1

[yt_analysis/yt] install_script builds lapack without -fPIC (issue #495)
by Kacper Kowalik Feb. 6, 2013

Feb. 6, 2013

--- you can reply above this line --- New issue 495: install_script builds lapack without -fPIC https://bitbucket.org/yt_analysis/yt/issue/495/install_script-builds-lapack… Kacper Kowalik: As reported on #yt by clovell numpy fails to link with lapack installed using `install_script.sh` with following error: ``` #!bash Installing numpy-1.6.1 (arguments: 'numpy-1.6.1 --fcompiler=gnu95') ******************************************** FAILURE REPORT: ******************************************** gcc: numpy/linalg/lapack_litemodule.c gcc: numpy/linalg/python_xerbla.c /usr/bin/gfortran -Wall -Wall -shared build/temp.linux-x86_64-2.7/numpy/linalg/lapack_litemodule.o build/temp.linux-x86_64-2.7/numpy/linalg/python_xerbla.o -L/home/chris/Documents/enzo/yt/src/lapack-3.4.2 -L/home/chris/Documents/enzo/yt/src/BLAS -Lbuild/temp.linux-x86_64-2.7 -llapack -lfblas -lgfortran -o build/lib.linux-x86_64-2.7/numpy/linalg/lapack_lite.so /usr/bin/ld: /home/chris/Documents/enzo/yt/src/lapack-3.4.2/liblapack.a(dgeev.o): relocation R_X86_64_32 against `.rodata' can not be used when making a shared object; recompile with -fPIC /home/chris/Documents/enzo/yt/src/lapack-3.4.2/liblapack.a: could not read symbols: Bad value collect2: ld returned 1 exit status ``` -- This is an issue notification from bitbucket.org. You are receiving this either because you are the owner of the issue, or you are following the issue.

1 0

[yt_analysis/yt] Provide answer testing coverage for all the code frontends. (issue #494)
by Nathan Goldbaum Feb. 1, 2013

Feb. 1, 2013

--- you can reply above this line --- New issue 494: Provide answer testing coverage for all the code frontends. https://bitbucket.org/yt_analysis/yt/issue/494/provide-answer-testing-cover… Nathan Goldbaum: Currently only the enzo frontends is tested via answer testing. We should use small and medium sized datsets from each code to construct a set of common consistency checks. Maybe we can even consolidate the answer testing and test all the codes simultaneously? -- This is an issue notification from bitbucket.org. You are receiving this either because you are the owner of the issue, or you are following the issue.

1 0

[yt_analysis/yt] Provide publicly-available sample datasets for all supported codes (issue #493)
by Cameron Hummels Feb. 1, 2013

Feb. 1, 2013

--- you can reply above this line --- New issue 493: Provide publicly-available sample datasets for all supported codes https://bitbucket.org/yt_analysis/yt/issue/493/provide-publicly-available-s… Cameron Hummels: Put some sample datasets for each supported code on the hub. Good for demonstration, testing, etc. Also, include all supported codes in answer testing framework. -- This is an issue notification from bitbucket.org. You are receiving this either because you are the owner of the issue, or you are following the issue.

1 0

[yt_analysis/yt] "suppressStreamLogging" doesn't work (issue #492)
by Anonymous Feb. 1, 2013

Feb. 1, 2013

--- you can reply above this line --- New issue 492: "suppressStreamLogging" doesn't work https://bitbucket.org/yt_analysis/yt/issue/492/suppressstreamlogging-doesnt… Anonymous: When I set from yt.config import ytcfg ytcfg["yt", "suppressStreamLogging"] = "True" it does suppress most yt output, but I get 'No handlers could be found for logger "yt"', once for every active processor. -- This is an issue notification from bitbucket.org. You are receiving this either because you are the owner of the issue, or you are following the issue.

1 0