Mailman 3 October 2009 - yt-dev

yt on dimensionless data
by j s oishi 28 Oct '09

28 Oct '09

Hi, Does anyone have any tips for using yt on dimensionless data? I am not using enzo, so some of the units stuff I'll have to hack into Chombo/Orion, but does anyone have any example scripts of how to get yt to work in dimensionless units in enzo? They would be most helpful. thanks, jeff

2 1

BinnedProfile issue
by Matthew Turk 26 Oct '09

26 Oct '09

Hi guys, I've been running into the issue where the rightmost bin of a BinnedProfile1D dangles and is always empty; this is a result of having one too many bins, and right-edge definitions of the values. I think this patch fixes it, wherein all the internal references -- the stuff that does the binning -- accesses .data but the __getitem__ returns [:-1]. I believe something similar should work for BinnedProfile2D. Sam, Britton, anybody -- can you test this out, make sure that it works when you put it through its paces? All my tests have worked. http://paste.enzotools.org/show/246/ -Matt

1 0

yt now loads chombo data
by j s oishi 23 Oct '09

23 Oct '09

Hi all, I don't know if this is useful to anyone here, but Matt and I were able to add Chombo support to yt. It's still super alpha, but it works. Attached is a slice through a test dataset ran by PS Li here at Berkeley. The code is in the mercurial repository under the "chombo" branch. I hope to move it back to the main yt branch (in mercurial) within the next week or so. Chombo's datafile format uses a single hdf5 datafile for all processors, levels, and grids, so it may be of some interest for parallel IO. take care, j

1 0

trunk on kraken
by Stephen Skory 22 Oct '09

22 Oct '09

FYI, I updated the install of yt on kraken to be current just now. /lustre/scratch/proj/yt_common/trunk _______________________________________________________ sskory(a)physics.ucsd.edu o__ Stephen Skory http://physics.ucsd.edu/~sskory/ _.>/ _Graduate Student ________________________________(_)_\(_)_______________

4 11

Stage two of branch consolidation
by Matthew Turk 20 Oct '09

20 Oct '09

Hi again, I've consolidated the yt and memory-optimization branches in the yt repo in mercurial. If you want to use either before this merger, the last changeset hashes before the merger are: 97e2160d876b (memory-optimization) ab99adbbe406 (yt) The new 'yt' branch now contains the massively parallel halo finder from Stephen Skory, as well as a fully rewritten hierarchy class. This will become the new dev trunk relatively soon. If you are concerned about changes in it, please clone it and test it. There were two bugs I was notified of in the svn trunk after last night's merger that I have now fixed. If you find a problem in this branch, you can either fix it or add it as a unit test to tests/test_*.py. -Matt

1 0

Merging mercurial back into svn trunk
by Matthew Turk 20 Oct '09

20 Oct '09

Hi everyone, I've merged the mercurial branch 'yt' back into the subversion trunk. There are two sets of changes I did not merge, because they are from other contributors -- I did merge a few of Sam's changes, which I hope he's okay with, because they are part of my own. However, I didn't merge either: yt/extensions/EnzoSimulation.py by Britton (wasn't sure if this was up to date or out of date) yt/lagos/PointsInVolume.* by John (substantial amount of code of yours) yt/lagos/HierarchyType.py by John (the find_particles_by_type) Neither of these should take more than one or two minutes to handle... You can see the diff by looking at the output of "hg diff -B -r trunk" when you're in the latest head of the 'yt' repo. This merger includes the VTK and time series stuff, as those are both going to be big parts of the 2.0 release. Rapid development will still take place in hg, but it had been too long since I brought it back. Let me know if you have any problems with the trunk following this big commit (r1485 for all the python, r1486 for the Cython-ized C code). The second stage of this will be to merge the hierarchy rewrite from the memory-optimization branch of mercurial. The third stage will be a reorganization of the source tree. -Matt

3 3

yt fof question
by Britton Smith 15 Oct '09

15 Oct '09

Hi everyone, I think this question is mainly for Stephen and possibly John, if he's on this list. I was trying to make a direct comparison between the halos found with yt FOF and with John Wise's FOF for the same simulation. The mass functions definitely look the same, but the particle membership for any given halo is totally different. I'm sure this is just a matter of the particle indexing. Does yt FOF use the particle IDs from enzo, or some different system? Any ideas for how I might correct this issue would be greatly appreciated. Thanks, Britton

4 7

yt-1.5 release
by Matthew Turk 08 Oct '09

08 Oct '09

Hi everyone, Although I'm sure it's mostly dropped off everyone's radar, the yt-1.5 release will happen in the next few days; even though the community is small, and it's a niche thing, I want to put a stamp on SOMEthing and say, hey, this is Sort Of Done. This is also a good reason to email the lists and bring up that the docs have been rewritten and that development is still ongoing -- and it also is a good time to start doing the reorganization and cruft cleaning the source tree so desperately needs. I'm leaning toward releasing Friday if possible, Tuesday if not. The docs have been almost completely rewritten, there's a whole new cookbook there and online, lots of bugs have been fixed, the docstrings are ready, and we're now shipping Stephen's parallel HOP (v1.0) and Britton's lightcone and halo profiling extensions. The docs now also have comments, the methods section of my thesis, and better narrative documentation with better structure for the API documentation. I'm sending this email out to show the release announcement and the new docs and get feedback/comments from everyone before they go out. I'd prefer if you didn't leave comments on the site just yet, because the URLs are still in flux, and instead either reply to this message or email me privately. I am going to spend a little bit of time tomorrow also adding in more cross-references to the methods section, which should be peppered throughout. The new docs can be found here: http://yt.enzotools.org/newdoc (pdf, 172 pages, several artifacts&problems: http://yt.enzotools.org/yt_manual.pdf ) and the release announcement is here: -- yt Version 1.5 Release Announcement We're proud to announce the release of yt version 1.5, an analysis and visualization toolkit for Adaptive Mesh Refinement data. yt features native support for Enzo (http://lca.ucsd.edu/projects/enzo/) data, providing a natural and intuitive way to address physical regions in space as well as processed data. Version 1.5 features many new improvements, most prominently that of the addition of parallel computing abilities and generalization for multiple AMR data formats. Additional improvements include: * Fully rewritten documentation, featuring a new cookbook with downloadable recipes and commenting facilities * Fully parallel slices, projections, cutting planes, profiles, quantities * Parallel HOP halo finder * Friends-of-friends halo finder * Object storage and serialization * Major performance improvements to the clump finder (factor of five) * Generalized domain sizes * Generalized field info containers * Dark Matter-only simulation support * 1D and 2D simulation support * Better IO for HDF5 sets * Support for the Orion AMR code * Spherical re-gridding of data * Automated halo profiler and imager * Limited non-axially aligned projection support ("disk stacking") * Light cone generator * Callback interface improved * Several new plot overlay callbacks * New data objects -- ortho and non-ortho rays, limited ray-tracing * Fixed resolution buffers * Spectral integrator for CLOUDY data * Substantially better interactive interface * Performance improvements *everywhere* * Command-line interface to *many* common tasks * Isolated plot handling, independent of PlotCollections Installation instructions, documentation, recipes, mailing list info and assorted other items can be found at the website, http://yt.enzotools.org/. This release is the product of a year of work by the development team, and we are very proud of the results and happy with what we have to offer. Thanks very much! Sincerely, The yt development team: Matthew Turk Britton Smith Jeff Oishi Stephen Skory Sam Skillman Devin Silvia John Wise David Collins -- Thoughts on either the announcement or the documentation, or, heck, even the yt-1.5 codebase? (branches/yt-1.5) Thanks! -Matt

1 0

HaloProfiler overhaul
by Britton Smith 05 Oct '09

05 Oct '09

Greetings, I have just finished an extensive overhaul of the HaloProfiler that includes a bunch of new features, as well as completely changes the running syntax. Other than the addition of various new features, the main goals of this overhaul were: 1) to make the tool more general (mostly by removing hard-coded filtering by virial quantities), essentially allowing someone to profile a random list of points that may or may not be actual halos Easy filtering of halos by virial quantities remains an option, but is now simply a specific instance of something more general and far more powerful: a halo filtering device that allow the user to create their own filter functions to comb through radial profile data and decide whether a halo meets certain criteria. 2) to remove the dependence on an additional parameter file for the various options. The parameter file is now gone. Most of the HaloProfiler parameters have been turned into instantiation keyword args. The keyword list is now considerably longer, but the benefit is that the number of files needed to run this thing has been reduced from 2 (running script and par file) to just 1 (running script). There are a large number of keywords options that are specific to either the profile or projection routines that are taken in at instantiation and stored as attributes. This was done to keep those function calls simple. I'm curious to know peoples' thoughts on whether these keyword args should stay put or move to the individual function calls. Adding fields for profiling and projections have been moved to functions addProfile and addProjection. Here is a brief list of the new features that I can remember: - the halo list read routine can be easily customized to read in columned ascii data of varying formats through the use of a dictionary (see attribute self.halo_list_format) - ability to change the function call, args, and kwargs for the halo finder Profiles: - filter halo list with user-written filter functions (see new file HaloFilters.py for an example of a function to filter based on virial quantities) - extract and output scalar quantities from halos using the filter functions - pre-filter halos based on values in the initial halo list (skipping the profiling altogether and saving a load of time) Projections: - choose the axes to be projected (instead of hard-coded to all three) - easily select the list of halos to be projected (the total list, filtered list, a new file, or an actual list) Before I commit all this, I'd like a little feedback, mostly on the migration of the par file parameters to keyword args. I put three files in the pastebin for people to download. I chose not to submit a diff since the changes were so sweeping. The new HaloProfiler.py: http://paste.enzotools.org/show/178/ HaloFilters.py (should go in yt/extensions with HaloProfiler.py) http://paste.enzotools.org/show/179/ runHaloProfiler.py (an example script to run the new HaloProfiler) http://paste.enzotools.org/show/180/ I will try my best to write up full documentation for this as soon as possible, perhaps even today. Please let me know what you think. Britton

3 5