Mailman 3 January 2011 - yt-dev

libconfig wrapper testing
by Matthew Turk 31 Jan '11

31 Jan '11

Hi all, I've added a wrapper for libconfig, along with a slightly modified version of the (LGPL'd, credits and license all included) libconfig source itself, to the current development tip. (The modification is to force all integers to be read as 64-bit.) By default it is turned off, because of an issue with locale setting for reading/writing floating point values. If I could get a few people, next time they update, to open up yt/utilities/setup.py and change: INSTALL_LIBCONFIG_WRAPPER = 0 to INSTALL_LIBCONFIG_WRAPPER = 1 and just see if it builds, that would be extremely helpful. The issue here is that it's not clear to me if the file "xlocale.h" is going to be present on all systems; I've located it on all of the systems I've tested on so far, but that is not many. If it doesn't work for you, that'd be really good to know. Worse comes to worse and we can disable the locale code completely, but that worries me a bit. I'm in the process of documenting how to use the libconfig wrapper, but for the next few days it's not being used by anything in yt. Thanks for any testing ... -Matt

4 8

Maintenace on hg.enzotools.org
by Matthew Turk 18 Jan '11

18 Jan '11

Hi all, I'm performing maintenance on hg.enzotools.org. Read access should continue working; write might not. Either way, it might experience a little up/down over the next hour or so. -Matt

1 0

Re: [Yt-dev] EnzoFOF merger tree
by Stephen Skory 18 Jan '11

18 Jan '11

Hi John & Matt, I've successfully converted John's dot output to use pydot, and in the process cleaned the output up quite a bit. Like my merger tree, it will attempt to guess the kind of file you want from the suffix of the file name. So below will output a png directly if "dot" is in your path: """ from yt.mods import * import yt.analysis_modules.halo_merger_tree.api as hm tree = hm.EnzoFOFMergerTree((7.0,15.0), load_saved=True) tree.build_tree(0, min_particles=500) tree.write_dot(filename = "tree_00000.png") """ Cleaner new version: http://screencast.com/t/ZrEpYqab Older version, that was missing strict horizontal alignment between the halo nodes and redshift label nodes, and had unnecessary redshift nodes: http://screencast.com/t/hnGom9VSf If there are no objections, I'll push it, as well as my merger tree changes, and adding the pydot file (I've put it in yt.utilities). I'll also update the docs for my merger tree simultaneously. I've also updated the docstrings, where applicable. Stephen Skory stephenskory(a)yahoo.com http://stephenskory.com/ 510.621.3687 (google voice)

3 2

Re: [Yt-dev] EnzoFOF merger tree
by Stephen Skory 17 Jan '11

17 Jan '11

Hi Matt, > I'd say go for it, but could you describe a bit more what's going on? > Does your new code (pydot based) communicate directly with graphviz? > Does it do this via named pipes, is it pure python, does it use a > shared library, etc etc? The code does not communicate directly with graphviz. It makes a temporary file on disk using tempfile.mkstemp(), and then calls the chosen executable (dot, neato, etc...) using subprocess to make the final image. It is pure python, and is self-contained in one file under a MIT license, so we can add it to yt with no problems, I think. Does that explain what you're curious about? Stephen Skory stephenskory(a)yahoo.com http://stephenskory.com/ 510.621.3687 (google voice)

2 1

Removing generated Cython code
by Matthew Turk 17 Jan '11

17 Jan '11

Hi everyone, Last week we briefly talked about removing the generated post-Cython .c and .cpp files from the repository. They have become something of a burden to maintain, and the diffs simply get too large. I've taken a stab at removing these, which I've placed in a bundle here: http://yt.enzotools.org/files/cython-changes.hg The changeset in question is bce1c775bb84 . This starts by removing .c and .cpp files generated by Cython and adding the appropriate information about how to generate the .c and .cpp files to the setup.py files for RAMSES and the amr_utils directories. Additionally, dependencies are included in these setup.py files. However, to get Cython to play nicely with NumPy, I've had to add in a routine that I found written by Matthew Brett. I updated it to include some C++ support. Additionally, as a safeguard for those systems were Cython is not already included, I have added a brief routine that checks whether or not Cython is installed. If it is not, it attempts to install it. This will still fail on systems where superuser privs are required, but that should not be the case in the majority of yt installations that are not on OSX. On OSX this will require sudo-ing the installation. I'd really appreciate if at least one or two people could unbundle this and test out that "python2.6 setup.py build_ext -i" works, and it would be *extra* helpful if you could try this on a machine where Cython isn't installed, and check that the auto-installation of Cython actually works. I've tested it a couple places and it works, but I hope that others can test it as well. Anyway, thanks for reading this far, and if you get a chance to test that would be amazing. And if you do, please refrain from pushing these changes until we get settled on how well they work. Hopefully by making amr_utils.c a bit lighterweight we can be more free to make changes to the Cython routines and people can get them faster. Best, Matt

2 1

yt-2.0 Release Announcement
by j s oishi 17 Jan '11

17 Jan '11

Hi All, We're proud to announce the release of yt version 2.0, an analysis and visualization toolkit for Adaptive Mesh Refinement data. This release features a complete reorganization of the yt internal codebase, a number of bug fixes, and numerous improvements to the code base and the documentation. This is also the first release since the publication of the yt method paper in ApJS (http://adsabs.harvard.edu/abs/2011ApJS..192....9T). The major changes in 2.0 since version 1.7 (Released on June 27, 2010) include: * Major reorganization of the codebase for speed, ease of modification, and maintainability * Re-organization of documentation and addition of Orientation Session * Support for FLASH code * Preliminary support for MAESTRO, CASTRO, ART, and RAMSES (contributions welcome!) * Perspective projection for volume rendering * Exporting to Sunrise * Preliminary particle rendering in volume rendering visualization * Drastically improved parallel volume rendering, via kD-tree decomposition * Simple merger tree calculation for FOF catalogs * New and greatly expanded documentation, with a "source" button in the API documentation (see http://yt.enzotools.org/doc/reference/api/generated/yt.data_objects.data_co… for an example) yt features native support for astrophysical data generated by several codes, providing a natural and intuitive way to address physical regions in space as well as processed data. Development has been sponsored by NSF, DoE, and University funding. If you are running an older version of yt, re-obtaining and re-running the installation script should happily upgrade your installation. If you are using an older version that is checked out from the mercurial repository, executing "yt instinfo -u" should also perform the upgrade appropriately. If you are still using an older version obtained from the old (discontinued) SVN repository, you will need to get the install script and reinstall. yt is a Free and Open Source project, and we invite you to get involved. For more information, join the yt-dev mailing list, or see the hacking guidelines on the Wiki: http://yt.enzotools.org/wiki/HackingGuidelines . We anticipate several more releases in the 2.0 series as documentation and docstring coverage progress. Future developments will include an overhaul of the plotting system for intuitive generation of plots, a web-based notebook for remote analysis, better time series support, and inline analysis of running simulations. Please forward this announcement to any individuals or mailing lists you think may be interested. Sincerely, The yt development team: Matthew Turk Britton Smith Jeff Oishi Sam Skillman Stephen Skory John Wise Chris Malone John ZuHone Christopher Moody

1 0

Re: [Yt-dev] EnzoFOF merger tree
by Stephen Skory 17 Jan '11

17 Jan '11

Hi All, I've converted my merger tree code to use pydot, and it seems to work well. Using it I was able to remove quite a bit of code, and add the ability to write the output directly to an image format. Looking around the teragrid world, all machines have 'dot' in the default path, which means that graphviz is supported to some extent on the systems. I also built graphviz from source on Triton with no difficulty. This means that direct-to-image output should work most places. The default behavior is still to a text file, which will always work in the absence of graphviz. I'm not going to push my changes now, seeing as yt-2.0 is being released by Matt soon. Would John and Matt mind if I converted the FOF merger tree, too? Stephen Skory stephenskory(a)yahoo.com http://stephenskory.com/ 510.621.3687 (google voice)

2 1

Re: [Yt-dev] Removing generated Cython code
by Stephen Skory 17 Jan '11

17 Jan '11

Hi Matt, > I'd really appreciate if at least one or two people could unbundle > this and test out that "python2.6 setup.py build_ext -i" works, and it > would be *extra* helpful if you could try this on a machine where > Cython isn't installed, and check that the auto-installation of Cython > actually works. I've tested it a couple places and it works, but I > hope that others can test it as well. I built a fresh install of yt, commenting out the cython stuff in the install script, applied the changeset, and ran the command above. Cython was downloaded, did it stuff, and there were no errors reported. I guess that's a success! Stephen Skory stephenskory(a)yahoo.com http://stephenskory.com/ 510.621.3687 (google voice)

1 0

Last call for 2.0
by Matthew Turk 16 Jan '11

16 Jan '11

Hi all, On Monday, yt 2.0 will be going out. Inasmuch as a release is important independent of the accompanying email, please go ahead and either hold off on big changes or commit any last minute bug fixes. The docs are now going to be rebuilt on a rolling schedule, so that's less of a concern at this point. Docstrings and so on can be added continually. The major changelist I have currently looks like: * Major reorganization of the codebase for speed, ease of modification, and maintainability * Re-organization of documentation and addition of Orientation Session * Support for the FLASH, CASTRO and MAESTRO codes * Preliminary support for ART and RAMSES (contributions welcome!) * Perspective projection for volume rendering * Exporting to Sunrise * Preliminary particle rendering in volume rendering visualization * Drastically improved parallel volume rendering, via kD-tree decomposition * Simple merger tree calculation for FOF catalogs Any additions / changes? Thanks everybody for your hard work! -Matt

1 0

Re: [Yt-dev] EnzoFOF merger tree
by Stephen Skory 15 Jan '11

15 Jan '11

Matt, > > Okay, sounds good. You've sold me, let's use pydot. I think we > should still use something that can recurse down and call pydot > appropriately, but this sounds fine to me. > I'll try to integrate pydot into the graphviz output for my merger tree this weekend. Perhaps that experience will fully inform me (us) of whether it's a good idea or not. Stephen Skory stephenskory(a)yahoo.com http://stephenskory.com/ 510.621.3687 (google voice)

1 0