libconfig wrapper testing
by Matthew Turk
Hi all,
I've added a wrapper for libconfig, along with a slightly modified
version of the (LGPL'd, credits and license all included) libconfig
source itself, to the current development tip. (The modification is
to force all integers to be read as 64-bit.) By default it is turned
off, because of an issue with locale setting for reading/writing
floating point values.
If I could get a few people, next time they update, to open up
yt/utilities/setup.py and change:
INSTALL_LIBCONFIG_WRAPPER = 0
to
INSTALL_LIBCONFIG_WRAPPER = 1
and just see if it builds, that would be extremely helpful. The issue
here is that it's not clear to me if the file "xlocale.h" is going to
be present on all systems; I've located it on all of the systems I've
tested on so far, but that is not many. If it doesn't work for you,
that'd be really good to know. Worse comes to worse and we can
disable the locale code completely, but that worries me a bit.
I'm in the process of documenting how to use the libconfig wrapper,
but for the next few days it's not being used by anything in yt.
Thanks for any testing ...
-Matt
7 years, 10 months
Maintenace on hg.enzotools.org
by Matthew Turk
Hi all,
I'm performing maintenance on hg.enzotools.org. Read access should
continue working; write might not. Either way, it might experience a
little up/down over the next hour or so.
-Matt
7 years, 10 months
Re: [Yt-dev] EnzoFOF merger tree
by Stephen Skory
Hi John & Matt,
I've successfully converted John's dot output to use pydot, and in the process cleaned the output up quite a bit. Like my merger tree, it will attempt to guess the kind of file you want from the suffix of the file name. So below will output a png directly if "dot" is in your path:
"""
from yt.mods import *
import yt.analysis_modules.halo_merger_tree.api as hm
tree = hm.EnzoFOFMergerTree((7.0,15.0), load_saved=True)
tree.build_tree(0, min_particles=500)
tree.write_dot(filename = "tree_00000.png")
"""
Cleaner new version:
http://screencast.com/t/ZrEpYqab
Older version, that was missing strict horizontal alignment between the halo nodes and redshift label nodes, and had unnecessary redshift nodes:
http://screencast.com/t/hnGom9VSf
If there are no objections, I'll push it, as well as my merger tree changes, and adding the pydot file (I've put it in yt.utilities). I'll also update the docs for my merger tree simultaneously. I've also updated the docstrings, where applicable.
Stephen Skory
stephenskory(a)yahoo.com
http://stephenskory.com/
510.621.3687 (google voice)
7 years, 10 months
Re: [Yt-dev] EnzoFOF merger tree
by Stephen Skory
Hi Matt,
> I'd say go for it, but could you describe a bit more what's going on?
> Does your new code (pydot based) communicate directly with graphviz?
> Does it do this via named pipes, is it pure python, does it use a
> shared library, etc etc?
The code does not communicate directly with graphviz. It makes a temporary file on disk using tempfile.mkstemp(), and then calls the chosen executable (dot, neato, etc...) using subprocess to make the final image. It is pure python, and is self-contained in one file under a MIT license, so we can add it to yt with no problems, I think. Does that explain what you're curious about?
Stephen Skory
stephenskory(a)yahoo.com
http://stephenskory.com/
510.621.3687 (google voice)
7 years, 10 months
Removing generated Cython code
by Matthew Turk
Hi everyone,
Last week we briefly talked about removing the generated post-Cython
.c and .cpp files from the repository. They have become something of
a burden to maintain, and the diffs simply get too large. I've taken
a stab at removing these, which I've placed in a bundle here:
http://yt.enzotools.org/files/cython-changes.hg
The changeset in question is bce1c775bb84 . This starts by removing
.c and .cpp files generated by Cython and adding the appropriate
information about how to generate the .c and .cpp files to the
setup.py files for RAMSES and the amr_utils directories.
Additionally, dependencies are included in these setup.py files.
However, to get Cython to play nicely with NumPy, I've had to add in a
routine that I found written by Matthew Brett. I updated it to include
some C++ support.
Additionally, as a safeguard for those systems were Cython is not
already included, I have added a brief routine that checks whether or
not Cython is installed. If it is not, it attempts to install it. This
will still fail on systems where superuser privs are required, but
that should not be the case in the majority of yt installations that
are not on OSX. On OSX this will require sudo-ing the installation.
I'd really appreciate if at least one or two people could unbundle
this and test out that "python2.6 setup.py build_ext -i" works, and it
would be *extra* helpful if you could try this on a machine where
Cython isn't installed, and check that the auto-installation of Cython
actually works. I've tested it a couple places and it works, but I
hope that others can test it as well.
Anyway, thanks for reading this far, and if you get a chance to test
that would be amazing. And if you do, please refrain from pushing
these changes until we get settled on how well they work. Hopefully
by making amr_utils.c a bit lighterweight we can be more free to make
changes to the Cython routines and people can get them faster.
Best,
Matt
7 years, 10 months
yt-2.0 Release Announcement
by j s oishi
Hi All,
We're proud to announce the release of yt version 2.0, an analysis and
visualization toolkit for Adaptive Mesh Refinement data. This release
features a complete reorganization of the yt internal codebase, a
number of bug fixes, and numerous improvements to the code base and
the documentation. This is also the first release since the
publication of the yt method paper in ApJS
(http://adsabs.harvard.edu/abs/2011ApJS..192....9T).
The major changes in 2.0 since version 1.7 (Released on June 27, 2010) include:
* Major reorganization of the codebase for speed, ease of
modification, and maintainability
* Re-organization of documentation and addition of Orientation Session
* Support for FLASH code
* Preliminary support for MAESTRO, CASTRO, ART, and RAMSES
(contributions welcome!)
* Perspective projection for volume rendering
* Exporting to Sunrise
* Preliminary particle rendering in volume rendering visualization
* Drastically improved parallel volume rendering, via kD-tree decomposition
* Simple merger tree calculation for FOF catalogs
* New and greatly expanded documentation, with a "source" button in
the API documentation (see
http://yt.enzotools.org/doc/reference/api/generated/yt.data_objects.data_...
for an example)
yt features native support for astrophysical data generated by several
codes, providing a natural and intuitive way to address physical
regions in space as well as processed data. Development has been
sponsored by NSF, DoE, and University funding.
If you are running an older version of yt, re-obtaining and re-running
the installation script should happily upgrade your installation. If
you are using an older version that is checked out from the mercurial
repository, executing "yt instinfo -u" should also perform the upgrade
appropriately. If you are still using an older version obtained from
the old (discontinued) SVN repository, you will need to get the
install script and reinstall.
yt is a Free and Open Source project, and we invite you to get
involved. For more information, join the yt-dev mailing list, or see
the hacking guidelines on the Wiki:
http://yt.enzotools.org/wiki/HackingGuidelines . We anticipate several
more releases in the 2.0 series as documentation and docstring
coverage progress. Future developments will include an overhaul of
the plotting system for intuitive generation of plots, a web-based
notebook for remote analysis, better time series support, and inline
analysis of running simulations.
Please forward this announcement to any individuals or mailing lists
you think may be interested.
Sincerely,
The yt development team:
Matthew Turk
Britton Smith
Jeff Oishi
Sam Skillman
Stephen Skory
John Wise
Chris Malone
John ZuHone
Christopher Moody
7 years, 10 months
Re: [Yt-dev] EnzoFOF merger tree
by Stephen Skory
Hi All,
I've converted my merger tree code to use pydot, and it seems to work well. Using it I was able to remove quite a bit of code, and add the ability to write the output directly to an image format. Looking around the teragrid world, all machines have 'dot' in the default path, which means that graphviz is supported to some extent on the systems. I also built graphviz from source on Triton with no difficulty. This means that direct-to-image output should work most places. The default behavior is still to a text file, which will always work in the absence of graphviz.
I'm not going to push my changes now, seeing as yt-2.0 is being released by Matt soon. Would John and Matt mind if I converted the FOF merger tree, too?
Stephen Skory
stephenskory(a)yahoo.com
http://stephenskory.com/
510.621.3687 (google voice)
7 years, 10 months
Re: [Yt-dev] Removing generated Cython code
by Stephen Skory
Hi Matt,
> I'd really appreciate if at least one or two people could unbundle
> this and test out that "python2.6 setup.py build_ext -i" works, and it
> would be *extra* helpful if you could try this on a machine where
> Cython isn't installed, and check that the auto-installation of Cython
> actually works. I've tested it a couple places and it works, but I
> hope that others can test it as well.
I built a fresh install of yt, commenting out the cython stuff in the install script, applied the changeset, and ran the command above. Cython was downloaded, did it stuff, and there were no errors reported. I guess that's a success!
Stephen Skory
stephenskory(a)yahoo.com
http://stephenskory.com/
510.621.3687 (google voice)
7 years, 10 months
Last call for 2.0
by Matthew Turk
Hi all,
On Monday, yt 2.0 will be going out. Inasmuch as a release is
important independent of the accompanying email, please go ahead and
either hold off on big changes or commit any last minute bug fixes.
The docs are now going to be rebuilt on a rolling schedule, so that's
less of a concern at this point. Docstrings and so on can be added
continually. The major changelist I have currently looks like:
* Major reorganization of the codebase for speed, ease of
modification, and maintainability
* Re-organization of documentation and addition of Orientation Session
* Support for the FLASH, CASTRO and MAESTRO codes
* Preliminary support for ART and RAMSES (contributions welcome!)
* Perspective projection for volume rendering
* Exporting to Sunrise
* Preliminary particle rendering in volume rendering visualization
* Drastically improved parallel volume rendering, via kD-tree decomposition
* Simple merger tree calculation for FOF catalogs
Any additions / changes?
Thanks everybody for your hard work!
-Matt
7 years, 11 months
Re: [Yt-dev] EnzoFOF merger tree
by Stephen Skory
Matt,
>
> Okay, sounds good. You've sold me, let's use pydot. I think we
> should still use something that can recurse down and call pydot
> appropriately, but this sounds fine to me.
>
I'll try to integrate pydot into the graphviz output for my merger tree this weekend. Perhaps that experience will fully inform me (us) of whether it's a good idea or not.
Stephen Skory
stephenskory(a)yahoo.com
http://stephenskory.com/
510.621.3687 (google voice)
7 years, 11 months