Mailman 3 March 2011 - yt-dev

Re: [Yt-dev] [yt-users] install problems
by Matthew Turk 12 Mar '11

12 Mar '11

(moved to yt-dev briefly) Hi all, I no longer have an OSX box to test on. But it looks like there are issues with the _locale.so module being built correctly on OSX 10.6. The build info for both Peng Oh and Emilio Romano-Diaz was: building '_locale' extension gcc-4.2 -fno-strict-aliasing -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I. -I/Users/peng/Dropbox/Dropbox_Docs/YT/yt-i386/src/Python-2.6.3/./Include -I/Users/peng/Dropbox/Dropbox_Docs/YT/yt-i386/src/Python-2.6.3/./Mac/Include -I/Users/peng/Dropbox/Dropbox_Docs/YT/yt-i386//include -I. -IInclude -I./Include -I/usr/local/include -I/Users/peng/Dropbox/Dropbox_Docs/YT/yt-i386/src/Python-2.6.3/Include -I/Users/peng/Dropbox/Dropbox_Docs/YT/yt-i386/src/Python-2.6.3 -c /Users/peng/Dropbox/Dropbox_Docs/YT/yt-i386/src/Python-2.6.3/Modules/_localemodule.c -o build/temp.macosx-10.4-i386-2.6/Users/peng/Dropbox/Dropbox_Docs/YT/yt-i386/src/Python-2.6.3/Modules/_localemodule.o gcc-4.2 -L/Users/peng/Dropbox/Dropbox_Docs/YT/yt-i386/lib/ -L/Users/peng/Dropbox/Dropbox_Docs/YT/yt-i386/lib64/ -L/Users/peng/Dropbox/Dropbox_Docs/YT/yt-i386/lib/ -L/Users/peng/Dropbox/Dropbox_Docs/YT/yt-i386/lib64/ -L/Users/peng/Dropbox/Dropbox_Docs/YT/yt-i386/lib/ -L/Users/peng/Dropbox/Dropbox_Docs/YT/yt-i386/lib64/ -L/Users/peng/Dropbox/Dropbox_Docs/YT/yt-i386/lib/ -L/Users/peng/Dropbox/Dropbox_Docs/YT/yt-i386/lib64/ -bundle -undefined dynamic_lookup -L/Users/peng/Dropbox/Dropbox_Docs/YT/yt-i386/lib/ -L/Users/peng/Dropbox/Dropbox_Docs/YT/yt-i386/lib64/ -L/Users/peng/Dropbox/Dropbox_Docs/YT/yt-i386/lib/ -L/Users/peng/Dropbox/Dropbox_Docs/YT/yt-i386/lib64/ -L/Users/peng/Dropbox/Dropbox_Docs/YT/yt-i386/lib/ -L/Users/peng/Dropbox/Dropbox_Docs/YT/yt-i386/lib64/ -L/Users/peng/Dropbox/Dropbox_Docs/YT/yt-i386/lib/ -L/Users/peng/Dropbox/Dropbox_Docs/YT/yt-i386/lib64/ build/temp.macosx-10.4-i386-2.6/Users/peng/Dropbox/Dropbox_Docs/YT/yt-i386/src/Python-2.6.3/Modules/_localemodule.o -L/Users/peng/Dropbox/Dropbox_Docs/YT/yt-i386//lib -L/Users/peng/Dropbox/Dropbox_Docs/YT/yt-i386/lib/ -L/usr/local/lib -o build/lib.macosx-10.4-i386-2.6/_locale.so -framework CoreFoundation ld: warning: directory '/Users/peng/Dropbox/Dropbox_Docs/YT/yt-i386/lib64/' following -L not found ld: warning: directory '/Users/peng/Dropbox/Dropbox_Docs/YT/yt-i386/lib64/' following -L not found ld: warning: directory '/Users/peng/Dropbox/Dropbox_Docs/YT/yt-i386/lib64/' following -L not found ld: warning: directory '/Users/peng/Dropbox/Dropbox_Docs/YT/yt-i386/lib64/' following -L not found ld: warning: directory '/Users/peng/Dropbox/Dropbox_Docs/YT/yt-i386/lib64/' following -L not found ld: warning: directory '/Users/peng/Dropbox/Dropbox_Docs/YT/yt-i386/lib64/' following -L not found ld: warning: directory '/Users/peng/Dropbox/Dropbox_Docs/YT/yt-i386/lib64/' following -L not found ld: warning: directory '/Users/peng/Dropbox/Dropbox_Docs/YT/yt-i386/lib64/' following -L not found *** WARNING: renaming "_locale" since importing it failed: dlopen(build/lib.macosx-10.4-i386-2.6/_locale.so, 2): Symbol not found: _libintl_bindtextdomain Referenced from: /Users/peng/Dropbox/Dropbox_Docs/YT/yt-i386/src/Python-2.6.3/build/lib.macosx-10.4-i386-2.6/_locale.so Expected in: dynamic lookup Could somebody with an OSX 10.6 box take a look, see if you can find the _libintl_bindtextdomain symbol? One of the posts I found online suggested that this issue was related to having a conflicting libintl.dylib somewhere in the LD path, but again, I can't replicate this anymore. If somebody (Stephen? Britton? John? Rick?) that has an OSX box can provide ANY help, could you follow up on yt-users with some ideas? Thanks, Matt On Fri, Mar 11, 2011 at 4:33 PM, Peng Oh <peng(a)physics.ucsb.edu> wrote: > > > Hello, > > I just tried installing YT on my machine (Mac OS X 10.6.6) and ran into some problems. I attach my install log. Any help would be much appreciated. > > Thanks! > Peng > _______________________________________________ > yt-users mailing list > yt-users(a)lists.spacepope.org > http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org > >

1 0

BitBucket mercurial extension
by Matthew Turk 09 Mar '11

09 Mar '11

Hey everybody, I just saw there's a bitbucket extension for hg. http://mercurial.selenic.com/wiki/BitbucketExtension https://groups.google.com/forum/#!topic/bitbucket-users/pXp6CtcpatQ You can get it and use it to create repos, look at forks, etc etc, by grabbing it: pip install lxml hg clone https://bitbucket.org/birkenfeld/hgbb And then you have to point to it in your .hgrc: [extensions] hgbb = /home/mturk/Packages/hgbb/hgbb.py If you read the top of hgbb.py you can see the config options, etc. Anyway, really cool -- especially looking at incoming/outgoing fork changesets! -Matt

1 0

Fwd: [Paraview] ANN: ParaView 3.10.0 Available for download.
by Matthew Turk 09 Mar '11

09 Mar '11

This may be of interest to some people here. I've visited with the ParaView developers, and hopefully in the very near future we'll have a few working examples of interoperability between ParaView and yt. There's overlap between the two tools, but there are also some pretty interesting and compelling places of synergy. -Matt ---------- Forwarded message ---------- From: David Partyka <david.partyka(a)kitware.com> Date: Wed, Mar 9, 2011 at 2:04 PM Subject: [Paraview] ANN: ParaView 3.10.0 Available for download. To: ParaView Developers <paraview-developers(a)paraview.org>, ParaView <paraview(a)paraview.org> Hello All, The ParaView team is pleased to announce the availability of the ParaView 3.10.0 final binaries for download on the ParaView download page. http://paraview.org/paraview/resources/software.html Release Notes: ParaView 3.10 This release features notable developments, including mechanisms to incorporate advanced rendering techniques, improved support for readers and several usability enhancements and bug fixes. For the 3.10 release, we have added 60 new readers. The new readers include: ANSYS, CGNS, Chombo, Dyna3D, Enzo, Mili, Miranda, Nastran, Pixie, Samrai, Silo, and Tecplot Binary. A full listing of supported readers can be found in the ParaView Users Guide. We also added the ability for developers to create ParaView reader plugins from previously developed VisIt reader plugins. You can find a full guide on how to do this on the VisIt Database Bridge: http://www.paraview.org/Wiki/VisIt_Database_Bridge With this release we have rewritten the ParaView User's Guide and are making it freely available for the first time. The complete guide can be obtained in the help system or online at: http://paraview.org/Wiki/ParaView/Users_Guide/Table_Of_Contents. We have included a Python-based calculator which makes it possible to write operations using Python. The Python calculator uses NumPy, which lets you use advanced functions such as gradients, curls, and divergence easily in expressions. Also the NumPy module is packaged in the ParaView binary and is importable from the ParaView Python shell. There should also be a marked performance improvement for users dealing with large multi-block datasets. We have cleaned up the rendering pipeline to better handle composite datasets, avoiding the appending of all blocks into a single dataset as was done previously. To better utilize multiple cores on modern multi-core machines, by default ParaView can now run using a parallel server, even for the built-in mode. This enables the use of all the cores for parallel data processing, without requiring the user to start a parallel server. ParaView binaries will also be distributed using an MPI implementation, making this feature available to users by simply downloading the binaries. Since this is an experimental feature, it is off by default, but users can turn it on by checking the Auto-MPI checkbox in the application settings dialog. Additionally, the 3.10 release includes several usability enhancements. 3D View now supports smart context menus, accessed by right-clicking on any object in the 3D View to change its color, representation, color map and visibility. Left-clicking on an object in the 3D View makes it active in the pipeline browser. Within the spreadsheet view, sorting is now supported and an advanced parallel sorting algorithm ensures that none of the benefits of the spreadsheet view, such as streaming and selection, are sacrificed. Python tracing and macro controls are no longer hidden on the Python shell dialog and instead are now easily found on the Tools menu. For developers interested in adding support for advanced multi-pass rendering algorithms to ParaView, this release includes a major refactoring of ParaView's rendering pipeline. View and representations have been redesigned and users should see improved performance in client-server mode from reduced interprocess communication during rendering. LANL's MantaView interactive ray tracing plugin has been restructured to make it easier to use. Version 2.0 of the plugin is now multi-view capable and no longer requires ParaView to be run in a client/server configuration. Similarly both of LANL's streaming aware ParaView derived applications have been merged into ParaView proper in the form of a new View plugin. The underlying streaming algorithms have been rewritten to be more usable and extensible. Both plugins are available in standard binary package for the first time in this release. For an exhaustive list of the new features and bug-fixes, please refer to the change log at: http://www.paraview.org/Bug/changelog_page.php. As always, we rely on your feedback to make ParaView better. Please use http://paraview.uservoice.com/ or click on the "Tell us what you think" link on paraview.org to leave your feedback and vote for new features. _______________________________________________ Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the ParaView Wiki at: http://paraview.org/Wiki/ParaView Follow this link to subscribe/unsubscribe: http://www.paraview.org/mailman/listinfo/paraview

1 0

Calling External Analysis Codes
by Matthew Turk 04 Mar '11

04 Mar '11

Hi everybody, I've been at the KITP Galaxy Clusters workshop today and yesterday, and after chatting with people here about a few fun ideas, I have started to realize that a very useful thing we could do is make it easier to call external, pre-existing analysis codes. To that end, last night I wrote up some very simple documentation that walks you through the idea of calling a C library using Cython. I wanted to throw it out there and see if anyone had any feedback. This is just a simple example, but I think it is pretty demonstrative. http://yt.enzotools.org/doc/advanced/external_analysis.html Anyway, I will probably write more later about this workshop; talking to some of the participants has been very informative and energizing. I'm presenting an intro/tutorial on yt this afternoon, as well, so hopefully that will garner some good feedback and ideas. Best, Matt

1 0

tree code for boundedness
by Stephen Skory 04 Mar '11

04 Mar '11

Hi all, I've been working with the clump finder, and am becoming impatient with the gravitationally bound test calculation. I'm a very important person! Ok, sorry, I'm back to serious now. I'm wondering if we think that implementing some kind of tree code or pixelation of cells to lower resolution levels would be practical and accurate enough? It would make the double loop much smaller and quicker, and there could be a single user-controlled parameter that sets the desired accuracy. There are even parallel tree codes out there that we may be able to crib from. Any thoughts would be appreciated! Stephen Skory stephenskory(a)yahoo.com http://stephenskory.com/ 510.621.3687 (google voice)

3 2

Re: [Yt-dev] moving issues and wiki --> BitBucket
by Stephen Skory 02 Mar '11

02 Mar '11

+1, provided that there is no regression in capability, which appears to not be the case. Stephen Skory stephenskory(a)yahoo.com http://stephenskory.com/ 510.621.3687 (google voice)

2 1

Re: [Yt-dev] clump finder problem
by Stephen Skory 01 Mar '11

01 Mar '11

Matt, > "hg bisect --skip" is the correct way to skip over changesets that > don't compile; I should have thought of that in my first mail. Good. I'll keep that in mind. Thanks! > The problematic changeset is, I think, c81ca2e61e6a. Can you try > using these files: > > yt/utilities/_amr_utils/ContourFinding.pyx > yt/analysis_modules/level_sets/contour_finder.py > > From changeset fd91f581b0e0? Use "hg cat -r fd91f581b0e0 ..." to get > 'em. Yup, replacing those two files from the earlier changeset does not create the error. Stephen Skory stephenskory(a)yahoo.com http://stephenskory.com/ 510.621.3687 (google voice)

1 0

Simulated Observations: Request for Corrections
by Matthew Turk 01 Mar '11

01 Mar '11

Hi all, I've been taking stock of where we are for simulated observations, and I wanted to run this past everyone to make sure I'm not missing anything. I think we have four mechanisms for generating something that could *generously* be called a "Simulated Observation," but for each approach there are caveats. If someone has something to add, in terms of capability or caveats, I would be eager to hear. I have not included any methods for generating spectra from galaxies, but I believe this may have been done using the second method below. = X-ray Observations = Under the assumption of optically thin gas, projections can be made using emissivity to generated simulated observations. yt includes a method for handling output from CLOUDY in the ROCO (Smith et al 2008) format, and generating integrated emissivity over given energy ranges. Caveats: The ROCO format for input requires some non-trivial handling of CLOUDY output. = SED Generation and Deposition = Using BC03 models for stellar population synthesis, star particles in a given calculation can be assigned an integrated flux for a specific bandpass. These fluxes can then be combined using either projections or volume rendering. This can use CIC interpolation to deposit a total flux into each cell (which should be flux-conserving, modulo a multiplicative factor not currently included) which is then either projected or volume rendered. Caveats: The deposition method produces far too washed out and murky results. The multiplicative factor is not currently set correctly universally. = Thermal Gas Emission = Applying a black body spectrum to the thermal content of the gas, we can volume render the domain and apply absorption based on broad arguments of scattering. One could theoretically include star particles as point sources in this, using recent changes to the volume renderer. Caveats: Scattering that results in re-emission is completely neglected, such as Halpha emission. Scattering that results in just attenuating the emission is set in an ad hoc fashion. Emission from point sources, if included at all, is included in a non-conservative fashion. = Export to Sunrise = Data can be exported to Sunrise for simulated observation generation. Caveats: This process is poorly documented. Does that all sound about right? Does anybody have any additions or corrections to this? Thanks, Matt

5 7

Re: [Yt-dev] clump finder problem
by Stephen Skory 01 Mar '11

01 Mar '11

Matt & Sam, using Sam's advice I get this: The first bad revision is: changeset: 3770:c81ca2e61e6a branch: yt user: Matthew Turk <matthewturk(a)gmail.com> date: Thu Feb 24 18:33:59 2011 -0800 summary: Adding field caching for inline runs; don't bother re-searching for them. And Matt's advice: % hg log --limit=2 yt/utilities/_amr_utils/ContourFinding.pyx changeset: 3770:c81ca2e61e6a branch: yt user: Matthew Turk <matthewturk(a)gmail.com> date: Thu Feb 24 18:33:59 2011 -0800 summary: Adding field caching for inline runs; don't bother re-searching for them. changeset: 3747:1b81f9d8847c branch: yt parent: 3744:73bbf5a5c797 user: Matthew Turk <matthewturk(a)gmail.com> date: Wed Feb 16 04:38:44 2011 -0800 summary: Moving the joining phase for contour finding into Cython so it looks like it's probably 3770. Thanks for looking into this, Matt! Stephen Skory stephenskory(a)yahoo.com http://stephenskory.com/ 510.621.3687 (google voice)

2 1

clump finder problem
by Stephen Skory 01 Mar '11

01 Mar '11

Hi All, using a script like this <http://paste.enzotools.org/show/1519/> I am getting this error below with 3782:f9c66b2cacfc. File "/duanestorage/home/faculty/stsk4691/plot_clumps.py", line 26, in <module> find_clumps(master_clump, c_min, c_max, step) File "/duanestorage/home/faculty/stsk4691/yt/src/yt-hg/yt/analysis_modules/level_sets/clump_handling.py", line 170, in find_clumps clump.find_children(min_val) File "/duanestorage/home/faculty/stsk4691/yt/src/yt-hg/yt/analysis_modules/level_sets/clump_handling.py", line 117, in find_children self.cached_fields) File "/duanestorage/home/faculty/stsk4691/yt/src/yt-hg/yt/analysis_modules/level_sets/contour_finder.py", line 303, in identify_contours boundary_tree = amr_utils.construct_boundary_relationships(fd) File "ContourFinding.pyx", line 92, in yt.utilities.amr_utils.construct_boundary_relationships (yt/utilities/amr_utils.c:26719) IndexError: Out of bounds on buffer access (axis 0) However, I do not get the bug with 3746:1b81f9d8847c, so I ran "hg bisect" in that range. However, part of the way through the bisect process a new bug was introduced: File "/duanestorage/home/faculty/stsk4691/yt/lib/python2.6/site-packages/yt-2.1dev-py2.6-linux-x86_64.egg/yt/visualization/api.py", line 59, in <module> from streamlines import \ ImportError: cannot import name StreamLines which polluted the bisect process, I think. The revision that bisect identified as being bad is this one: The first bad revision is: changeset: 3766:20a49bc35fbd branch: yt parent: 3763:badcec4be5c9 user: Sam Skillman <sam.skillman(a)gmail.com> date: Thu Feb 24 19:18:17 2011 -0700 summary: Adding an initial go at streamlines. It uses the amr kD-tree to But like I said, this might not be the culprit. Can anyone think of what they might have done to clump finding (contour_finder.py) in that revision range that could make this first bug above? Thanks! Stephen Skory stephenskory(a)yahoo.com http://stephenskory.com/ 510.621.3687 (google voice)

3 2