Mailman 3 - yt-dev - python.org

Fwd: [astro-viz] AAS Splinter Meeting on Astro Visualization Metadata/VAMP
by Matthew Turk 17 Dec '09

17 Dec '09

Anybody at AAS might want to check this out. ---------- Forwarded message ---------- From: Robert Hurt <hurt(a)ipac.caltech.edu> Date: Wed, Dec 16, 2009 at 5:20 PM Subject: [astro-viz] AAS Splinter Meeting on Astro Visualization Metadata/VAMP To: astro-viz(a)yahoogroups.com While it has been a bit quiet on the astronomy metadata front and the Virtual Astronomy Mulitmedia Project (VAMP), the state of the art has actually been progressing rapidly and today there are many new tools to assist in adding astronomy-specific metadata to image galleries. The January AAS meeting provides a perfect opportunity to bring together anyone involved in astronomical visualization, imagery, and outreach to discuss tools, techniques, and advancements in the use of Astronomy Visualization Metadata (AVM). So if you are going to be at the AAS, please do set aside time to join us for the splinter session: Tuesday, January 5th, 5:30-7:00 PM Harding Room The meeting will be an informal discussion between interested parties with a goal of bringing everyone up to speed of what is being done, which projects are starting to implement AVM, what kind of funding has been granted, and what tools and workflows exist. If you have anything to contribute, please come prepared to chat about any of these topics, with or without slides. We will have a projector and screen for our own use (do not have to go through the AAS Speaker's Room). As a quick teaser of some of the topics, here is an incomplete list of some of the latest developments: - CS4 compatible tagging tools - Support in Microsoft's WorldWide Telescope (stand-alone and web clients) - New Spitzer website built on AVM specifications - PinpointWCS tool from Chandra for recovering WCS data from reference FITS images For more about AVM/VAMP: www.virtualastronomy.org Cheers, Robert Hurt | Dr. Robert L. Hurt | Spitzer Science Center/Public Affairs | Caltech MS 220-6 | 626-395-1825 (office) 626-568-0673 (fax) | Pasadena, CA 91125 | http://www.spitzer.caltech.edu __._,_.___ Reply to sender<hurt@ipac.caltech.edu?subject=AAS%20Splinter%20Meeting%20on%20Astro%20Visualization%20Metadata/VAMP>| Reply to group<astro-viz@yahoogroups.com?subject=AAS%20Splinter%20Meeting%20on%20Astro%20Visualization%20Metadata/VAMP> Messages in this topic<http://groups.yahoo.com/group/astro-viz/message/344;_ylc=X3oDMTMzOTkxbmxnBF…>( 1) Recent Activity: Visit Your Group<http://groups.yahoo.com/group/astro-viz;_ylc=X3oDMTJmaDB0Z3E3BF9TAzk3MzU5Nz…> Start a New Topic<http://groups.yahoo.com/group/astro-viz/post;_ylc=X3oDMTJmcDQ4OThwBF9TAzk3M…> MARKETPLACE Going Green: Your Yahoo! Groups resource for green living<http://us.ard.yahoo.com/SIG=14kdpqvdi/M=493064.13814333.13821539.13298430/D…> ------------------------------ Mom Power: Discover the community of moms doing more for their families, for the world and for each other<http://us.ard.yahoo.com/SIG=14kncdmj1/M=493064.13814537.13821737.10835568/D…> [image: Yahoo! Groups]<http://groups.yahoo.com/;_ylc=X3oDMTJlMTNrYmRpBF9TAzk3MzU5NzE0BGdycElkAzE0N…> Switch to: Text-Only<astro-viz-traditional@yahoogroups.com?subject=Change%20Delivery%20Format:%20Traditional>, Daily Digest<astro-viz-digest@yahoogroups.com?subject=Email%20Delivery:%20Digest>• Unsubscribe <astro-viz-unsubscribe(a)yahoogroups.com?subject=Unsubscribe> • Terms of Use <http://docs.yahoo.com/info/terms/> . __,_._,___

1 0

Star Spectrum Code
by Stephen Skory 16 Dec '09

16 Dec '09

Hi all, I've added code to yt.extensions.StarAnalysis (in hg, of course) to calculate the spectral flux and spectral energy distribution for a collection of stars. This is based on code from Ken Nagamine, and it uses the data tables provided by Bruzual & Charlot. You can pass it a data_source, or arrays containing the pertinent information (star mass, creation time and metallicity) and it will build you a spectrum. The rub is in the B&C data tables. They are distributed as ASCII files from an unreliable foreign server. The data required to build a spectrum is at least 150MB in ASCII format and takes a looong time to download. And then once you have it, reading ASCII data in is super sloooow. I have converted the data to HDF5 files which are nice and fast and only 72MB in size in total. However, I'm not sure exactly if we can distribute them as HDF5 files. I am going to email B&C and ask them if we can, with proper acknowledgement, of course. Otherwise, I'll have to include my (currently ridiculously slow) conversion script with any instructions on how to use the spectrum code. I'll let you guys know! _______________________________________________________ sskory(a)physics.ucsd.edu o__ Stephen Skory http://physics.ucsd.edu/~sskory/ _.>/ _Graduate Student ________________________________(_)_\(_)_______________

3 7

SciPy 2009 proceedings
by Matthew Turk 16 Dec '09

16 Dec '09

Hi everyone, the proceedings for SciPy 2009 are out: http://conference.scipy.org/proceedings/SciPy2009/ including a paper on FEMhub, a paper on Sherpa (used with Chandra), and CorePy (an assembly language programming environment in Python). There's also a Cython tutorial and a paper on fast computation with Cython. -Matt

1 0

HaloFinding change
by Stephen Skory 15 Dec '09

15 Dec '09

Hi all, I've changed the way haloes are sorted in HaloFinding.py in hg. Now they are sorted by mass, rather than by number of particles. I figure this makes more sense, especially when dealing with nested cosmology and different DM particle masses. I hope no one finds this objectionable. Feel free to put me in my place if you feel it is! _______________________________________________________ sskory(a)physics.ucsd.edu o__ Stephen Skory http://physics.ucsd.edu/~sskory/ _.>/ _Graduate Student ________________________________(_)_\(_)_______________

2 1

a few new features added to hg
by Stephen Skory 12 Dec '09

12 Dec '09

Hi all, I wanted to make you all aware of a few new features I just added to the devel repo. For a given data_source, like 'sp = pf.h.sphere([0.5]*3, .2)', you can get the volume of the region in your favorite units: 'sp.volume('mpc')'. No units gives it in code units 'sp.volume()'. I added yt.extensions.StarAnalysis, which has a class StarFormationRate, which when given a data_source with stars in it, will calculate various star formation rates binned over time. Let me know if you have any questions! And yes, I should eventually document the StarAnalysis stuff! _______________________________________________________ sskory(a)physics.ucsd.edu o__ Stephen Skory http://physics.ucsd.edu/~sskory/ _.>/ _Graduate Student ________________________________(_)_\(_)_______________

2 1

dm only viriral masses
by Stephen Skory 11 Dec '09

11 Dec '09

Hi all, I wrote a simple script that finds the virial masses of DM only haloes in yt. My thought is that this could be easily added to Britton's HaloProfiler and selected with a dm_only flag. What does everyone think? Thanks! _______________________________________________________ sskory(a)physics.ucsd.edu o__ Stephen Skory http://physics.ucsd.edu/~sskory/ _.>/ _Graduate Student ________________________________(_)_\(_)_______________

2 4

more non-unique particles in enzo and how to deal with them
by Britton Smith 05 Dec '09

05 Dec '09

Hi everyone, Stephen's latest update of his new parallel hop to check for unique-ness of particle indices has shed light on another problem. In a recent simulation run with the devel/trunk version of enzo, I have found the particle numbers are no longer unique when star particles are generated. Originally, the star particle indices by adding to the number of dark matter particles, such that if you had N dm particles and M star particles, the dm particle indices when from 0 to N-1 and the star particle indices went from N to N+M-1. However, now it seems that the star particles simply go from 0 to M. Before I go any further, I will mention that the star particles can be differentiated from the dark matter particles by the particle_type data, so we're not totally screwed. The point is that for the time being, hop no longer works. The question is this: what is the best solution for dealing with this? I am more than willing to write something to remap the particle indices, but that doesn't fix the underlying problem. I have already done something similar to this for a different application, so I can do it quickly. OR, something can be done to hop to read in particle type. Clearly, this question is mainly for Stephen, but anyone should feel free to weigh in. What is the best solution? Britton

5 7

nickel instruction for using parallel HOP (not on Kraken)
by Stephen Skory 02 Dec '09

02 Dec '09

Hi all, Matt asked me to outline how to get parallel HOP (in the hg devel version) working. In easy to follow list format: 0. Build yt like you did already (you're on this list, no?). 1. Get the latest hg, choose the 'yt' branch, which is the default. 2. Install Forthon in the standard python way. http://hifweb.lbl.gov/Forthon/ 3. Go into yt-devel/yt/extensions/kdtree and I think 'make' will work on most machines. If not, let me know. It should make 'fKDpy.so' in that same dir. 4. Go to the top level of yt-devel, and type 'python setup.py install' like usual, but then do 'cp -r yt/extensions/kdtree ../../lib/python2.6/site-packages/yt-version-num-sys-etc/yt/extensions/'. I haven't added the kdtree stuff to setup utils yet, so this step isn't automated. 5. One runs parallel HOP very similarly to old HOP: h = yt.lagos.HaloFinding.parallelHF(pf, threshold=160.0, safety=1.5, \ dm_only=False,resize=True, fancy_padding=True, rearrange=True) - safety - is a padding multiplier that adjusts the size of the padding around each subvolume. 1.5 is usually sufficient, and 1.35 probably is. There's no reason to try going lower from 1.5 unless memory is a concern. - dm_only - I currently have things a little messed up in HaloFinding.py in the dev, so this needs to be set unless you have a ['creation_time'] field. - resize - Load balancing. This is a recursive bisection that subdivides the domains like a kd tree, but it can do arbitrary numbers of cuts, so you can run with any number of tasks. Generally, I turn it on for datasets smaller than 300 Mpc, and for ones larger, the particles are already pretty evenly balanced and it doesn't help. - fancy_padding - Each subvolume has six different faces, and six different values for the padding, one for each face. There is really no reason to turn this off. - rearrange - The fortran kdtree can make an internal copy of the point data which increases the speed of the searches by almost 20%. It however increases the memory by 3 fields, so if memory is a concern, turn this off. 6. The usual halo convenience functions work as well, but the output is a bit different. h.write_out('dist-chain.out') - Just like the normal one. h.write_particle_lists("chain") - One hdf5 file per task, that contains the particle data for all the haloes on that task. h.write_particle_lists_txt("chain") - One text file that lists which tasks own a piece of a halo. Since haloes can exist on more than one task, each halo may have several entries on it's line. 7. There is a rough memory scaling law: it takes 1 MB of RAM per 4,500 particles, with rearrange turned on, regardless of task count. This does not include the yt hierarchy costs, so if you have a dataset like L7, which uses 1GB per task, you will need to take that into account, too. Good luck! _______________________________________________________ sskory(a)physics.ucsd.edu o__ Stephen Skory http://physics.ucsd.edu/~sskory/ _.>/ _Graduate Student ________________________________(_)_\(_)_______________

3 10

particle io issue
by Britton Smith 29 Nov '09

29 Nov '09

Hey everyone, The following error pertains to the yt branch of the hg repo. I'm getting a particle io error when using the various yt halo finders in parallel. It doesn't appear to have anything to do with the halo finders, but I don't know what else uses ParticleIO.py. This only happens when running in parallel. P000 yt INFO 2009-11-27 15:16:59,775 Getting ParticleMassMsun using ParticleIO Setting period equal to 1.000000 Setting period equal to 1.000000 Setting period equal to 1.000000 Traceback (most recent call last): File "do_hop.py", line 5, in <module> h = FOFHaloFinder(pf) File "/Users/britton/Documents/work/yt-hg/yt/lagos/HaloFinding.py", line 1024, in __init__ self._parse_halolist(1.) File "/Users/britton/Documents/work/yt-hg/yt/lagos/HaloFinding.py", line 747, in _parse_halolist this_max_dens = halo.maximum_density_location() File "/Users/britton/Documents/work/yt-hg/yt/lagos/ParallelTools.py", line 130, in single_proc_results return func(self, *args, **kwargs) File "/Users/britton/Documents/work/yt-hg/yt/lagos/HaloFinding.py", line 322, in maximum_density_location return self.center_of_mass() File "/Users/britton/Documents/work/yt-hg/yt/lagos/ParallelTools.py", line 130, in single_proc_results return func(self, *args, **kwargs) File "/Users/britton/Documents/work/yt-hg/yt/lagos/HaloFinding.py", line 306, in center_of_mass pm = self["ParticleMassMsun"] File "/Users/britton/Documents/work/yt-hg/yt/lagos/ParallelTools.py", line 130, in single_proc_results return func(self, *args, **kwargs) File "/Users/britton/Documents/work/yt-hg/yt/lagos/HaloFinding.py", line 136, in __getitem__ return self.data.particles[key][self.indices] File "/Users/britton/Documents/work/yt-hg/yt/lagos/ParticleIO.py", line 46, in __getitem__ self.get_data(key) File "/Users/britton/Documents/work/yt-hg/yt/lagos/ParticleIO.py", line 104, in get_data if len(to_add) != 1: raise KeyError KeyError I checked the contents of to_add, and it was a list with two items (hence the exception): ['particle_mass', 'particle_mass'] I was able to get around this by removing non-unique entries in to_add, but I don't think that really fixes the underlying problem. Anyone have any ideas? Regards, Britton

2 2

Cython 0.12
by Matthew Turk 24 Nov '09

24 Nov '09

Cython 0.12 has been released: http://pypi.python.org/pypi/Cython/0.12 The release notes: http://wiki.cython.org/ReleaseNotes-0.12 are interesting, as they indicate type inference is now available, which should make writing fast extensions much easier. C pointers can also now be sliced and iterated over. -Matt

1 0