Mailman 3 February 2012 - yt-users

Enzo cic_particle_* error
by Casey W. Stark Feb. 27, 2012

Feb. 27, 2012

Hi all. I'm trying to make a covering grid of the full domain of an enzo (cosmological) simulation, getting the dark matter density and velocity fields. This works fine with the gas: In [1]: cg = pf.h.covering_grid(0, [0,0,0], [256,256,256], fields=["Density", "x-velocity", "y-velocity", "z-velocity"]) But then I try the DM fields: In [2]: cg = pf.h.covering_grid(0, [0,0,0], [256,256,256], fields=["Dark_Matter_Density", "cic_particle_velocity_x", "cic_particle_velocity_y", "cic_particle_velocity_z"]) ... ipython cruft ... /project/projectdirs/nyx/yt-installs/yt-hopper/src/yt-hg/yt/data_objects/data_containers.pyc in __init__(self, level, left_edge, dims, fields, pf, num_ghost_zones, use_pbar, **kwargs) 3342 self.domain_width = na.rint((self.pf.domain_right_edge - 3343 self.pf.domain_left_edge)/self.dds).astype('int64') -> 3344 self._refresh_data() 3345 3346 def _get_list_of_grids(self, buffer = 0.0): /project/projectdirs/nyx/yt-installs/yt-hopper/src/yt-hg/yt/data_objects/data_containers.pyc in _refresh_data(self) 3359 3360 def _refresh_data(self): -> 3361 AMR3DData._refresh_data(self) 3362 self['dx'] = self.dds[0] * na.ones(self.ActiveDimensions, dtype='float64') 3363 self['dy'] = self.dds[1] * na.ones(self.ActiveDimensions, dtype='float64') /project/projectdirs/nyx/yt-installs/yt-hopper/src/yt-hg/yt/data_objects/data_containers.pyc in _refresh_data(self) 307 """ 308 self.clear_data() --> 309 self.get_data() 310 311 def keys(self): /project/projectdirs/nyx/yt-installs/yt-hopper/src/yt-hg/yt/data_objects/data_containers.pyc in get_data(self, fields) 3377 try: 3378 #print "Generating", field -> 3379 self._generate_field(field) 3380 continue 3381 except NeedsOriginalGrid, ngt_exception: /project/projectdirs/nyx/yt-installs/yt-hopper/src/yt-hg/yt/data_objects/data_containers.pyc in _generate_field(self, field) 3404 # First we check the validator; this might even raise! 3405 self.pf.field_info[field].check_available(self) -> 3406 self[field] = self.pf.field_info[field](self) 3407 else: # Can't find the field, try as it might 3408 raise KeyError(field) /project/projectdirs/nyx/yt-installs/yt-hopper/src/yt-hg/yt/data_objects/field_info_container.pyc in __call__(self, data) 314 ii = self.check_available(data) 315 original_fields = data.keys() # Copy --> 316 dd = self._function(self, data) 317 dd *= self._convert_function(data) 318 for field_name in data.keys(): /project/projectdirs/nyx/yt-installs/yt-hopper/src/yt-hg/yt/frontends/enzo/fields.pyc in _cic_particle_field(field, data) 332 particle_field = field.name[4:] 333 top = na.zeros(data.ActiveDimensions, dtype='float32') --> 334 if data.NumberOfParticles == 0: return top 335 particle_field_data = data[particle_field] * data['particle_mass'] 336 amr_utils.CICDeposit_3(data["particle_position_x"].astype(na.float64), AttributeError: 'AMRCoveringGrid' object has no attribute 'NumberOfParticles' Are you not supposed to use a covering grid with these fields or am I creating the covering grid incorrectly? I get reasonable looking plots if I slice/project cic_particle_velocity_x, so I believe it's working otherwise. Best, Casey

2 5

Hop in parallel?
by Michael Kuhlen Feb. 24, 2012

Feb. 24, 2012

Is running Hop in parallel currently broken? I get way fewer halos in my catalog when I run Hop with 8 processors than with 1. Am I doing something wrong? This is my script: $ cat find_halos.py from yt.mods import * pf = load('RD0004/RedshiftOutput0004') halo_list = HaloFinder(pf, padding=0.02) halo_list.write_out('RD0004/RedshiftOutput0004_Hop.out') When I run 'python find_halos.py' I get 3716 halos, but when I run it with 'mpiexec -np 8 python find_halos.py --parallel' I only get 244. This is independent of the value of padding used. Note that the parallel catalog is a subset of the serial one. The problem occurs within HaloFinder, not during the write_out stage, since len(halo_list) = 242 on all 8 processors. Mike -- ********************************************************************* * * * Dr. Michael Kuhlen Theoretical Astrophysics Center * * email: mqk(a)astro.berkeley.edu UC Berkeley * * cell phone: (831) 588-1468 B-116 Hearst Field Annex # 3411 * * skype username: mikekuhlen Berkeley, CA 94720 * * * *********************************************************************

5 12

memory usage of HOP halo finder
by Michael Kuhlen Feb. 24, 2012

Feb. 24, 2012

I have a question about memory usage of yt's HOP HaloFinder. I have a N=256^3 DM only Enzo simulation that I ran with a 512^3 root grid and fairly aggressive DM refinement with MaximumRefinementLevel=7. Although the run only has 256^3 particles, the AMR has resulted in 163,951 grids and more than 1.5e9 grid cells. Running HOP like so halo_list = HaloFinder(pf) I'm finding that yt is using around 14GB of memory during the particle reading (prior to actually starting the HOP process), which is way out of proportion to the relatively small number of particles. It seems that the memory usage is being driven by the huge number of grids rather than the number of particles. I've traced the memory increase to the calculation of the total mass, specifically to this line: total_mass = self.comm.mpi_allreduce(self._data_source["ParticleMassMsun"].sum(dtype='float64'), op='sum') and again further down: sub_mass = self._data_source["ParticleMassMsun"].sum(dtype='float64') When I specify the total mass as a keyword and comment out the sub_mass calculation (forcing sub_mass = total_mass), then the memory usage remains small. So there's something about the summing up that is leaking memory. Can anyone here shed any light on this puzzling memory hunger? Mike -- ********************************************************************* * * * Dr. Michael Kuhlen Theoretical Astrophysics Center * * email: mqk(a)astro.berkeley.edu UC Berkeley * * cell phone: (831) 588-1468 B-116 Hearst Field Annex # 3411 * * skype username: mikekuhlen Berkeley, CA 94720 * * * *********************************************************************

3 9

yt installation problem in fedora 14
by Prakash Sarkar Feb. 24, 2012

Feb. 24, 2012

While Installing yt through installation script it fails while installing numpy-1.6.1 in fedora 14. the output of the script is shown below [root@shashank yt]# bash install_script.sh ======================================================================== Hi there! This is the YT installation script. We're going to download some stuff and install it to create a self-contained, isolated environment for YT to run within. Inside the installation script you can set a few variables. Here's what they're currently set to -- you can hit Ctrl-C and edit the values in the script if you aren't such a fan. INST_ZLIB = 1 so I will be installing zlib INST_BZLIB = 1 so I will be installing bzlib INST_PNG = 1 so I will be installing libpng INST_FTYPE = 1 so I will be installing freetype2 INST_SQLITE3 = 1 so I will be installing SQLite3 INST_FORTHON = 1 so I will be installing Forthon (for Halo Finding, etc) INST_HG = 1 so I will be installing Mercurial INST_ENZO = 1 so I will be checking out Enzo HDF5_DIR is not set, so I will be installing HDF5 Installation will be to /usr/local/yt/yt-x86_64 and I'll be logging the installation in /usr/local/yt/yt-x86_64/yt_install.log I think that about wraps it up. If you want to continue, hit enter. If you'd rather stop, maybe think things over, even grab a sandwich, hit Ctrl-C. ======================================================================== [hit enter] Awesome! Here we go. Using wget Downloading HDF5 Downloading hdf5-1.8.7.tar.gz from yt-project.org Downloading zlib-1.2.3.tar.bz2 from yt-project.org Downloading bzip2-1.0.5.tar.gz from yt-project.org Downloading libpng-1.2.43.tar.gz from yt-project.org Downloading freetype-2.4.4.tar.gz from yt-project.org Downloading sqlite-autoconf-3070500.tar.gz from yt-project.org Downloading Python-2.7.2.tgz from yt-project.org Downloading numpy-1.6.1.tar.gz from yt-project.org Downloading matplotlib-1.1.0.tar.gz from yt-project.org Downloading mercurial-2.0.tar.gz from yt-project.org Downloading ipython-0.10.tar.gz from yt-project.org Downloading h5py-2.0.1.tar.gz from yt-project.org Downloading Cython-0.15.1.tar.gz from yt-project.org Downloading Forthon-0.8.5.tar.gz from yt-project.org Downloading ext-3.3.2.zip from yt-project.org Downloading ext-slate-110328.zip from yt-project.org Downloading PhiloGL-1.4.2.zip from yt-project.org Installing Mercurial. Setting YT_DIR=/usr/local/yt/yt-x86_64/src/yt-hg/ Installing distribute Installing pip Installing numpy-1.6.1 (arguments: 'numpy-1.6.1') Failure. Check /usr/local/yt/yt-x86_64/yt_install.log. [root@shashank yt]# I have attached the yt_install.log file Please help thanks in advance Prakash

2 5

question about parameter file
by Ali Snedden Feb. 23, 2012

Feb. 23, 2012

Hello, If load in a parameter file ( *pf = load("~/data/Enzo/DD0252/DD0252")*) and access the dictionary via *pf['Density'] *it returns a value of 2.49711e-30. I realize that if I enter *pf['mpc'] *it returns the conversion factor to go from system units to physical units. If yt keeps everything in cgs, then what conversion factor (if any) does *pf['Density'] *return? Thanks! ~ali

2 1

yt on kraken
by Nathan Goldbaum Feb. 23, 2012

Feb. 23, 2012

Hi all, I'm trying to run yt on a login node on kraken. All I'm doing is invoking python from the command line and importing yt. However, python hangs on 'from yt.mods import *'. The python interpreter also takes a very long time (~ a minute or so) to start from the command line. This happens using both the yt/dev and yt/2.3 module. Anyone have any ideas? Nathan Goldbaum Graduate Student Astronomy & Astrophysics, UCSC goldbaum(a)ucolick.org http://www.ucolick.org/~goldbaum

8 15

pf.h.region error
by Elizabeth Tasker Feb. 22, 2012

Feb. 22, 2012

Occasionally, when I try and create a region, I get this error; ValueError Traceback (most recent call last) /work/taskere/orca/yt-2-orca/src/yt-hg/scripts/iyt in <module>() 20 pc = PlotCollection(pf, center=center) 21 ---> 22 reg = pf.h.region(center, [center[0]-0.5*width, center[1]-0.5*width, center[2]-0.5*depth], [center[0]+0.5*width, center[1]+0.5*width, center[2]+0.5*dept h]) 23 24 p = pc.add_projection('Density', 2, data_source=reg) /work/taskere/orca/yt-2-orca/src/yt-hg/yt/data_objects/static_output.pyc in hierarchy(self) 167 raise RuntimeError("You should not instantiate StaticOutput.") 168 self._instantiated_hierarchy = self._hierarchy_class( --> 169 self, data_style=self.data_style) 170 return self._instantiated_hierarchy 171 h = hierarchy # alias /work/taskere/orca/yt-2-orca/src/yt-hg/yt/frontends/enzo/data_structures.pyc in __init__(self, pf, data_style) 160 self.float_type = 'float64' 161 --> 162 AMRHierarchy.__init__(self, pf, data_style) 163 # sync it back 164 self.parameter_file.data_style = self.data_style /work/taskere/orca/yt-2-orca/src/yt-hg/yt/data_objects/hierarchy.pyc in __init__(self, pf, data_style) 68 69 mylog.debug("Parsing hierarchy.") ---> 70 self._parse_hierarchy() 71 72 mylog.debug("Constructing grid objects.") /work/taskere/orca/yt-2-orca/src/yt-hg/yt/frontends/enzo/data_structures.pyc in _parse_hierarchy(self) 226 return line.split()[2:] 227 if os.path.exists(self.hierarchy_filename[:-9] + "harrays"): --> 228 if self._parse_binary_hierarchy(): return 229 t1 = time.time() 230 pattern = r"Pointer: Grid\[(\d*)\]->NextGrid(Next|This)Level = (\d*)\s+$" /work/taskere/orca/yt-2-orca/src/yt-hg/yt/frontends/enzo/data_structures.pyc in _parse_binary_hierarchy(self) 300 except h5py.h5.H5Error: 301 return False --> 302 self.grid_dimensions[:] = f["/ActiveDimensions"][:] 303 self.grid_left_edge[:] = f["/LeftEdges"][:] 304 self.grid_right_edge[:] = f["/RightEdges"][:] ValueError: shape mismatch: objects cannot be broadcast to a single shape WARNING: Failure executing file: <image_object.py> What I don't understand is that it'll happen on one data set, but then the next one (or the one before) in the simulation will be totally fine. My dtdump steps are small too, so I'm confused as to why this would occur. Both "width" and "depth" are neither very small or very large compared to "centre". (In kpc, centre = 16, 22, 16 and width = 4 and depth = 1.) Elizabeth

3 4

width of projection
by Elizabeth Tasker Feb. 21, 2012

Feb. 21, 2012

Hi, Is there a way to control how deep the projection is with add_projection? I'd like to project along only 1kpc of my disk, if that were possible? Elizabeth

3 3

Call for homepage viz
by Matthew Turk Feb. 15, 2012

Feb. 15, 2012

Hi all, I'd like to update the viz on the yt-project.org home page. If you have some yt-created visualization you're particularly proud of, please drop me a line off-list with a reasonably sized image and appropriate credit, and if it fits the feel of the site, I'll put it up in the main rotation. A few of the existing images will get rotated out, too. Thanks, Matt

1 0

Enzo installation problem
by Khan Muhammad Feb. 15, 2012

Feb. 15, 2012

Hi all, I have been trying to install enzo for a cosmological simulation project. I used the instructions given in: http://code.google.com/p/enzo/wiki/EnzoBootCamp install_script.sh ran just fine, I also got a message pointing towards the enzo path /home/md/enzo/yt-x86_64/src/enzo-hg-stable But when I actually check I see that there is no directory named "enzo-hg-stable" in the "src" directory. It would be great if someone could help me in this regard. :) Regards, Khan Muhammad Bin Asad University of Goettingen, Germany

2 3