yt-users July 2011

yt-users@python.org

22 participants
23 discussions

by Elizabeth Tasker

Hi, Is there any way in yt to get a script to delete (or ignore) all the .yt and .harray files and clear its memory? I run into errors when I'm re-doing a plot or I've re-run the simulation and over-written the last data set I looked at with yt. I have to exit yt, delete the above files by hand and restart, which is a bit of a pain. Elizabeth

9 years, 7 months

Re: [yt-users] Fwd: Re: Problems finding halos

by Miguel Angel González

Hi Bretton, Matt, All your advices came in handy, thank you very much! I already did some trials with the files you (Matt) gave me, and they worked, now I'm doing experiments on other simulations (with particles of course). Hopefully won't be any issues. Greetings

9 years, 8 months

Problem Running yt in Parallel

by meecegre＠msu.edu

Hi everyone, I've come across an error while trying to use yt for parallel analysis on Kraken. Wondering if its something people have seen before. I have my simple script that does slices of a few different quanties. When I ran it with 12 tasks on Kraken, it worked fine for the first four datasets, but crashed during the fifth one. I tried a simpler script that just does a slice of density for that dataset, and it crashed again with the same error: IndexError: arrays used as indices must be of integer (or boolean) type When I ran this code again with only 1 task (still on Kraken, everything else the same), it worked fine. More of the output, as well as my job script and plotting script, are below. Since it works in serial , and works for some data sets, the problem seems to be something to do with parallelism. Is there any sort of simple explanation or something I'm doing wrong? Just wondering if this is something people have come across before. --Greg (Last few lines of output) mg, mc, mv, pos = self.find_max_cell_location(field, finest_levels) mg, mc, mv, pos = self.find_max_cell_location(field, finest_levels) File "/lustre/scratch/proj/sw/yt/current/lib/python2.7/site-packages/yt-2.1stable-py2.7-linux-x86_64.egg/yt/data_objects/object_finding_mixin.py", line 69, in find_max_cell_location File "/lustre/scratch/proj/sw/yt/current/lib/python2.7/site-packages/yt-2.1stable-py2.7-linux-x86_64.egg/yt/data_objects/object_finding_mixin.py", line 69, in find_max_cell_location File "/lustre/scratch/proj/sw/yt/current/lib/python2.7/site-packages/yt-2.1stable-py2.7-linux-x86_64.egg/yt/data_objects/object_finding_mixin.py", line 69, in find_max_cell_location max_grid = self.grids[mg] IndexError: arrays used as indices must be of integer (or boolean) type max_grid = self.grids[mg] max_grid = self.grids[mg] IndexError: arrays used as indices must be of integer (or boolean) type IndexError: arrays used as indices must be of integer (or boolean) type (Job script) #!/bin/bash #PBS -A TG-AST090040 #PBS -l size=12,walltime=2:00:00 #PBS -j oe #PBS -N slicer cd /lustre/scratch/oshea/nested_ics_real1/analysis ls pwd export MPLCONFIGDIR=${PBS_O_WORKDIR}/.matplotlib/ [ ! -d ${MPLCONFIGDIR} ] && mkdir ${MPLCONFIGDIR} module unload PrgEnv-pgi module load PrgEnv-gnu module load yt aprun -n 12 python ./simple_slice.py --parallel (simple_slice.py) from yt.mods import * DD = 40 fileName = "../DD%s/DD%s" % ((str(DD)).zfill(4), (str(DD)).zfill(4)) pf = load(fileName) pc = PlotCollection(pf) pc.add_slice("Density", 0) pc.save("./%s" % str(DD).zfill(4))

9 years, 8 months

stuck script?

by Elizabeth Tasker

Hi, I've written a script to calculate the contours in the potential field but I'm having a couple of problems: Firstly, the script is too slow. It is running for several days and still hasn't completed. I think it does not even reached the contour calculation and is still reading in the data set. It is printing to the screen: yt : [INFO ] 2011-07-28 18:52:23,722 Getting field PotentialField from 1720 yt : [INFO ] 2011-07-28 18:52:51,350 Getting field dx from 1720 yt : [INFO ] 2011-07-28 18:53:00,316 Getting field dy from 1720 yt : [INFO ] 2011-07-28 18:53:05,817 Getting field dz from 1720 yt : [INFO ] 2011-07-28 18:53:18,258 Getting field x from 1720 yt : [INFO ] 2011-07-28 18:54:21,300 Getting field y from 1720 yt : [INFO ] 2011-07-28 18:55:23,956 Getting field PotentialField from 1720 yt : [INFO ] 2011-07-28 18:55:47,502 Getting field dx from 1720 yt : [INFO ] 2011-07-28 18:55:55,878 Getting field dy from 1720 yt : [INFO ] 2011-07-28 18:56:01,247 Getting field dz from 1720 yt : [INFO ] 2011-07-28 18:56:13,819 Getting field x from 1720 yt : [INFO ] 2011-07-28 18:57:00,063 Getting field y from 1720 . . . This looks to me like it is somehow getting stuck on grid number 1720. Is that right? It creates a slice just fine, but that might well avoid that particular grid. Secondly, occasionally when I run I see this error: ---> 52 escapevel = sqrt(2.0)*data["SubtractBackgroundPotential"]/sqrt(fabs(data["SubtractBackgroundPotential"])) 53 return (escapevel) 54 NameError: global name 'sqrt' is not defined I have no idea why it is suddenly complaining about sqrt? Sometimes when I run it is fine and then I'll re-run and it'll hit this error. Is there an import command that might only sometimes get called? I've attached my script below and would appreciate any advice! (I'm about to return from Japan via Seoul so I may have my internet connection knocked out for a couple of days; I'm very sorry if there is a delay before I respond). Thank you, Elizabeth #!/usr/bin/python from yt.mods import * from yt.analysis_modules.level_sets.api import identify_contours import pickle def _DiskRadius(field, data): center = 0.5*(data.pf.domain_right_edge - data.pf.domain_left_edge) DW = data.pf.domain_right_edge - data.pf.domain_left_edge dradius = na.zeros(data["x"].shape, dtype='float64') for i, ax in enumerate('xy'): r = na.abs(data[ax] - center[i]) dradius += na.minimum(r, na.abs(DW[i]-r))**2.0 na.sqrt(dradius, dradius) return dradius add_field("DiskRadius", function=_DiskRadius) def _SubtractBackgroundPotential(field, data): np = 500 source = data.pf.h.disk((16,16,16), (0,0,1), 20, 15.6e-3/pf['kpc']) profile = BinnedProfile1D(source, np, "DiskRadius", 0.1, 18.0, log_space=False) profile.add_fields("PotentialField", weight="CellVolume") # print profile["PotentialField"], profile["DiskRadius"] pot = na.zeros(data["PotentialField"].shape, dtype='float64') for r in range(np): # unlike where, this produces a true/false array of the same size as data index = ((data["DiskRadius"] >= profile["DiskRadius"][r]) & (data["DiskRadius"] < profile["DiskRadius"][r+1])) # oddly, it's ok to then put this straight back into data to pull out the right index. Python magic backpot = profile["PotentialField"][r]+(profile["PotentialField"][r+1]-profile["PotentialField"][r])*(data["DiskRadius"][index]-profile["DiskRadius"][r])/(profile["DiskRadius"][r+1]-profile["DiskRadius"][r]) pot[index] = data["PotentialField"][index]-backpot pot[(pot==0)] = -1.0e-10 return pot add_field("SubtractBackgroundPotential", function=_SubtractBackgroundPotential) def _EscapeVelocity(field, data): escapevel = na.zeros(data["SubtractBackgroundPotential"].shape, dtype="float64") escapevel = sqrt(2.0)*data["SubtractBackgroundPotential"]/sqrt(fabs(data["SubtractBackgroundPotential"])) return (escapevel) add_field("EscapeVelocity", function=_EscapeVelocity) def _NegEscapeVelocity(field, data): # sign flip to allow multi-contour finding schemes work in yt return (-1*data["EscapeVelocity"]) add_field("NegEscapeVelocity", function=_NegEscapeVelocity) # Grab data fn = "GravPotential/DD0301/GT_BTAccel_256AMR4_PeHeat_sf5_SNe_0301" pf = load(fn) dd = pf.h.all_data() min, max = dd.quantities["Extrema"]("NegEscapeVelocity") contouredclouds = dd.extract_connected_sets("NegEscapeVelocity", 12, 15.0, max, log_space=False)

9 years, 8 months

Tiled projections

by Brian Crosby

Hi everyone, I was making some projections using the YT module installed on Nautilus and encountered an odd issue: The projection is broken up and appears to be sort of tiled and jumbled around. You can see the result here: http://galactica.icer.msu.edu/~bcrosby/data/1024box/images/RedshiftOutput00… It looks similar to problems that arise from off-axis projections, but this projections are straight down the axis. I'm puzzled about what would be causing this, as the same script produced normal, smooth images earlier. Here's the plotting call that I used: fn = "RD%04i/RedshiftOutput%04i" %(fnum,fnum) pf = load(fn) pc = PlotCollection(pf, center=[0.5,0.5,0.5]) for axis in range(3): p = pc.add_projection("particle_density",axis) Any clues as to what might be going on? Thanks, Brian

9 years, 8 months

Fwd: Re: Problems finding halos

by Miguel Angel González

Hi Bretton, After I put the command you gave me, this appears: ['0', '(do', 'not', 'modify')'] I guess the reason is what you said, but then I have another problem: I can't find in the documentation any example I can use the HOP with; there are examples of commands to use, but no one says what parameter file are they using, if you can help I'd would be very grateful (I already am actually). Thank you. Miguel

9 years, 8 months

Problems finding halos

by Miguel Angel González

Hi YT user, I'm new using YT, and I'm already having problems when I've tried to use the HOP, when I write the commands in Python, this comes out: >>> from yt.mods import * >>> from yt.analysis_modules.halo_finding.api import * >>> pf = load("data0004") >>> halo_list = HaloFinder(pf) yt INFO 2011-07-21 19:38:42,838 Getting the binary hierarchy yt INFO 2011-07-21 19:38:42,846 Finished with binary hierarchy reading yt INFO 2011-07-21 19:38:42,849 Adding Phi to list of fields Warning: invalid value encountered in sqrt Warning: invalid value encountered in sqrt yt WARNING 2011-07-21 19:38:43,009 No particle_type, no creation_time, so not distinguishing. yt INFO 2011-07-21 19:38:43,013 Getting ParticleMassMsun using ParticleIO Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/usr/local/yt-unknown/src/yt-hg/yt/analysis_modules/halo_finding/halo_objects.py", line 1821, in __init__ self._mpi_allsum((self._data_source["ParticleMassMsun"][select]).sum(dtype='float64')) File "/usr/local/yt-unknown/src/yt-hg/yt/data_objects/data_containers.py", line 279, in __getitem__ self.get_data(key) File "/usr/local/yt-unknown/src/yt-hg/yt/data_objects/data_containers.py", line 1918, in get_data self.particles.get_data(field) File "/usr/local/yt-unknown/src/yt-hg/yt/data_objects/particle_io.py", line 92, in get_data conv_factors) File "/usr/local/yt-unknown/src/yt-hg/yt/frontends/enzo/io.py", line 154, in _read_particles filenames, ids, conv_factors = zip(*sorted(zip(filenames, ids, conv_factors))) ValueError: need more than 0 values to unpack Does anyone know how to fix this trouble? Thanks in advance, Miguel Gonzalez.

9 years, 8 months

YT from 2.1 to 2.2

by Jun-Hwan Choi

Hi YT user, I recently move the stable 2.1 to development 2.2. Then I try to run simple slice script that worked with 2.1, but it crash My simple script is as follow: ------------------------------------- from yt.mods import * from yt.analysis_modules.api import EnzoSimulation import matplotlib.pylab as pylab pf = load("DD0000/DD0000") # create density slices pc = PlotCollection(pf, center=[0.5,0.5,0.5]) pc.add_slice("Density", 0) pc.save("DD0000") -------------------------------------- and the error message is Traceback (most recent call last): File "SliceDenItr.py", line 1, in <module> from yt.mods import * File "/home/jhchoi/common/lib/python2.6/site-packages/yt-2.2dev-py2.6-linux-x86_64.egg/yt/mods.py", line 43, in <module> from yt.utilities.cookbook import Intent File "/home/jhchoi/common/lib/python2.6/site-packages/yt-2.2dev-py2.6-linux-x86_64.egg/yt/utilities/cookbook.py", line 30, in <module> import argparse ImportError: No module named argparse Does anyone know the reason of error and way to fix it? Thanks in advance, Junhwan Choi -- -------------------------------------------------------------- Jun-Hwan Choi, Ph.D. Department of Physics and Astronomy, University of Kentucky Tel: (859) 897-6737 Fax: (859) 323-2846 Email: jhchoi(a)pa.uky.edu URL: http://www.pa.uky.edu/~jhchoi --------------------------------------------------------------

9 years, 8 months

yt installation sans wget

by Elizabeth Tasker

Hi, I'm trying to install yt on the xt4 machine in Tokyo, which obnoxiously refuses to let you use wget. I scp-ed a copy from my desktop to the xt4 and then told the install_script.sh where to find yt so it didn't try and download the packages. I also removed 'done' from each of the packages in src. Unfortunately, I get: gcc -fPIC -fPIC -o bzip2. bzip2.o -L. -lbz2 bzip2.o file not recognized: File format not recognized xt4 does use modules for loading packages, but I don't know if I'm lacking a module or if my self-cleaning is flawed. Elizabeth

9 years, 9 months

derived field and TotalQuantity from two sources possible?

by gso＠physics.ucsd.edu

Hi, I was wondering if it is possible to specify two different sources of input to create a derived field. Currently with a single source, I have a derived field defined as: def _HII_HFraction(field, data): return data["HII_Density"]/(data["HI_Density"]+data["HII_Density"]) add_field("HII_HFraction", function=_HII_HFraction, units=r"\frac{\rho_{HII}}{\rho_H}") And I can access the total quantity in parallel pf=load(file) dd = pf.h.all_data() dd.quantities["TotalQuantity"]("HII_HFraction")[0] But I'm dealing with pf1=load(file1) pf2=load(file2) pf1 has the HI_Density field data and pf2 has the HII_Density field data. Is there a way to create a derived field and use TotalQuantity to operate on the data in parallel? From G.S. PS. I guess my alternative is to glue the two HDF5 files into one, but want to avoid it if possible.

9 years, 9 months

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

yt-users July 2011